How to unbrick a Netgear switch

Searching the internet for bricked Netgear devices brings up quite a lot of results. This can happen due to misconfiguration, bugs or broken firmware updates. The good thing is, many of them have a UART interface onboard, which allows to connect to the device via a USB-TTL adapter and get access via serial console. In my case this allowed to fix a Negear GS110TP, but the following steps should work for different devices.

1. Open device

switch opened switch UART pin header

Typically you need to losen only some screws to open the case. Leave the device powered off.

2. Find the UART pin header

This can be the hardest part, since different Netgear devices use different pin header layouts. For UART you need three lines: RX (receive), TX (transmit) and GND (ground). GND of your USB-TTL adapter is connected with GND of the Netgear device pin header. RX of your adapter is connected with TX on the device and TX of your adapter is connected with RX on the device (so RX and TX are crossed). Now how to find out what is RX, TX and GND?

In the best case, the PIN header is marked and only has 4 pins. This was the case for the GS110TP and can be seen from the images. The (unused) pin with the small white dot (see smaller image, right hand side of the PIN) is VCC. You need to make sure, that you don't use that PIN, since this might damage the USB-TTL adapter and/or the Netgear device if you connect both VCC (the one from the USB TTL adapter and the one from the pin header). The PIN next to it (second one from right) is TX. This one is connected with the USB-TTL adapter's RX line (white cable). The other one is RX, which is connected with the USB-TTL adapter's TX line (green). The last one is GND (black) which can be connected directly.

In the worst case, the header is not marked and has more than 4 PINs. In that case there are YouTube videos outside, which demonstrate how to find the right PINs. I trust the reader to check one of them.

3. Connect the USB-TTL adapter and run putty/minicom/...

Run dmesg -w in a terminal window. Then connect the adapter. You should see in the dmesg output, that the adapter was connected. It should show up as /dev/ttyUSB0 or something similar.

To get output and to be able to type in commands, you need a terminal emulator which supports serial lines like putty. Use the following connection parameter:

9600 baud, 8-N-1

4. Power on the device

When you power on your switch, you should see an output similar to that one:

CFE-NTSW-B5.1.0.2 for GS1XXT (32bit,SP,BE,MIPS)
Build Date: Wed Aug 11 18:05:01 IST 2010 (yrdreddy@lc-hyd-001)
Copyright (C) 2000,2001,2002,2003,2004,2005 Broadcom Corporation.

Initializing Arena.
Initializing Devices.
Board : GS110TP
CPU type 0x29050: 200MHz
Total memory: 0x4000000 bytes (64MB)

Total memory used by CFE:  0x83EA0000 - 0x83FFF720 (1439520)
Initialized Data:          0x83EEA0D0 - 0x83EEB260 (4496)
BSS Area:                  0x83EEB260 - 0x83EFD720 (74944)
Local Heap:                0x83EFD720 - 0x83FFD720 (1048576)
Stack Area:                0x83FFD720 - 0x83FFF720 (8192)
Text (code) segment:       0x83EA0000 - 0x83EE97F7 (301047)
Boot area (physical):      0x03E5F000 - 0x03E9F000
Relocation Factor:         I:E42A0000 - D:E42A0000
Compression Supported:     7zip

Troubleshooting: If you don't see anything, you likely have cabled the device incorrectly. If you see weird characters, likely the baud rate is not correct. Try to increase the baud rate step by step and check the output (power cycle the switch). Weird characters could also show up if other serial port parameter are incorrect, like hardware flow control being enabled. Check your settings!

Assuming everything works, you can press CTRL + C early in the boot process. You should then end up in the bootloader prompt and see something like this:

Automatic startup canceled via Ctrl-C
CFE>

If you type help here, you will get a list of possible commands:

  CFE> help
  Available commands:

  copydisk            Copy a remote disk image to a local disk device via TFTP
  test fatfs          Do a FAT file system test
  test disk           Do a disk test, read/write sectors on the disk
  @                   Boot VxWorks
  c                   Change the VxWorks boot string
  p                   Parse and display VxWorks boot string
  M                   Show or set Ethernet MAC address
  ptable save         Save flash partition table.
  ptable clear        Clear flash partition table.
  ptable del          Delete flash partition table.
  ptable add          Delete flash partition.
  ptable show         Show flash partition table.
  etx                 start eth loopback application
  lbtest              start eth loopback application
  ethstat             eth drv stats
  cpw                 write p0 register
  ttlb                test TLB
  tdc                 test data cache
  flprobe             Probe flash
  erase               Serial Flash erase one sector
  flush               Serial Flash read cache flush
  sflashw             Serial Flash write
  sflashr             Serial Flash read
  miiw                MII write
  miir                MII read
  scr                 Schan write
  scw                 Schan write
  envdev              set environment device
  reset               Reset the system.
  nvram import        Import nvram variables from standard environment.
  nvram erase         Delete all nvram variables.
  nvram commit        Commit nvram variable bindings.
  nvram show          Show all nvram variables.
  nvram unset         Delete an nvram variable.
  nvram set           Set the value of an nvram variable.
  nvram get           Get the value of an nvram variable.
  set console         Change the active console device
  sleep               Wait for some period of time
  loop                Loop a command
  flash               Update a flash memory device
  memtest             Test memory.
  f                   Fill contents of memory.
  e                   Modify contents of memory.
  d                   Dump memory.
  u                   Disassemble instructions.
  reserve             Mark a region of memory as reserved
  autoboot            Automatic system bootstrap.
  batch               Load a batch file into memory and execute it
  go                  Start a previously loaded program.
  boot                Load an executable file into memory and execute it
  load                Load an executable file into memory without executing it
  save                Save a region of memory to a remote file via TFTP
  ttcp                TCP test command.
  tcp constest        tcp console test.
  tcp listen          port listener.
  tcp connect         TCP connection test.
  rlogin              mini rlogin client.
  ping                Ping a remote IP host.
  arp                 Display or modify the ARP Table
  ifconfig            Configure the Ethernet interface
  device              Erase the flash devices
  show flash          Display information about a flash device.
  show boot           Display boot block from device,
  show config         Dump CP0 configuration registers
  show heap           Display information about CFE's heap
  show memory         Display the system physical memory map.
  show devices        Display information about the installed devices.
  clearenv            Clear environment variables.
  unsetenv            Delete an environment variable.
  printenv            Display the environment variables
  setenv              Set an environment variable.
  help                Obtain help for CFE commands

  For more information about a command, enter 'help command-name'
  *** command status = 0

If you waited too long with pressing CTRL + C, the bootloader will pass and the device will boot. So you won't end up in the bootloader. This should look similar to that:

Loader:elf Filesys:raw Dev:flash0.os1 File: Options:(null)
Loading:
Validating the code file..
Flash stk image is 4122714 bytes,  CRC 00002590
0x80041000/15741096 0x80f440a8/3023552 0xa0001000/262144 Entry at 0x80041000
Starting program at 0x80041000

16x5x SERIAL init - dev: b8000300.1
 in flashDrvLibInit 47xx
Vpd crc valid.
IFP: 0x80eed01c, next: 0x81225fc0
IFP: 0x81225fc0, next: 0x00000000

Vpd crc valid.
ramfs crc OK (0x68b3e6d)
..FastPATH software Version 5.4.2.11 Build Date: Mon Jan 20 08:12:28 EST 2014
Starting fpmain

 i am in ecosbde_createICS unit 0: Dev 0xc312, Rev 0x11, Chip BCM53312_B0, Driver BCM53314_A0
GPIO Board ID = 1
SOC unit 0 attached to PCI device BCM53312_B0

Tuning MMU with 4096 cells and 4 CoS queues for 13 ports of which 12 are Ethernet ports
Fan in is targeted at 4 while the over subscription is set to 8

calling init_bcm_53312, board_id=110 power=60 gpio_in boardid bits=1
.started!
[osapiPipeCreate-61]
[osapiPipeOpen-67]
..
(Unit 1)>

Applying configuration, please wait ...
FastPATH Debug >Sending PoE Init TRAP


Applying Global configuration, please wait ...

Applying Interface configuration, please wait ..

Now, if the device has booted, there is often another hidden menu. For the Netgear device I tested, help plus Enter resulted in the following output (a list of commands you can run):

FastPATH Debug >help

Available Commands:
        0x80077ba8      cablediag
        0x803bda2c      cat
        0x803bb314      changeMallocDebug
        0x8036578c      cliInfoDump
        0x8029e14c      cnfgrDump
        0x802f2560      configClear
        0x802e79ec      configDump
        0x802f2588      configSave
        0x803b95dc      crashPrint
        0x8005a720      debugPolicyGroup
        0x80054b44      debugPolicyTable
        0x803cc5ac      debugTftp
        0x803bd388      del
        0x8004d1a4      dev
        0x803bd1f4      dir
        0x802b6938      driv
        0x803b6b0c      dtlDebug
        0x803b7dac      dtlEndStats
        0x803b9250      dtlMacToPortShow
        0x803b6710      dtlNetDebugSet
        0x8049b420      dumpFdbStats
        0x803c6540      dumpMemory
        0x803c2e68      ecos_net_stats
        0x80046cb8      hapiBroadDebugPkt
        0x80046cec      hapiBroadDebugPktFilterGet
        0x80046d1c      hapiBroadDebugPktFilterSet
        0x80074424      hapiBroadPolicyDebug
        0x802f5a4c      if
        0x802f5a4c      ifconfig
        0x802d38a0      logClear
        0x802cf1d4      logConsole
        0x803c6b30      logError
        0x802d3b78      logShow
        0x80306508      mbufFree
        0x80305720      mbufHistoryClear
        0x80305698      mbufHistoryDelete
        0x80305798      mbufHistoryDump
        0x8030557c      mbufHistoryInit
        0x803058e4      mbufShow
        0x803ba72c      memCheck
        0x803bc0dc      memShow
        0x8004e8f8      mmuConfig
        0x80051e00      mmuCount
        0x80050ff4      mmuState
        0x803c672c      modMemory
        0x803c0b04      msgQ
        0x803c0ef4      msgQprint
        0x803c0bac      msgQshow
        0x802d7f8c      nimDebugDump
        0x802d8f54      nimPortDump
        0x803baf18      osapiDebugMallocDetail
        0x803bae34      osapiDebugMallocDetailEnable
        0x803ba468      osapiDebugMallocSummary
        0x803bad0c      osapiDebugMemoryInfo
        0x803baadc      osapiDebugMemoryStats
        0x803c0b04      osapiDebugMsgQueuePrint
        0x803ba468      osapiMemShow
        0x803c0bac      osapiMsgQueueShow
        0x803ba244      osapiRedir
        0x803c6408      osapiTaskShow
        0x8004d1d0      phyDump
        0x800be064      phyget
        0x800be464      physet
        0x802ea874      poeCfgDump
        0x80074424      policy
        0x80073f04      policyTable
        0x8029b4c8      poolShow
        0x80046310      reboot
        0x8004e088      regDump
        0x803c2e30      routePrint
        0x800498f0      rxShow
        0x802f2494      setdhcp
        0x80286e74      shadowDump
        0x803c6ad4      showCS
        0x8023f9c4      soc_property_get
        0x803000b0      sysShow
        0x803bc1f4      taskShow
        0x803bb8d4      upTime

Please see the source code for parameter lists.

Now that you've seen the two basic menus (let's call them bootloader and main menu), you could have a look around. How to proceed now depends on what is wrong with your device. In my case, it was not clear what the actual problem is. I first assumed a faulty firmware update but later it turned out that the device configuration was broken and even a hardware reset didn't reset the configuration. Therefore, I demonstrate below how to flash the firmware and how to erase a broken configuration via bootloader and main menu commands.

5. Flash firmware image via bootloader commands

The flashing is done over network via TFTP. So you need to setup a TFTP daemon on your machine first. The easiest way is to use python ptftpd. You can simply install the package via pip:

$ pip install ptftpd

Assign a static IP to your ethernet NIC e.g. 10.0.0.1/24 to eth0 (via NetworkManager, ifconfig, ip or whatever you want to use). Once done, you can run the server via the following command:

# mkdir /tmp/tftp
# cd /tmp/tftp
# ptftpd eth0

The TFTP server should then listen on port 69/udp for incoming connections. Note, that you need to run this as root, since port 69/udp is a privileged port. Now download the firmware file you want to flash and put it into the TFTP service root directory, which is /tmp/tftp in our case.

Connect the switch with your machine via ethernet cable. Boot up the switch into bootloader menu (see instructions above). Most Netgear devices seem to have two flash partitions. We will flash the first one, since the bootloader will try this partition first. The flash procedure now looks like that:

  CFE> ifconfig -addr=10.0.0.2 -gw=10.0.0.1 -mask=255.255.255.0 eth0
  Device eth0:  hwaddr 00-00-C0-FF-EE-00, ipaddr 10.0.0.2, mask 255.255.255.0
          gateway 10.0.0.1, nameserver not set
  *** command status = 0
  CFE> flash 10.0.0.1:GS108Tv2_GS110TP_V5.4.2.33.stk flash0.os
  Reading 10.0.0.1:GS108Tv2_GS110TP_V5.4.2.33.stk: Done. 4196342 bytes read
  Flash stk image is 4196278 bytes,  CRC 00006A5F
  Programming...done. 4196342 bytes written
  nvram_commit: will write 14c bytes from 83f1f9c0
   result 14c (332)
  *** command status = 0
  CFE>

We manually assign 10.0.0.2/24 to the switch (first command). Then you flash by running the second command. Here the GS108Tv2_GS110TP_V5.4.2.33.stk is the name of the firmware file you put into /tmp/test and flash0.os is the first flash partition. Once the second command finished, the command status should be 0. You have now successfully flashed the firmware to partition. If you power on and off the device it should properly boot the new firmware.

5. Erasing broken configuration via main menu commands

This one is very easy. Just boot the switch up completly and then run the configClear command in the main menu. The output should look like that:

FastPATH Debug >configClear

 Executing - configClear


Reference platform resetting ...

If you don't have a configClear command, you could type help and watch for a similar command.

Once you are finished, poweroff the switch and unplug it, then remove the USB-TTL adapter. That's it!

comments (0) - add comment

No comments so far, leave one?