Installing new network gear? Update that firmware first!

One of my clients had a problem recently that turned out to be caused by a new network switch that was running old firmware. I’ll be checking for firmware updates of a lot more equipment from now on before I install it. Here’s why….

The client bought a new Dell PowerConnect 5324 Smart Switch as part of a new iSCSI SAN that they were implementing. They ran through the initial configuration to assign it an IP address and change the default password and then installed it in their server rack and started using it. The switch was only connected to the iSCSI network so they didn’t do any further configuration.

Shortly after getting everything configured and transferring data to the SAN, they started to experience intermittent short-term loss of connectivity to the iSCSI targets. Looking at the event logs of the Windows servers connected to the SAN showed the network cards on the iSCSI subnet were losing connectivity to the network. The error stated the the Network Link was down. Checking the event logs of the new switch revealed a Fatal Error had occurred and the switch had rebooted.

An internet search of the error message turned up that this was a known issue that had been corrected in version of the firmware. The switch was running version – one version older. They have another switch of the same model that was at least a year old. Interestingly, it was also running version of the firmware.

Checking the Dell support web site, it turns out that at that point in time, version of the firmware was available. So this brand new switch had arrived with firmware that appeared to be 10 versions old. It was a little surprising to me (and the client) that a brand new switch was running such old firmware.

Now to be fair, if you read the 255 page Dell manual that comes as a .pdf file on the CD that accompanies the switch, it clearly states that you should update the firmware before installing the switch. However, in my opinion, a sticker on the switch that emphasizes this important point would be very helpful.

Updating firmware is something that is often recommended by the manufacturers of servers and their components like Raid Cards, NICS, etc. I know Dell and HP release update disks that will help you with this. And I’ve also updated the firmware of routers and firewalls to correct issues and add features. But updating the firmware of a switch is not something that occurred to either my client or myself as being necessary. That is probably partly because they weren’t really using the smart features of those particular switches. The need to do that became painfully obvious to my client and it will be something I will add to my own practice when I’m installing new systems equipment.

It’s worth mentioning that PC and motherboard manufacturers seem less eager for you to update the firmware or bios of their products. Most of them have warnings on the support sections that basically state “don’t upgrade the bios if the PC is working”. I suspect those instructions are aimed at enthusiasts rather than IT professionals.

Updating the firmware on one of these switches is definitely worth doing before you get the unit installed in a rack. Other than the fact that the latest firmware can prevent some potential issues, it’s been my observation that these smart switches require a serial connection to the switch in order to access the command line interface to upload and select the new firmware. But a notebook with a serial port that you can take to the rack where the switch is installed is becoming increasingly hard to find. And getting to the serial port of  a switch in a rack can be physically challenging depending on where the port is located. At least if the switch is out of the rack, you can take it to a PC that has a serial port in order to do the upgrade.

I’ll do another post on what I went through to update the firmware on this and a couple of other switches at this client.

Explore posts in the same categories: Computers and Internet

%d bloggers like this: