Archive for December 2011

ASR Recovery must be Activated on First Boot

December 28, 2011

After performing an ASR recovery of a Windows Server 2003 O/S, the first time you attempt to log into the server, you may receive a prompt to re-activate Windows. If you can’t or don’t want to activate Windows right away, you can reset the Windows Activation so that you can at least log into the server.

As part of the Disaster Recovery Planning that I do for my clients, I do a full recovery of their server to a test environment to make sure that the server can be recovered and to document any issues that may arise. That way, in the event of a real emergency, I’ll be prepared to perform the recovery as quickly and as smoothly as possible.

During a recent recovery test for a client, I was restoring a server running a Volume License version of Windows Server 2003 R2 x64 and Exchange 2007. It was running on a Dell PE1950 server and I was restoring it to very dissimilar hardware. The machine I was using was just a desktop PC that had the equivalent ram and hard drive space as the Dell. I was documenting the steps I went through and preparing a checklist of the steps necessary. This unit wasn’t going into production so I wasn’t concerned that the hardware wasn’t up to server specs. But even if I was restoring to new server class hardware, I’m pretty sure I would have run into the same Activation issues. Even restoring to a virtual environment has given me similar results.

In my opinion, if you don’t have a 3rd party backup solution that performs an image based backup of the server, the best and fastest way to get a Windows Server 2003 based O/S back up and running is with an Automated Server Recovery (ASR) restore. I use that to restore the O/S and system files and then use the latest available backup to restore the data. If any applications are installed on the system partition, they usually come along as part of the system files, but they could be restored from backup as well. That’s a lot less work than installing the O/S from scratch, patching and re-installing all the apps.

Running an ASR backup is fairly simple. Open NTBackup and step through the ASR wizard. The only thing that complicates the process these days is that an ASR backup requires a floppy disk – not something you find that often on newer hardware. I use a USB floppy drive for machines that don’t have one. (For example, this Dell PE1950 didn’t). I like to update the ASR backup of each server at least 3 or 4 times a year. Otherwise, you may have to run a lot of updates to get the machine up to the same patch level as your most recent backup.

Anyway, I ran the test ASR restore of this particular server and after it completed, the server rebooted. When the Ctrl-Alt-Del prompt finally came up, I entered the local administrator credentials. Before I got to the desktop, I was presented with a dialogue box that stated that the hardware had changed since Windows was first installed and that I needed to re-activate windows to continue. That wasn’t going to happen, even if I wanted it to. I couldn’t do an internet based activation because the NIC drivers hadn’t been installed yet. I suppose I could have tried a telephone activation, but I didn’t want to replace the original server – I just wanted to test the restore.

This seems like a bit of catch 22 situation. You can’t log into Windows until you re-activate and you can’t re-activate until you log into Windows and configure the NIC. Restarting in safe mode isn’t much help because you can’t install drivers in safe-mode.

The way around this is to reset the Windows Activation – essentially return Windows to an un-activated state. After doing that, you have the traditional 30 days to activate Windows – just like you do with a new install. This is more than enough time for my recovery testing procedure. And in the event of a real emergency, it would get you up and running again and give you enough time to sort out the licensing. Since this particular server was running a volume license copy, it would have been permissible to transfer the license to new hardware. If it was an OEM copy, I would still have 30 days to purchase a new copy of Windows Server and get the machine properly licensed.

The procedure to reset the activation is surprisingly simple.

  1. Reboot the server in safe mode.
  2. Log in with the local administrator credentials.
  3. open a command prompt
  4. enter the command – “rundll32.exe syssetup,SetupOobeBnk” without the quotes to reset Windows activation.
  5. Reboot.

When the server reboots, Windows will be un-activated. You will now have 30 days to activate, just like you would with a fresh install. You can log in, install drivers and continue to restore the data.

This trick gave me enough time to complete my recovery testing. If this was a real Disaster Recovery instead of a test and I was going to put the recovered server into production, I would still need to activate Windows with a valid license. But I could do it at a more convenient time after I got the server up and running and the users back to work.

Installing new network gear? Update that firmware first!

December 13, 2011

One of my clients had a problem recently that turned out to be caused by a new network switch that was running old firmware. I’ll be checking for firmware updates of a lot more equipment from now on before I install it. Here’s why….

The client bought a new Dell PowerConnect 5324 Smart Switch as part of a new iSCSI SAN that they were implementing. They ran through the initial configuration to assign it an IP address and change the default password and then installed it in their server rack and started using it. The switch was only connected to the iSCSI network so they didn’t do any further configuration.

Shortly after getting everything configured and transferring data to the SAN, they started to experience intermittent short-term loss of connectivity to the iSCSI targets. Looking at the event logs of the Windows servers connected to the SAN showed the network cards on the iSCSI subnet were losing connectivity to the network. The error stated the the Network Link was down. Checking the event logs of the new switch revealed a Fatal Error had occurred and the switch had rebooted.

An internet search of the error message turned up that this was a known issue that had been corrected in version of the firmware. The switch was running version – one version older. They have another switch of the same model that was at least a year old. Interestingly, it was also running version of the firmware.

Checking the Dell support web site, it turns out that at that point in time, version of the firmware was available. So this brand new switch had arrived with firmware that appeared to be 10 versions old. It was a little surprising to me (and the client) that a brand new switch was running such old firmware.

Now to be fair, if you read the 255 page Dell manual that comes as a .pdf file on the CD that accompanies the switch, it clearly states that you should update the firmware before installing the switch. However, in my opinion, a sticker on the switch that emphasizes this important point would be very helpful.

Updating firmware is something that is often recommended by the manufacturers of servers and their components like Raid Cards, NICS, etc. I know Dell and HP release update disks that will help you with this. And I’ve also updated the firmware of routers and firewalls to correct issues and add features. But updating the firmware of a switch is not something that occurred to either my client or myself as being necessary. That is probably partly because they weren’t really using the smart features of those particular switches. The need to do that became painfully obvious to my client and it will be something I will add to my own practice when I’m installing new systems equipment.

It’s worth mentioning that PC and motherboard manufacturers seem less eager for you to update the firmware or bios of their products. Most of them have warnings on the support sections that basically state “don’t upgrade the bios if the PC is working”. I suspect those instructions are aimed at enthusiasts rather than IT professionals.

Updating the firmware on one of these switches is definitely worth doing before you get the unit installed in a rack. Other than the fact that the latest firmware can prevent some potential issues, it’s been my observation that these smart switches require a serial connection to the switch in order to access the command line interface to upload and select the new firmware. But a notebook with a serial port that you can take to the rack where the switch is installed is becoming increasingly hard to find. And getting to the serial port of  a switch in a rack can be physically challenging depending on where the port is located. At least if the switch is out of the rack, you can take it to a PC that has a serial port in order to do the upgrade.

I’ll do another post on what I went through to update the firmware on this and a couple of other switches at this client.