Floyd Mayweather may be the pound-for-pound best fighter in boxing, but my apartment is likely the bit-for-byte champ when it comes to having the most electronically decorated dwelling space within my complex. With a staggering amount of electricity flowing through the veins that power this little corner of my world, it's no surprise that I have to occasionally make some sacrifices to keep things working. Sometimes it's trying out new things, other times I roll the dice and break something, and more frequently than not, I find myself just purchasing new toys. When it comes to these activities, it's always a game of chance and sometimes like a boxing match between me and my digital opponents in the other room. Sometimes I'm the recipient of that right hook, other times I'm the deliverer. So, this story is going to cover my last two weekends in which I made some upgrades to the servers I have running around here. You see, I'm a pretty avid user of Ubuntu Linux. Sure, it doesn't have the enterprise support distributions like Red Hat has, but it's easy to install, has a strong community for support, and I love the Debian-style packaging system. I've migrated all my servers from Gentoo to Ubuntu over the last year with good success and the amount of maintenance is minimal, most of the time.
I've been eagerly awaiting the release of the new version of Ubuntu, code-named Feisty Fawn. Scheduled for an April 2007 release, I was just chomping at the bit to hear the announcement. As it turns out, I had forgotten about its scheduled release date. Luckily for me, my friend Ryan notified me promptly when the bits were available on the mirrors. This was good news and bad news. The good news was that the release I'd been awaiting for was available, including the Apache 2.2 package that wasn't in Edgy, but the bad news was I had some work ahead of me in server upgrades.
Most people would wait a few weeks for the initial bugs to be worked out rather than facing the inevitable problems that typically occur if you are quick to upgrade. So rather than jumping the gun right away, I went ahead and upgraded a few virtual machines that were running the desktop version of Ubuntu Edgy 6.10. The upgrades were flawless. Just a few button clicks and I had a fully-functioning Feisty Fawn.
Feisty Fawn Upgrade – First attempt not so good
With my initial upgrades tests being successful, I decided to toss myself into the ring with my file server. Out of all my systems, this was the one likely to cause the least amount of pain if it were unavailable. In accordance with the instructions on the Ubuntu website, I executed the following two commands.
- Install the update manager
sudo apt-get install update-manager-core - Launch the upgrade tool
sudo do-release-upgrade
Seemed pretty harmless, so I gave it a go. My first attempt, like many others based on threads in the Ubuntu Forums, was thwarted by the infamous message that informed me that there wasn't enough disk space on my /boot partition. My boot partition is only 64MB. After further reading, I ended up having to remove some of the previous kernels to free enough space. apt-get --purge remove linux-kernel-2.6-15-23-server
After I removed all but one kernel and re-ran the sudo do-release-upgrade command, the upgrade commenced. As the bits were being updated, I was reminded of something I hadn't considered until it was too late. Yep, that whole Murphy's Law thing. What I forgot is that I was running my disks in a RAID1 configuration.
Half way through the install, it was asking me questions about how to proceed with the upgrade to the RAID configuration. This kind of thing never fails for me. Since this was the first time I've ever encountered this message, and quite frankly forgot how the partitions were setup, I made some adjustments that in hindsight, probably sealed my fate with this upgrade.
I had a number of partitions all running in a RAID1 configuration, including the root partition. The prompt asked me something relating to what md devices I wanted to auto-start and which one was my root partition. Let's just say I chose wrong. After the upgrade finished, I was asked to restart. Naturally, I clicked ok. Upon reboot, Feisty was well, feisty and basically informed me that it could not find my root partition and dropped me to some shell. I won't go into the details, let's just say it resulted in the following:
- Booting the system and mounting all the drives so I could back up the files to an external USB drive
- Re-installing the entire operating system from scratch
- Restoring all the backup files and reconfiguring all the services
After a very long night that quickly turned to morning with birds chirping, at approximately 7am the next morning, Feisty Fawn was powering my file server and everything seemed to be working. Whew, what a mess. Needless to say, I wasn't so excited about upgrading the three other servers.
Feisty Fawn Upgrade Part II – Second attempt much better
So I was already skeptical about upgrading the remaining servers at all, being that my first attempt cost me an endless night of pain. Not only that, but all my servers are in a RAID1 configuration, so I figured I'd run into the same problem. I wasn't about to let the computers get the upper hand so I went for it, but this time I trained harder and took precautionary measures. I did my homework on the forums to read about other users who were experiencing similar upgrade problems. I also backed up the entire filesystem to my external drive ahead of time.
At approximately 4:30pm on Saturday, April 28, 2007 I applied my experience from the previous installation and proceeded to begin the upgrade procedure on the router. About 1 hour later, the upgrade was complete and I had answered the questions posed by the installer differently than I did the first time, hoping for the best. Upon restart, all went well.
I moved onto system 3 of 4, the web server. This was probably the riskiest upgrade since failure met that I would have to reconstruct the most customized server. Once again, I proceeded and about 30 minutes later, a restart landed me in a good mood again. It all just worked, no glitches.
Now with only one server remaining, the mail server, I've decided to exercise some patience. This server runs Zimbra which handles all mail for a number of domains I host. Right now, they have a number of installer packages for various distributions, but there isn't a Feisty package yet. According to the Zimbra Forums, they should have one sometime around their 5.0 release. I'll wait until then unless others report success.
For now, I'm just glad I got this far. But just when I thought things were going well, technology was applying the infamous rope-a-dope technique.
Death of a gaming system
A number of years ago I built a custom system from bare-bones parts. It was more of an exercise in learning how to construct a system from scratch than anything else. At the time, I figured why not mod the system and as you can see, I did just that. Nothing special, just a laser-cut side panel with a few florescent lights on the inside. Prior to ordering all the parts, I did a fair amount of research to make sure I was buying quality components. Unlike most of my systems, this one worked flawlessly for at least about 5 years....until the other night.
I had just rebuilt the system about a week ago just to get a clean system on it again. Four hours, numerous Windows XP updates, and a few VMWare virtual machines later, I had a solid system again. After booting the system yesterday for some miscellaneous tasks, I left the room. Upon coming back the system was off. Strange. I turned it on again and it made it to the initial Windows XP boot screen then just powered off. Sigh.
Sounds like a hardware problem to me, time to troubleshoot. I won't bore you with the details, but let's just say I spent about 2 hours narrowing down the problem with little success. Since I didn't have extra hardware laying around (e.g., motherboard, power supply, memory), I wasn't able to do too much, but remove any parts that could be causing the problem. The symptom was that when I booted the system, all that happened is that the motherboard and power/HD LEDs started blinking. No POST, no monitor output, just the mysterious blinking lights. It was an ABIT motherboard, but their website forums didn't help very much.
At this point in time, I had a decision to make. I knew one of the following components was causing the problem:- Motherboard
- Memory
- Power supply
- Processor
My bet was the motherboard or power supply, but I wasn't in the mood to pursue weeks of buying parts on Newegg. Around 11pm on Saturday and after about an hour of troubleshooting with my friend Kevin, I decided it just wasn't worth it anymore. This is the point were I typed "www.dell.com" into my Safari browser.
That's right, your eyes are not fooling you. Dude, I'm getting a Dell. I've had a fairly good success rate with Dells, with the exception of their Inspiron line from a number of years ago. This desktop isn't anything spectacular, just an XPS with some muscle and a warranty so I don't have to deal with this in the future. Right now, it's in "production" as they call it. It should arrive about a week from now. Luck being what it is for me, I'm hoping I don't get a lemon. For now, I'm just happy that the little computer community in my office is healthy.
