The past few weeks at work have been the most stressful I’ve had in several years. The network gremlins have been out in full force: students can’t login to the server, printers drop off the network for no reason, users can’t access the internet, etc., etc.

Normal troubleshooting hasn’t unearthed any regular pattern or source of the trouble. I’ve slowly been replacing and/or eliminating individual items – firewall, switches, cabling, server settings. Lots of little things have turned up, but nothing that has completely eliminated the troubles. At this point, almost everything has been touched at one point or another and it seems like I’ve rebuilt the network from scratch.

While I was on vacation, I had a consultant come in and do some troubleshooting and benchmarking. It was a bit expensive, but I needed the backup and I thought another set of eyes may see something I didn’t.

One of the outside recommendations was to increase server capacity. We’ve added a lot of computers this year, but I had more computers in LA last year with the same basic setup. Still, we had a great stroke of luck in that a company in St. Cloud wanted to donate some older servers to a school. The picture shown here is our old rack enclosure with all of the donated gear added. The new machines aren’t online yet (needed to order software, a new power supply and an extra RAID card), but we’ll soon find out if that helps with our peak login issues.

Once the HSRA network and server infrastructure is stable again, I have about 10 web projects to work on, including several non-profit side projects. Speaking of side projects, I’m also trying to learn iPhone programming. More on that in a future post…

UPDATE: Just after I posted this story, I checked the main HSRA server from home and couldn’t see it. I drove in to troubleshoot, and five hours later, I think I found the source of all our trouble. I’ll post a full report tonight.

UPDATE 2: All was well until about 1PM yesterday, when all hell broke lose again. Back to the drawing board…