Tropical shores. A red convertible. Buddhist temples. And the WebReboot Enterprise.

Why do all of these things belong together? Because Chip and I went down to the University of Hawaii to participate in an InfoWorld Enterprise Shootout - “Pimp My Datacenter” - where we spent a week working with the sys admins building their state-of-the-art new computing research facility.

We actually came across Brian Chee (U of H, InfoWorld), Oliver Rist (PC Mag, InfoWorld), and the other fantastic folks running the event at the last Interop in Las Vegas. Ken of Silverback fame introduced us, so many thanks to Ken.

Our trip consisted of:

• 10% scuba diving
• 20% sightseeing
• 30% Oliver pinning the brute force of his wit upon me when he discovered my age in the first 10 seconds of conversation
• 3% Oliver hassling everyone else
• 7% attention reclaimed, Oliver hassling me
• 30% oh crap, we have a lot of work to do!

In terms of scope, it was pretty simple… on paper. Move a whole bunch of IT equipment from around the campus into the brand new DC room. Many vendors were there donating either time, materials, or both. All showing off the latest and greatest in their IT management arsenals. All unprepared for the challenges ahead.

From severely strict weight-limits-per-square-foot (we’re talking “only a couple people in at a time, please” strict), to cooling systems that love to corrode in the Pacific sea breeze, to missing parts… you name it. Brian went through a heck of a time getting this event going, and like any good geek would do, he’s documented all his trouble, pain, and solutions so you don’t suffer through it yourself.

But hey, there are few things worth doing that are easy. And this was a week’s worth of work crammed into one overnight geek fest, so there was no chance of easy.

Our part? We set the lab up with a 48-port WebReboot Enterprise showcasing a distributed installation (i.e., one of our units connected to servers all over the place) and a 24-port WebReboot Enterprise in a centralized and confined installation (i.e., responsible for a single 24-server cluster in a single cabinet). Above and beyond that, we got a server up and running for Nagios to monitor and alert on all of their servers, added the WebReboot plugin to it for automatically recovering crashed equipment, and we got to be honorary-gorillas-for-a-day with the Silverback Migration’s crew hauling in and connecting all of the equipment.

University of Hawaii Staff in the Foreground, APC Folks in the Background

Getting a final plan together for cabling and arranging the new datacenter.

Brian Chee from the University of Hawaii and Ken Jamaca from Silverback Migration Solutions

Common Question: “What time is it?”
Common Answer: “You don’t want to know.”
We unpacked, racked up, cabled, migrated, and tested 12 cabinets and over 60 servers in about 36 hours. How’s that for service?

The 24-Server Cluster for the 24-port WebReboot Enterprise

This cluster consisted of 12 Dell PowerEdge SC1425 servers and 12 Dell PowerEdge 2650 servers. A perfect fit for the 24-port WebReboot Enterprise.

Chip Installing the WebReboot Enterprise in the Cluster Cabinet

The 1U form factor fit nicely at the top of the rack.

The Cluster Fully Cabled, with WebReboot Enteprise at Top

Ready for card installation. Each server gets an Advanced Server Card installed in the expansion slot, that in turn connects to the WebReboot Enterprise. This gives us the ability to reboot the server in the same fashion as pressing the power or reset switch on the front of the chassis. As a bonus, we also get temperature monitoring, power monitoring, and asset tracking.

Cluster Powered Up

Beyond the Cluster - The Rest of the Datacenter

We deployed enough WebReboot ports to cover the entire facility. For the remaining servers, we centrally located a 48-port WebReboot Enterprise in the patch panel rack, and then used standard Cat6 cabling to each cabinet to carry our necessary signals. Above, a Sun Fire V210, Dell PowerEdge 1950, and Dell PowerEdge 2950 were connected to the WebReboot Enterprise (not shown, since it’s in the patch panel rack!).

Testing it All, and Nagios for Monitoring

Above you see the 48-port WebReboot Enterprise, one of our Power Cycle Modules (little orange box on top), our Servprise Demo Server (square, orange-faced box), a server to run Nagios (the 1U Dell PowerEdge SC1420 under laptop) with the Servprise Nagios plugin for automatically rebooting and power controlling crashed/overheated servers, and a laptop connected to the WebReboot Enterprise.

Ross Assembling the “Skittles Railroad”

The engineering folks at the university designed and built a custom cable carrier to run in a loop over the entire facility. Upon which, color coded Cat6 was placed between the patch panel rack and each cabinet in the room. As far as I can recall, green cabling was for the production network, red for the KVM over IP network, black for the WebReboot Enterprise network, and I must admit I don’t recall what blue was for.

While all of our equipment is installed, we held off on fully configuring Nagios for production use until the datacenter folks have a chance to regroup and take it all in. I’m looking forward to automating as much as I can for them. Ross, let me know when you are ready!

For more photos, I have a full Flickr set dedicated to the trip.

There were definitely three highlights of the trip that I will always remember:

1) The sheer gratitude and hospitality by the U of H computing staff. Ross, Pat and Sharon were great to work with, loved our products, shared some good laughs, and that made the long hours all worthwhile.

2) A tour of practically the entire island with Brian and his wife Cathy. What an incredible place Oahu is!

3) Driving down H3 through the mountains in a bright red convertible with the top down. Few things equal it.