Russ' Do It Yourself Home Workshop

I have a couple of Supermicro servers that are running 3Ware 9550SX SATA controllers.  I use these servers for conducting Microsoft SharePoint training classes and run lots of virtual machines on them, so I’m always looking for more space to save my "master" copies of my virtual machines.

The C: drive is set up as a mirror and, until recently, has had two Seagate 500GB 7200 rpm ES.2 SATA drives (that are over 3 years old and still going strong) that comprise the RAID array.  I recently picked up a couple of Western Digital 1TB 5400 rpm SATA drives and thought I’d use them to replace and upgrade the 500GB drives.

In my mind, I had my upgrade path all figured out.  It should be easy, I thought.

1. Pull one 500GB drive on the mirror
2. Insert new 1TB drive in its place
3. Let it rebuild
4. Pull second 500GB drive on the mirror
5. Insert second 1TB drive in its place
6. Let it rebuild
7. Extend the volume into the new space

So, that’s what I did.  I got through step 6 just fine.  I literally yanked a single drive out while the server was running (it was not doing anything important) and slammed the new drive in (well, I actually gently replaced it).  The server recognized the new drive and automatically began the rebuild process.  Each rebuild took 2-3 hours and, by the end of the day, I had the new drives in place.  However, the controller didn’t automatically recognize the new space.  So I started looking around for "the utility" that I needed to run to make the controller see the new space.

I pulled out the manual and did some reading and read about OCE (Online Capacity Expansion) and RAID level migration, but I didn’t see the answer there.  I tried using the 3Ware knowledge base, but no luck.  Searching the web, I didn’t find too much detail on the subject.  I found some blog posts where folks were talking about doing this (lots of Linux posts), but nothing with a definitive answer to my question.  I read where someone suggested hanging another set of drives off another controller and blah, blah, blah.  Yeah, like I have lots of $400 controllers lying around.  I even read a blog post where someone asked "why" you would do this (?) and you couldn’t do it without basically starting from scratch.  Finally I ran across a post where someone suggested to "call 3Ware support."  That sounded like a reasonable move.

I spoke with Joseph Infelise at 3Ware support and he was able to answer my question quickly.  Yes, there was a way to do this but it required running a scripted utility that he could put together for me in a few minutes.

This is a two-part process.  First, you need to run a utility (DUMPDCB.BAT batch file) that queries the controller and dumps the DCB (disk control block) and gives support the detail they need and then you send this detail back to support so they can create the script that will write a new DCB (WRITEDCB.BAT) for your controller.  Both of these scripts use TW_CDIAG.EXE.  If you have a simple configuration like mine, and you know, for sure and without a doubt, that the mirror is on ports 0 and 1 of the controller, you can dispense with the DUMPDCB.BAT and go straight to the WRITEDCB.BAT.  This is what Joseph did for me.  He sent me a script that was set up for this exact configuration.

Joseph sent me the utility in an ISO image so it would make it past firewalls.  I thought I could mount it with MagicISO, but that didn’t work, so I downloaded and installed UltraISO, which worked.  Now that you have the files, you need to copy them to a device that can be read after you boot to DOS.  I do this so infrequently I always have to find a disk from which to boot.  From a DOS prompt, type WRITEDCB and let ‘er rip!  What I saw reminded me of the old mainframe days or even the scene from Jurassic Park ("It’s a UNIX system!  I know this!").  Lots of green screen data blocks whizzing by…pretty impressive and a little scary!  It ended up with a series of lines that said something like "missing command parameter."  At this point, it was reboot time, according to Joseph.

To make a long story short, after rebooting everything came up just fine.  Running disk manager (diskmgmt.msc) now showed the extra space that was added.  Yeah!

Right-clicking on the new C: drive and selecting Extend began the Extend Volume Wizard.  I want it all!

After completing the wizard.

My new C: drive with 931.31GB of space!

And, now the glass (i.e. disk) is half empty…or half full…depending on how you look at it.

I’ve always heard that many of the WD 5400 rpm drives perform as good as or better than many of the 7200 rpm drives on the market, so I thought I’d do a quick test and see how much better/worse they were.  I downloaded and installed IOMeter to check out the performance.  Since I failed to do this before the upgrade, I installed it on both servers, since my other server still has the 500GB Seagate array on a 9550SX controller, just like the one I upgraded.  Here are the results for a quick 32K, 1000 sector test.

New array

Old array

Total I/Os per Second is much improved (1522.5 vs. 3661.43).

Total MBs per Second also improved (23.79 vs. 57.21).

CPU utilization is lower (3.2% vs. 6.65%).

Not sure what the deal is with the Maximum I/O Response Time (ms).  (670 vs. 108)  Perhaps that’s due to the 64MB cache getting spooled upon the WD.  I’m not a disk expert, so if anyone wants to comment, I’m all ears!

Update 2011-09-11

I recently upgraded my 500GB mirror on BIGTEX2 to a 1TB mirror following my previously documented procedure.  I’m adding the IOmeter results so I can now compare the previous array on BIGTEX2 to the new array.  Here’s my setup.  Again, I’m not a disk performance expert!

32K, 100% read access specification.

Total MBs per Second went from about 23 to 26. 

Average IO response time increased from about .65 to 1.2.  I’m guessing it’s the lower performance Seagate AS drives I’m using.  Wonder what would happen if I replaced the drives with a higher performance model?