@dkirwan could you take a look at this one?

Spoke with Dell tech support, they are going to ship us out a replacement for the failed disk.

Contacted internal engineer to provide details to Dell.

The dell techs are going to be at DC around 14:00 EST today to replace this harddrive.

The dell technician has replaced this drive now.

Yep. So there's a tiny bit more manual action here. :)

• Go into the mgmt console for the host. The drive should show 'ready' or something.
• Make a new virtual drive with just that disk in it. You should be able to tell it to do it now, no need for reboot.
• Then the drive should appear on the linux side.
• Use fdisk / parted / gparted/whatever to partition the disk like the other disks are exactly the same.
• Use 'mdadm /dev/mdN --add /dev/sdXY' to add partitions to each raid.

Let me know if you need me to do any of that, or if it's unclear. ;)

Ok should be back in action shortly. Raid rebuilding here:

Personalities : [raid1] [raid6] [raid5] [raid4] 
md1 : active raid1 sdb2[8] sde2[7] sdd2[2] sda2[0] sdh2[5] sdg2[6] sdc2[4] sdf2[3]
      488384 blocks super 1.0 [8/8] [UUUUUUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md2 : active raid6 sdb3[8] sdc3[4] sdf3[3] sda3[0] sde3[7] sdg3[6] sdh3[5] sdd3[2]
      2800041984 blocks super 1.2 level 6, 512k chunk, algorithm 2 [8/7] [U_UUUUUU]
      [===>.................]  recovery = 16.2% (75786424/466673664) finish=39.1min speed=166542K/sec
      bitmap: 2/4 pages [8KB], 65536KB chunk

md0 : active raid1 sdb1[8] sdf1[3] sdc1[4] sda1[0] sde1[7] sdd1[2] sdg1[6] sdh1[5]
      1022976 blocks super 1.2 [8/8] [UUUUUUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

Awesome. Thanks!