Installing Linux on Software RAID in 10 Steps

While this guide is primarily targeted at Gentoo, the concepts and core operations apply to any distro. This guide is a hybrid of several sources, including the Gentoo wiki (which seems to come and go and come and go…) as well as my own notes from doing many Gentoo installs on Software RAID over the years.

Step 1: Partitioning
Partition the first disk using your favorite tool, whatever that may be. fdisk, cfdisk, sfdisk, parted, whatever.

IMPORTANT NOTE: You’ll probably want to create a small RAID 1 boot partition. Grub doesn’t work on some RAIDs, and RAID 1 will ensure that you should be able to boot provided at least one disc is still operational.

IMPORTANT NOTE: For partitions you attend to add to the array, you BETTER set the type to fd (RAID autodetect). Failure to do so could lead to boot problems (kernel panics), and general destruction.

Step 2: Copy that partition scheme to all disks
# sfdisk -d /dev/sda | sfdisk /dev/sdb

Step 3: Create your RAID array
We’ll be using mdadm. It’s the Linux swiss army knife for software RAID. Install it if your distro doesn’t come with it preinstalled.

# mdadm –create –verbose /dev/md[1] –level=[2] –raid-devices=[3] /dev/sd[4] /dev/sd[4]

Key:
[1]: Refers to the device to be created in /dev. Start with md1, then md2, ….
[2]: Refers to the RAID level. RAID 1 (mirrored), RAID 0 (striped), ….
[3]: Refers to the number of discs in your volume.
[4]: Refers to the partition on each disc in your volume to add to this RAID array. Since you have identical partition layouts (unless you’re a masochist), this should look like /dev/sda1 /dev/sdb1 … or /dev/sda3 /dev/sdb3 … etc.

If you’re creating a RAID 1 or 10 array, and virgin/reformated discs, you can use –assume-clean to skip the sync operation normally ran upon the creation of a mirrored array. Do not attempt this on RAID 4, 5, or 6 … or you will feel much pain.

You can check on the status of any syncing or other things going on with your newly created arrays at any time by simply running
# cat /proc/mdstat

Step 4: Set the mdadm config file
# mdadm –detail –scan >> /etc/mdadm.conf

Step 5: Proceed to install your distro of choice
You can now use the /dev/md* devices you created for your RAID arrays as though they were physical discs. You don’t even have to wait for the syncing to finish to begin formatting partitions or mounting. It’s fully transparent goodness.

Step 6: Kernel config for Gentoo
All of the follow must be compiled directly into the kernel. Modules will not work.

Multi-device support (RAID and LVM) —>
[*] Multiple devices driver support (RAID and LVM)
<*> RAID support
<*> (Whichever mode(s) of RAID you’re going to use)
<*> Device mapper support

Step 7: Make sure the mdadm service starts as soon as possible
In Gentoo: # rc-update add mdadm boot

Step 8: Manual installation of GRUB
# grub –no-floppy

This will lunch the interactive GRUB command line, denoted by “grub>” starting each line.

grub> find /boot/grub/stage1

You should get a reply with the location of every boot partition on every drive (you DID make a RAID 1 boot, RIGHT?). So you should see “(hd0,0)” … “(hd1,0)” … “(hd2,0)” …

Keep in mind on the device line of the follow command set the GRUB conventions for naming hard drives. hd0,0 means HD #1, partition #1. hd1,0 means HD #2, partition #1. hd1,1 means HD#2, partition #2. And so on.

grub> device (hd0) /dev/sda
grub> root (hd0,0)
grub> setup (hd0)

Repeat this for the boot partition of every single drive you have.

grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)

After you’ve done so, exit the GRUB interactive shell.

grub> exit

Step 9: Setting your grub config file
Your grub config file should look like normal. The only change is the root device. This is where having that RAID 1 boot partition pays off. If a drive drops off the map, when you boot grub whichever drive is now the new first drive has its own boot partition ready to go and boot.

/boot/grub/grub.conf (or menu.lst … depends on your distro):

default 0
timeout 15

title=Gentoo Linux 2.6.25-hardened-r11
root (hd0,1)
kernel /boot/kernel-2.6.25-hardened-r11 root=/dev/md2


Step 10: Final check and reboot
Check /proc/mdstat to ensure all of your arrays are done and fully synced. If so you can reboot into your new Linux, running software RAID.