Ubuntu RAID-1, LUKS, LVM2

Here's something you don't do every day: install an LVM-managed encrypted root over RAID-1. I just managed to do so with Ubunty 6.10. Here's how.

Setup

You need the Ubuntu Server CD. Desktop does not include RAID, and has no way to install onto RAID.

Install

Boot the Ubuntu install. When you get to the partition screens, hit Alt-F2 for shell. Then manually fdisk your disks. Here's what I made mine:

hda1      1     30515   245111706   5  Extended
hda5      1        13      104359+ fd  raid    /boot
hda6     14       201     1510078+ fd  raid    swap and temp-install
hda7    202     30515   243497173+ fd  raid    crypt-lvm

Note the fd disk type. That means RAID.

Start RAID:

# modprobe raid0    I installed into raid0
# modprobe raid1

Create the md devices. I think the Ubuntu installer requires /dev/md/# (i.e. /dev/md is a subdirectory) instead of /dev/md#:

# mkdir /dev/md
# mknod /dev/md/0 b 9 0
# mknod /dev/md/1 b 9 1
# mknod /dev/md/2 b 9 2

Create the RAID volumes. You need to do these mdadm commands once, and never again:

/boot
# mdadm --create /dev/md/0 --level=1 --raid-devices=2 /dev/hda5 /dev/hdc5
mdadm: /dev/hda5 appears to contain an ext2fs file system
    size=10498304K  mtime=Sun Nov 19 16:42:39 2006
mdadm: /dev/hdc5 appears to contain an ext2fs file system
    size=10498304K  mtime=Sun Nov 19 20:06:27 2006
Continue creating array? y
mdadm: array /dev/md/0 started.

swap, temp install
# mdadm --create /dev/md/1 --level=0 --raid-devices=2 /dev/hda6 /dev/hdc6
mdadm: array /dev/md/1 started.

LVM partition for real install
# mdadm --create /dev/md/2 --level=1 --raid-devices=2 /dev/hda7 /dev/hdc7
mdadm: array /dev/md/2 started.

Partitioning Complete

Now pick the partitions you want for the install. Go back to the install screen (Alt-F1) and make selections so you end up with:

RAID1 device #0 - 106.8 MB Software RAID device
      #1 106.8 MB   f ext3       /boot
RAID0 device #1 - 3.1 GB Software RAID device
      #1   3.1 GB   f ext3       /
RAID1 device #2 - 249.3 GB Software RAID device
      #1 249.3 GB                (ignore this one for now)

Ignore the warning about swap space. (Do you want to go back? "No")

Create the accounts, and let things install. This will take a while.


First Boot

The Ubuntu install is finished. Now reboot.

If you get this GRUB error:

    Error 17: Cannot mount selected partition

...then edit your GRUB boot line. Change (hd0,0) to (hd0,4). You can do this from the boot menu by using the 'e' key.


Apt-get

At this point you're in a shell. Edit /etc/apt/sources.list and uncomment the two universe lines. This expands your repertoire. Then install some necessities:

# apt-get update
# apt-get install cryptsetup     (will also install dmsetup)
# apt-get install lvm2           (will also install lvm-common)
# apt-get install yaird          (will install about 3 others)

Encrypt

Create the LUKS encrypted device:

# cryptsetup luksFormat /dev/md2

WARNING!
========
This will overwrite data on /dev/md2 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:mypassphrase
Verify passphrase:mypassphrase
Command successful.

If you get this error:

  Failed to setup dm-crypt key mapping.
  Check kernel for support for the aes-cbc-essiv:sha256 cipher spec and verify that /dev/md2 contains at least 133 sectors.

Try:

# modprobe aes
# modprobe sha256
# modprobe dm-crypt

You now have the encrypted device, but need to "mount" it:

# cryptsetup luksOpen /dev/md2 pvcrypt
Enter LUKS passphrase:mypassphrase
key slot 0 unlocked.
Command successful.

LVM

Create the Physical Volume, Volume Group, and Logical Volumes. You only need to do these once:

# pvcreate /dev/mapper/pvcrypt
  Physical volume "/dev/mapper/pvcrypt" successfully created

# vgcreate vgcrypt /dev/mapper/pvcrypt
  Volume group "vgcrypt" successfully created

# lvcreate -n lvroot -L 8G vgcrypt
  Logical volume "lvroot" created
# lvcreate -n lvhome -L 100G vgcrypt
  Logical volume "lvhome" created

You can play with the names and sizes. If you do, simply s/// as needed below.

Initialize filesystems

# mkfs.ext3 /dev/mapper/vgcrypt-lvroot
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1048576 inodes, 2097152 blocks
104857 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2147483648
64 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
  32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 27 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

# mkfs.ext3 /dev/mapper/vgcrypt-lvhome
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
13107200 inodes, 26214400 blocks
1310720 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=29360128
800 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
  4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

Mount

# mkdir /mnt/target
# mount /dev/mapper/vgcrypt-lvroot /mnt/target
# mkdir /mnt/target/home
# mount /dev/mapper/vgcrypt-lvhome /mnt/target/home

Copy the install from the tmp fs to the encrypted one:

# cp -axv / /mnt/target

Create two temporary device nodes under the new /dev:

# mknod /mnt/target/dev/console c 5 1
# mknod /mnt/target/dev/null c 1 3
If you don't do the mknods, you will get this on your next boot:
   Switching root ...
   /usr/lib/yaird/exec/run_init: opening console: No such file or directory
   Kernel panic - not syncing: Attempted to kill init!

Chroot

# mount --bind /dev  /mnt/target/dev
# mount --bind /sys  /mnt/target/sys
# mount --bind /proc /mnt/target/proc

# chroot /mnt/target
# mount /boot

Update System Files

You are now cd'ed to what will be your real disk.

Edit /etc/crypttab. Add this line:

pvcrypt /dev/md2

Edit /etc/fstab, changing the mount points:

/dev/mapper/vgcrypt-lvroot   /      ...
/dev/mapper/vgcrypt-lvhome   /home  ...

Edit /boot/grub/menu.lst, changing the root= lines as needed. E.g.:

kernel    /vmlinuz-2.6.17-10-server root=/dev/mapper/vgcrypt-lvroot ro quiet splash

Edit /etc/mdadm/mdadm.conf, adding auto=md to all lines. Without this, your next boot will barf with fsck errors.

Create initrd

Preserve your original initrd, and create a new one:

# mv /boot/initrd.img-2.6.17-10-server{,.XX}
# yaird -o /boot/initrd.img-2.6.17-10-server

Cross Your Fingers, and Reboot

Reboot.


----- 11/20/06, 17:09 -- finally, apt-get install ubuntu-desktop /data0/esm/.logdir/0611 (END)

Acknowledgments