FreeBSD 8.0 with a single root ZFS partition

So I finally took the plunge this weekend and converted my home FreeBSD workstation/server to have a single root ZFS partition.  Setting it up was something of a challenge, but in hindsight the steps were pretty easy.

Initially, I looked at the FreeBSD Wiki article about ZFS partitions, which pointed me to another web page with information about setting up a ZFS-only system.  However, those instructions were written for FreeBSD 7.x and they didn't work for me (although it was probably something that I did wrong).  I then read about mfsBSD with it's handy zfsinstall script, which is the route I finally used successfully.  In a nutshell, here's what I did:

  1. Download the mfsBSD ISO and burn it to a CD.  You need the "special edition" ISO as that one contains the needed zfsinstall script.
  2. Boot your system into single user mode:
    shutdown now
  3. Mount your external backup media (I used an external USB hard drive; I highly recommend using something that you can easily detach from the system during the install process):
    mount /dev/da0 /mnt
  4. Make a complete backup of your entire system:
    rsync --progress --recursive --delete --times --perms --owner --group --links --hard-links --exclude=mnt / /mnt/
  5. Boot your system off the mfsBSD ISO and then run zfsinstall.  In my case, I have two identical drives that I wanted configured in a mirror:
    zfsinstall -d /dev/ad4 -d /dev/ad8 -r mirror -t /cdrom/8.0p2-amd64-v3.tbz -s 4G
  6. Pop out the mfsBSD CD-ROM and reboot your computer and select "Single User Mode" at the boot loader menu.
  7. Once in single-user mode, you can look at your new ZFS pool:
    zpool list
  8. By default, the zinstall script creates separate root, usr, var and tmp partitions. I wanted everything in one big root partition, so I did the following to eliminate the extra partitions:
    chflags -R noschg /usr
    mkdir /usr-tmp
    mv /usr/* /usr-tmp/
    zfs destroy tank/usr
    mv /usr-tmp /usr
    chflags -R noschg /var
    mkdir /var-tmp
    mv /var/* /var-tmp/
    zfs destroy tank/var
    zfs destroy tank/tmp
    mkdir /tmp

    Then I edited /ets/fstab and removed the extra partitions from there.
  9. Reboot and go back into single user mode to make sure that all your partitions look good. Once you're sure everything is right, restore your system backup (excluding /etc/fstab and /boot):
    mount -o ro /dev/da0 /mnt
    rsync --progress --recursive --delete --times --perms --owner --group --links --hard-links --exclude=mnt --exclude=etc/fstab --exclude=boot /mnt/ /
  10. Reboot and you -should- have a working system with a single ZFS root partition.  After all this, I ran through a "make buildworld buildkernel installkernel installworld" in single user mode to fix all the file permissions that I clobbered with the "chflags" commands above.  Of course, be sure to update /usr/src before you do this with the latest kernel so that you don't render your system unbootable.

This procedure worked for me (and keep in mind that I'm creating this procedure from memory a day later).  Of course, I can't stress the importance of step 3 enough!  A good working backup is ESSENTIAL for this sort of activity.