Howto: Root on ZFS with FreeBSD 9

ZFS is an extremely powerful filesystem and volume manager, ported from SUN Solaris, which supports a lot of interesting features like snapshots, export via NFS (and in theory Samba/CIFS and iSCSI), de-duplication, different mirror/raid modes, and impressive performance for hosts with enough RAM (it caches heavily).

Unfortunately, FreeBSD's sysinstall (and the new bsdinstall as from 9.0) don't support installing FreeBSD in a way that it can boot and entirely run off ZFS yet. This very short guide wants to make up for it :)


This guide was originally written for 9.0-BETA1, but is still applicable, if you don't want bsdinstall to do the job for you.

After you've booted the 9.0-BETA1 CD, choose shell, and follow this example here. Beware, this isn't a newbie guide! Your device names may vary, and also you may want to prepare your /etc/rc.conf properly before first booting into the freshly installed system. Also, the datasets aren't limited in size, r/o restrictions haven't been applied where appropriate etc. That's an exercise for you. The beauty of ZFS is that you can do almost anything on a mounted dataset.

Here's the "log" of what I've done to get FreeBSD 9.0-BETA1 running on ZFS on a physical box:

# create GPT partitioning scheme
gpart create -s gpt ada0
gpart create -s gpt ada1

# reserve space on both harddisks for boot loader
gpart add -b 34 -s 64k -t freebsd-boot ada0
gpart add -b 34 -s 64k -t freebsd-boot ada1

# add SWAP partitions
gpart add -s 16G -t freebsd-swap ada0
gpart add -s 16G -t freebsd-swap ada1

# create actual ZFS partitions, which will hold the root system
gpart add -s 400G -t freebsd-zfs ada0
gpart add -s 400G -t freebsd-zfs ada1

# leave space for mountable UFS partitions in the previous step,
# and fill rest with UFS partitions.
# you need them, because / will not be writable in the shell,
# hence these extra partitions give us some space to write to
gpart add  -t freebsd-ufs ada0
gpart add  -t freebsd-ufs ada1

# now we need to write bootcode to the small slices at the beginning of our disks:
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1

# this temporarily holds the zpool.cache
newfs /dev/ada0p4
mount /dev/ada0p4 /boot/zfs

# load required modules for ZFS
kldload opensolaris
kldload zfs

# create mirrored pool
zpool create zroot mirror ada0p3 ada1p3

# it will throw error, because / is r/o.
# consequently we need to change the mountpoint and remount the pool:
zfs set mountpoint=/mnt zroot
zfs mount -a

# define boot volume of the pool  (this might be obsolete; leave it there anyway)
zpool set bootfs=zroot zroot

# now create some datasets according to your own preferences:
zfs create zroot/tmp
zfs create zroot/var/tmp
zfs create zroot/usr
zfs create zroot/usr/local
zfs create zroot/usr/ports
zfs create zroot/usr/src
zfs create zroot/var
zfs create zroot/var/db

# make sure permissions are set correctly for tmp directories
chmod 1777 /mnt/tmp
chmod 1777 /mnt/var/tmp

# install base, kernel, docs
# (working around broken charset/keyboard layout; could as well directly pipe into tar)

cd /mnt
xz -dc /usr/freebsd-dist/base.txz > base.tar && tar xvf base.tar
xz -dc /usr/freebsd-dist/kernel.txz > kernel.tar && tar xvf kernel.tar
xz -dc /usr/freebsd-dist/lib32.txz > lib32.tar && tar xvf lib32.tar
xz -dc /usr/freebsd-dist/docs.txz > docs.tar && tar xvf docs.tar
rm -f /mnt/*.tar

# now put the mandatory configurations in place (we're chroot-ing for our convenience)
chroot /mnt
echo 'zfs_enable="YES"' > /etc/rc.conf
echo 'zfs_load="YES"' > /boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
cd /etc/mail
make aliases

# important: zpool.cache needs to be copied over;
# otherwise the pool is detected as inactive and won't be mounted during boot
cp /boot/zfs/zpool.cache /mnt/boot/zfs
umount /boot/zfs

# add swaps to fstab
cat << EOF > /mnt/etc/fstab
 /dev/ada0p2      none      swap    sw              0       0
 /dev/ada1p2      none      swap    sw              0       0

# leave /mnt directory and unmount pool
cd /
zfs umount -a

# adjust mountpoints
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/usr/local zroot/usr/local

# in the end verify that all mountpoints look alright:
zfs list

# and then we're done

If you've done everything correctly (and changed things according to your requirements and hardware), your system will reboot straight into your new installation.

Enjoy! :)

Comments !