30 July 2008

Howto install FreeBSD under ZFS (including root) (Part 2)

I think you know what you doing. If you mess your system up, it's your own problem. You've been warned.







Requirements


Minimum processor: 1.2Ghz (Pentium4/Athlon-XP works fine)
Minimum memory: 768Mb of RAM (1Gb or more is recommended).


Instalation


Let's suppose that you have a 40Gb HD to use:
  1. Boot FreeBSD 7.0 CD
  2. Partition the whole disk
  3. Make 3 slices
    / -> 1Gb
    swap -> you decide with your own practices (or FreeBSD best practices)
    The rest of the disk.
  4. Now install the Minimal, then reboot.


Creating the pool


After the installation is complete, jump to single user mode and mount your / rw:


# mount -w /


Now we create the disk pool, let's suppose that the 3rd partition that you create is ad0s1d:


# zpool create rootvg /dev/ad0s1d


But after start creating the filesystems, stop the automount:


# zfs set mountpoint=none rootvg


Now we'll create the mount points:


# zfs create rootvg/root
# zfs create
rootvg/usr
# zfs create
rootvg/var
# zfs create
rootvg/tmp


Now, we can set the first set of mountpoints (to install data):


# zfs set mountpoint=/rootvg rootvg/root
# zfs set mountpoint=/rootvg/usr rootvg/usr
# zfs set mountpoint=/rootvg/var rootvg/var
# zfs set mountpoint=/rootvg/tmp rootvg/tmp



Use a df and you'll understand.


Now, copy the whole filesystem from / to /rootvg (there's a lot of methods to do that)


*DON'T BOOT YOUR OS, YET*


Booting FreeBSD


I don't remeber where I get this trick: You know, FreeBSD bootloader doesn't know what ZFS is (yet), so, we need to do some tricks:


Remove the boot from the /rootvg:


# rm -fr /rootvg/boot


Create a new dir inside the /rootvg and link the outside-boot with the inside-boot:


# mkdir /rootvg/bootdist
# cd /rootvg
# ln -s bootdist/boot boot



Now, tell the loader to get the ZFS support and mount the ZFS root partition
Don't forget to set vm stuff (since ZFS in FreeBSD is experimental, it's easily to run out of memory and crash the kernel).


# echo 'zfs_load="YES"' >>  /boot/loader.conf
# echo 'vfs.root.mountfrom="zfs:rootvg/root"'  >>  /boot/loader.conf
# echo 'vm.kmem_size_max="512M"' >> /boot/loader.conf
# echo 'vm.kmem_size="512M"' >> /boot/loader.conf
# echo 'vfs.zfs.zil_disable=1' >> /boot/loader.conf



Don't forget to put the boot into the first line of fstab:


/dev/ad0s1a  /bootdist        ufs     rw      1       1


Now, let's do the magic:


# cd /
# zfs set mountpoint=/tmp rootvg/tmp
# zfs set mountpoint=/usr rootvg/usr
# zfs set mountpoint=/var rootvg/var



Set root  to legacy because the loader will to the job:


# zfs set mountpoint=legacy rootvg/root


All done. Now you can reboot your FreeBSD and start to install things.

15 comments:

  1. [...] Howto install FreeBSD under ZFS (including root) ( Daniel de Oliveira) Daniel explains the background of ZFS, the advantages and how it works.This is my first post talking about BSD systems, so, I will talk about my first (bleeding edge) experience making tests on freebsd (FreeBSD 8.0-CURRENT) Part 1 and Part 2 [...]

    ReplyDelete
  2. Andreas Nilsson14.8.08

    Hello,

    i tried the procedure outlined above, but i can't get it to work. Upon reboot it hangs with "init: can't exec getty '/usr/libexec/getty' for port ...

    If i reboot into single user mode i see that rpool/root is mounted but no other filesystems (usr,var,tmp). Any idea's to what i could've missed?

    ReplyDelete
  3. Maybe you forgot to change the mount points:

    # cd /
    # zfs set mountpoint=/tmp rootvg/tmp
    # zfs set mountpoint=/usr rootvg/usr
    # zfs set mountpoint=/var rootvg/var
    # zfs set mountpoint=legacy rootvg/root

    ReplyDelete
  4. Andreas Nilsson14.8.08

    I thought so too, but check listed those mountpoints correctly. But i'll start over and hope for some good luck.

    I'm trying it in virtualbox, so no big deal ;) I also installed bash for ease of use, but that should not affect things to much i hope?

    ReplyDelete
  5. Since this howto is about zfs, theres no problem at all to install in vbox, also no problem using bash.

    By the way, try using single mode to make the partitioning.

    ReplyDelete
  6. Andreas Nilsson14.8.08

    Well, i tried again. It still doesnt work, but i copied all files from rootvg/{var,tmp,usr} to rootvg/root/ so now it boots.

    May this be because i use freebsd 7-release?

    ReplyDelete
  7. No, because I test this on 7.0-release and 8.0-current.

    What is your rootvg/root? Maybe you're forgeting something... Maybe fstab?

    ReplyDelete
  8. Andreas,

    You might have figured this out already, but it seems to me like you are missing

    zfs_enable="YES"

    in your /etc/rc.conf or /etc/rc.conf.local or /etc/rc.conf.d/zfs

    zfs set mountpoint=legacy rootvg/root

    will only mount the root "/" fs
    the rest of the non legacy mountpoints are handled when ZFS is started up via /etc/rc.d/zfs

    (shameless plug) check out the zfsboot script I wrote which automates all this albeit using a slightly different installation method.
    http://yds.CoolRat.org/zfsboot.shtml

    ReplyDelete
  9. I believe you're missing a zfs export somewhere before the reboot.

    ReplyDelete
  10. Elric21.2.09

    Export is not needed. Quoting from zpool(1M):
    "By default, whenever a pool is created or imported on a system, it is permanently added so that it is available whenever the system boots."

    ReplyDelete
  11. Thomas Munn11.4.09

    Your loader.conf file varies from the freebsd tuning guide. Specifically, the tuning guide says to use the following:

    vfs.zfs.prefetch_disable=1

    But yours says to use
    vfs.zfs.zil_disable=1

    Should yours be used in place of or in addition to the tuning guide??

    ReplyDelete
  12. YAA Adding this to my bookmarks. Thank You

    ReplyDelete
  13. Depends on how much memory you have (giant locks).

    ReplyDelete
  14. Just followed these instructions and they failed with the “init: can’t exec getty ‘/usr/libexec/getty" problem as above. Fix was to boot into single user, zfs mount /usr. /var and /tmp and then boot into multi-user. From there edit /etc/rc.conf.local to include:

    zfs_enable=”YES”

    I then rebooted and it booted okay.

    ReplyDelete