View Single Post
  #1   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,131
Default Makefile for Vermaden's FreeBSD ZFS root install adapted for 4K sector disks

Introduction

What does the Makefile do? The comments at the beginning of the file gives an explanation:

Code:
# This BSD makefile merely automates Vermaden's "'ZFS madness / boot
# environments" mirrored disk setup.
# 
# For a complete description of this particular ZFS only setup see below.
# 
# The ZFS configuration described by Vermaden does not align the
# disk partitions for 'Avanced Format' 4K disks and also does not
# instruct ZFS to use 4K blocks for the write and read operations.
#
# The procedure implemented here does align the EFI/gpt partitions
# used on a 4K (8 x 512) # sector boundary. By  using the 'gnop' trick
# the FreeBSD ZFS implementation is coached  to use an 'ashift' value
# of 12 (2^12 = 4096)
# This value makes ZFS read and write on 4K boundaries
# 
# Credits for the ZFS install procedure:
#
# Slawomir Wojciech Wojtczak (vermaden)
#   http://forums.freebsd.org/showthread.php?t=31662 
#   http://www.daemonforums.org/showthread.php?t=7099 
#
# Sources and /credits for the 4K alignment issue and gnop hack:
# 
# Warren Block (wblock@) for his numerous posts about partition alignment: 
#     http://forums.freebsd.org (Installation and Storage sections)
#     http://www.wonkity.com/~wblock/docs/...w_standard_gpt
# Ivan Voras for the gnop workaround:
#     http://ivoras.sharanet.org/blog/tree...or-drives.html 
# George Kontostanos (gkontos) for using gnop and a ZFS cache file: 
#    http://forums.freebsd.org/showthread.php?t=23544
#    http://www.aisecure.net/2012/01/16/rootzfs/
The advantage of using a makefile scripted approach:
  • No more typing or pasting errors. Especially if you wipe and partition the wrong disk because of a typo.
  • Speed. After having set the configurable settings you can do a reproducible FreeBSD ZFS install within a few minutes.
  • Easy testing of your changes by using memory disks.
  • If one of your makefile modifications halts with an error you can fix it and redo it easily.
  • There are utilities to wipe all partitioning info from the disks, to destroy the ZFS pool and ZFS datasets and to remove the memory disks. Because they only operate on the devices defined in the makefile, you are less prone to make mistakes.

Overview of the Makefile targets

The utility ones:
  • show
    • Display all Makefile variables
    • Check presence/existence of installation file sets
    • Check availibility of a template for rc.conf
  • diskinfo - run diskinfo(8) on the disks
  • zfsload - load ZFS kernel modules
  • pool_destroy - destroy the ZFS datasets and pool
  • gpart_destroy - remove gpart partioning from disks
  • md_create - create swap backed memory disks
  • md_destroy - remove the memory disks

The pre-installation tasks can be performed individually or by using the pre_install target:
  • partition
    • destroy existing partitioning and initializes a fresh GPT partition scheme
    • create freebsd-boot and freebsd-zfs partitions on a 4K boundary and labels them
    • write appropiate bootcode to disk
  • Creating a ZFS pool with ashift=12 (2^12 = 4096):
    • gnop4k - create gnop devices with 4k sectors for the disks
    • pool4k - create a ZFS storage pool with the gnop devices
    • export - export the storage pool
    • gnop_destroy - destroy the gnop devices
    • import - import the storage pool with the original disks
    • chk_ashift - run # zdb to verify an ashift value of 12
  • zfs_options - set several ZFS options
  • zfs_fs - create the ZFS data sets
  • zfs_swap - configure ZFS swap space

The actual installation:
  • install - extract the installation file sets base.txz and kernel.txz

The post-installation tasks done by target post_install:
  • loader.conf - initialize the loader.conf file
  • fstab - create an empty '/etc/fstab'
  • rc.conf
    • copy rc.conf template
    • add ZFS magic to rc.conf
  • zfs_boot - copy the cachefile to /zfs/boot
  • zfs_umount - unmount the ZFS datasets
  • mountpoint - set the ZFS mountpoint to 'legacy' so that ZFS will not handle the mounts automagically. This way these can be done manually or with Vermaden's port of the Solaris beadm utility.

For the really lazy ones among us there is a target 'all' that does it all:
Code:
pre_install: trailer partition gnop4k pool4k export gnop_destroy import \
        chk_ashift zfs_options zfs_fs zfs_swap

install: trailer
        for THIS in ${INSTALL_SETS} ; do \
        tar --unlink -xvpJf $${THIS} -C ${MOUNT} ;\
        done
        ls -l ${MOUNT}
        zpool status ${POOL}
        zpool iostat ${POOL}
        zfs list

post_install: trailer loader.conf fstab rc.conf zfs_boot zfs_umount mountpoint

all: trailer pre_install install post_install
Configuration

By changing the makefile variables you can adapt the installation to your wishes.

Code:
POOL            = super
ROOT_SET        = ${POOL}/ROOT
BOOT_SET        = ${ROOT_SET}/default

# --- installation file sets
SETS            = base.txz kernel.txz
DIR             = /usr/freebsd-dist     # On FreeBSD liveCD/DVD/memstick
.for X in ${SETS}
INSTALL_SETS += ${DIR}/${X}
.endfor
If you specify memory disks:
Code:
DISKTYPE        = ada
DISKTYPE        = md

.if ${DISKTYPE} == "md"

# --- memory disks see md(4) and mdconfig(8)
# size of disk needs to be at least 64M else you encounter this error:
# "cannot create 'pool': one or more devices is less than the minimum size (64M)"

MD_SIZE         = 2g
SWAPSIZE        = 256m
DISK_1          = /dev/md1
DISK_2          = /dev/md2
DISKS           = ${DISK_1} ${DISK_2}
# --- gpt/EFI labels
LABEL_ZFS       = mdisk_
LABEL_BOOT      = mdboot
For non-memory disks:
Code:
.else

# --- real spinning rust disks
DISK_1          = /dev/ada1
DISK_2          = /dev/ada2
DISKS           = ${DISK_1} ${DISK_2}
SWAPSIZE        = 4G
# --- gpt/EFI labels
LABEL_ZFS       = SYSTEM_
LABEL_BOOT      = BOOT_

.endif
On the FreeBSD live CD the /tmp and /mnt directories are writeable.

Code:
TMP             = /tmp
CACHEFILE       = ${TMP}/zpool.cache
TEMPLATE_RC_CONF= ${TMP}/template_rc.conf
MOUNT           = /mnt
RC_CONF         = ${MOUNT}/etc/rc.conf
LOADER_CONF     = ${MOUNT}/boot/loader.conf
FSTAB           = ${MOUNT}/etc/fstab
Check these settings with:

Code:
# make show
-------------------------
current variable settings
-------------------------
DEBUG                : []
POOL                 : [xpool]
ROOT_SET             : [xpool/ROOT]
BOOT_SET             : [xpool/ROOT/default]
SETS                 : [base.txz kernel.txz]
DIR                  : [/usr/freebsd-dist]
INSTALL_SETS         : [/usr/freebsd-dist/base.txz /usr/freebsd-dist/kernel.txz]
DISKTYPE             : [ada]
MD_SIZE              : []
SWAPSIZE             : [4G]
DISK_1               : [/dev/ada1]
DISK_2               : [/dev/ada2]
DISKS                : [/dev/ada1 /dev/ada2]
LABEL_ZFS            : [disk_]
LABEL_BOOT           : [boot_]
SWAPSIZE             : [4G]
LABEL_ZFS            : [disk_]
LABEL_BOOT           : [boot_]
TMP                  : [/tmp]
CACHEFILE            : [/tmp/zpool.cache]
MOUNT                : [/mnt]
TEMPLATE_RC_CONF     : [/tmp/template_rc.conf]
RC_CONF              : [/mnt/etc/rc.conf]
LOADER_CONF          : [/mnt/boot/loader.conf]
FSTAB                : [/mnt/etc/fstab]
-------------------------
After displaying these values there will be check for the installation files:

Code:
Checking for installation sets ...
Set /usr/freebsd-dist/base.txz : found!
Set /usr/freebsd-dist/kernel.txz : found!
The last check is for a rc.conf template file.
Code:
Checking for 'rc.conf' template ...
/tmp/template_rc.conf not found
exit 100
*** [show] Error code 100
If the file is found we get this:
Code:
----------- rc.conf template ----------
cat /tmp/template_rc.conf
# ------ rc.conf
#defaultrouter=192.168.222.10
hostname='althusser.utp.xnet'
network_interfaces='bge0'
ifconfig_bge0="SYNCDHCP"

# --- daemons
inetd_enable=NO
sshd_enable=yes
sendmail_enable=yes

# --- time synchronization
ntpd_enable=YES
#openntpd_enable=YES
Make sure that these essential files are present before you continue with the installation.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 8th August 2013 at 09:01 PM.
Reply With Quote