DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default Makefile for Vermaden's FreeBSD ZFS root install adapted for 4K sector disks

Introduction

What does the Makefile do? The comments at the beginning of the file gives an explanation:

Code:
# This BSD makefile merely automates Vermaden's "'ZFS madness / boot
# environments" mirrored disk setup.
# 
# For a complete description of this particular ZFS only setup see below.
# 
# The ZFS configuration described by Vermaden does not align the
# disk partitions for 'Avanced Format' 4K disks and also does not
# instruct ZFS to use 4K blocks for the write and read operations.
#
# The procedure implemented here does align the EFI/gpt partitions
# used on a 4K (8 x 512) # sector boundary. By  using the 'gnop' trick
# the FreeBSD ZFS implementation is coached  to use an 'ashift' value
# of 12 (2^12 = 4096)
# This value makes ZFS read and write on 4K boundaries
# 
# Credits for the ZFS install procedure:
#
# Slawomir Wojciech Wojtczak (vermaden)
#   http://forums.freebsd.org/showthread.php?t=31662 
#   http://www.daemonforums.org/showthread.php?t=7099 
#
# Sources and /credits for the 4K alignment issue and gnop hack:
# 
# Warren Block (wblock@) for his numerous posts about partition alignment: 
#     http://forums.freebsd.org (Installation and Storage sections)
#     http://www.wonkity.com/~wblock/docs/...w_standard_gpt
# Ivan Voras for the gnop workaround:
#     http://ivoras.sharanet.org/blog/tree...or-drives.html 
# George Kontostanos (gkontos) for using gnop and a ZFS cache file: 
#    http://forums.freebsd.org/showthread.php?t=23544
#    http://www.aisecure.net/2012/01/16/rootzfs/
The advantage of using a makefile scripted approach:
  • No more typing or pasting errors. Especially if you wipe and partition the wrong disk because of a typo.
  • Speed. After having set the configurable settings you can do a reproducible FreeBSD ZFS install within a few minutes.
  • Easy testing of your changes by using memory disks.
  • If one of your makefile modifications halts with an error you can fix it and redo it easily.
  • There are utilities to wipe all partitioning info from the disks, to destroy the ZFS pool and ZFS datasets and to remove the memory disks. Because they only operate on the devices defined in the makefile, you are less prone to make mistakes.

Overview of the Makefile targets

The utility ones:
  • show
    • Display all Makefile variables
    • Check presence/existence of installation file sets
    • Check availibility of a template for rc.conf
  • diskinfo - run diskinfo(8) on the disks
  • zfsload - load ZFS kernel modules
  • pool_destroy - destroy the ZFS datasets and pool
  • gpart_destroy - remove gpart partioning from disks
  • md_create - create swap backed memory disks
  • md_destroy - remove the memory disks

The pre-installation tasks can be performed individually or by using the pre_install target:
  • partition
    • destroy existing partitioning and initializes a fresh GPT partition scheme
    • create freebsd-boot and freebsd-zfs partitions on a 4K boundary and labels them
    • write appropiate bootcode to disk
  • Creating a ZFS pool with ashift=12 (2^12 = 4096):
    • gnop4k - create gnop devices with 4k sectors for the disks
    • pool4k - create a ZFS storage pool with the gnop devices
    • export - export the storage pool
    • gnop_destroy - destroy the gnop devices
    • import - import the storage pool with the original disks
    • chk_ashift - run # zdb to verify an ashift value of 12
  • zfs_options - set several ZFS options
  • zfs_fs - create the ZFS data sets
  • zfs_swap - configure ZFS swap space

The actual installation:
  • install - extract the installation file sets base.txz and kernel.txz

The post-installation tasks done by target post_install:
  • loader.conf - initialize the loader.conf file
  • fstab - create an empty '/etc/fstab'
  • rc.conf
    • copy rc.conf template
    • add ZFS magic to rc.conf
  • zfs_boot - copy the cachefile to /zfs/boot
  • zfs_umount - unmount the ZFS datasets
  • mountpoint - set the ZFS mountpoint to 'legacy' so that ZFS will not handle the mounts automagically. This way these can be done manually or with Vermaden's port of the Solaris beadm utility.

For the really lazy ones among us there is a target 'all' that does it all:
Code:
pre_install: trailer partition gnop4k pool4k export gnop_destroy import \
        chk_ashift zfs_options zfs_fs zfs_swap

install: trailer
        for THIS in ${INSTALL_SETS} ; do \
        tar --unlink -xvpJf $${THIS} -C ${MOUNT} ;\
        done
        ls -l ${MOUNT}
        zpool status ${POOL}
        zpool iostat ${POOL}
        zfs list

post_install: trailer loader.conf fstab rc.conf zfs_boot zfs_umount mountpoint

all: trailer pre_install install post_install
Configuration

By changing the makefile variables you can adapt the installation to your wishes.

Code:
POOL            = super
ROOT_SET        = ${POOL}/ROOT
BOOT_SET        = ${ROOT_SET}/default

# --- installation file sets
SETS            = base.txz kernel.txz
DIR             = /usr/freebsd-dist     # On FreeBSD liveCD/DVD/memstick
.for X in ${SETS}
INSTALL_SETS += ${DIR}/${X}
.endfor
If you specify memory disks:
Code:
DISKTYPE        = ada
DISKTYPE        = md

.if ${DISKTYPE} == "md"

# --- memory disks see md(4) and mdconfig(8)
# size of disk needs to be at least 64M else you encounter this error:
# "cannot create 'pool': one or more devices is less than the minimum size (64M)"

MD_SIZE         = 2g
SWAPSIZE        = 256m
DISK_1          = /dev/md1
DISK_2          = /dev/md2
DISKS           = ${DISK_1} ${DISK_2}
# --- gpt/EFI labels
LABEL_ZFS       = mdisk_
LABEL_BOOT      = mdboot
For non-memory disks:
Code:
.else

# --- real spinning rust disks
DISK_1          = /dev/ada1
DISK_2          = /dev/ada2
DISKS           = ${DISK_1} ${DISK_2}
SWAPSIZE        = 4G
# --- gpt/EFI labels
LABEL_ZFS       = SYSTEM_
LABEL_BOOT      = BOOT_

.endif
On the FreeBSD live CD the /tmp and /mnt directories are writeable.

Code:
TMP             = /tmp
CACHEFILE       = ${TMP}/zpool.cache
TEMPLATE_RC_CONF= ${TMP}/template_rc.conf
MOUNT           = /mnt
RC_CONF         = ${MOUNT}/etc/rc.conf
LOADER_CONF     = ${MOUNT}/boot/loader.conf
FSTAB           = ${MOUNT}/etc/fstab
Check these settings with:

Code:
# make show
-------------------------
current variable settings
-------------------------
DEBUG                : []
POOL                 : [xpool]
ROOT_SET             : [xpool/ROOT]
BOOT_SET             : [xpool/ROOT/default]
SETS                 : [base.txz kernel.txz]
DIR                  : [/usr/freebsd-dist]
INSTALL_SETS         : [/usr/freebsd-dist/base.txz /usr/freebsd-dist/kernel.txz]
DISKTYPE             : [ada]
MD_SIZE              : []
SWAPSIZE             : [4G]
DISK_1               : [/dev/ada1]
DISK_2               : [/dev/ada2]
DISKS                : [/dev/ada1 /dev/ada2]
LABEL_ZFS            : [disk_]
LABEL_BOOT           : [boot_]
SWAPSIZE             : [4G]
LABEL_ZFS            : [disk_]
LABEL_BOOT           : [boot_]
TMP                  : [/tmp]
CACHEFILE            : [/tmp/zpool.cache]
MOUNT                : [/mnt]
TEMPLATE_RC_CONF     : [/tmp/template_rc.conf]
RC_CONF              : [/mnt/etc/rc.conf]
LOADER_CONF          : [/mnt/boot/loader.conf]
FSTAB                : [/mnt/etc/fstab]
-------------------------
After displaying these values there will be check for the installation files:

Code:
Checking for installation sets ...
Set /usr/freebsd-dist/base.txz : found!
Set /usr/freebsd-dist/kernel.txz : found!
The last check is for a rc.conf template file.
Code:
Checking for 'rc.conf' template ...
/tmp/template_rc.conf not found
exit 100
*** [show] Error code 100
If the file is found we get this:
Code:
----------- rc.conf template ----------
cat /tmp/template_rc.conf
# ------ rc.conf
#defaultrouter=192.168.222.10
hostname='althusser.utp.xnet'
network_interfaces='bge0'
ifconfig_bge0="SYNCDHCP"

# --- daemons
inetd_enable=NO
sshd_enable=yes
sendmail_enable=yes

# --- time synchronization
ntpd_enable=YES
#openntpd_enable=YES
Make sure that these essential files are present before you continue with the installation.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 8th August 2013 at 09:01 PM.
Reply With Quote
  #2   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

Pre-installation steps

For demonstration purpose two memory or RAM disks will be used.

Code:
# make md_create

Creating Memory disk devices /dev/md1 /dev/md2:
Memory disk devices:
md1     swap     2048M
md2     swap     2048M
crw-r-----  1 root  operator    0, 162 Aug  5 16:50 /dev/md1
crw-r-----  1 root  operator    0, 163 Aug  5 16:50 /dev/md2
crw-------  1 root  wheel       0,  63 Aug  5 14:58 /dev/mdctl
-------------end of md_create --------------------
Running # diskinfo on these disks.

Code:
# make diskinfo
if [ -e /dev/md1 ] ; then diskinfo -v /dev/md1 ; fi
/dev/md1
        512             # sectorsize
        2147483648      # mediasize in bytes (2.0G)
        4194304         # mediasize in sectors
        0               # stripesize
        0               # stripeoffset

if [ -e /dev/md2 ] ; then diskinfo -v /dev/md2 ; fi
/dev/md2
        512             # sectorsize
        2147483648      # mediasize in bytes (2.0G)
        4194304         # mediasize in sectors
        0               # stripesize
        0               # stripeoffset

-------------end of diskinfo --------------------
It is not always needed to edit the Makefile to have it do something differently.
To use this target on one of my disks reporting a 512 sectorsize to the OS but having a stripesize of 4096, I can override the makefile variable DISKS on the command line;
Code:
# make DISKS=/dev/ada1 diskinfo

if [ -e /dev/ada1 ] ; then diskinfo -v /dev/ada1 ; fi
/dev/ada1
        512             # sectorsize
        2000398934016   # mediasize in bytes (1.8T)
        3907029168      # mediasize in sectors
        4096            # stripesize
        0               # stripeoffset
        3876021         # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        S1E160MR        # Disk ident.

-------------end of diskinfo --------------------
After this intermezzo we start the real pre-installation work by creating 4K aligned GPT partitions.

WARNING: the following command will remove all partitioning information from the disks as defined in the makefile. Please make sure that this is correct and double-check the settings you configured by running # make show. You will lose data if the settings refer to the wrong disk.

Code:
# make partition

if gpart show /dev/md1 ; then  gpart destroy -F /dev/md1 ; fi
gpart: No such geom: /dev/md1.
gpart create -s gpt /dev/md1
md1 created
NR=$( echo /dev/md1 | tr -c -d '0-9' ) ; gpart add  -b 40 -s 128k -t freebsd-boot -l mdboot${NR} /dev/md1 ; gpart add              -t freebsd-zfs  -l mdisk_${NR}  /dev/md1
md1p1 added
md1p2 added
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 /dev/md1
bootcode written to md1
gpart show /dev/md1
=>     34  4194237  md1  GPT  (2.0G)
       34        6       - free -  (3.0k)
       40      256    1  freebsd-boot  (128k)
      296  4193975    2  freebsd-zfs  (2G)
[snip]
ls -l /dev/gpt
crw-r-----  1 root  operator    0, 170 Aug  5 16:56 mdboot1
crw-r-----  1 root  operator    0, 176 Aug  5 16:56 mdboot2
crw-r-----  1 root  operator    0, 168 Aug  5 16:56 mdisk_1
crw-r-----  1 root  operator    0, 174 Aug  5 16:56 mdisk_2
-------------end of partition --------------------
Here we see that the partitions start at sector 40 and 296. A quick command line modulo calculation with bc(1) confirms that the start sectors on a 8 x 512 = 4096 boundary. An alternative method, a division shows no remainder.

Code:
$ echo '296 % 8' | bc
0
$ echo 'scale=4 ; 296 / 8' | bc  
37.0000
Creating xthe gnop(8) devices with 4K sectors.

Code:
# make gnop4k

Creating gnop devices with 4K sectors .....
for X in /dev/md1 /dev/md2 ; do  NR=$( echo ${X} | tr -c -d '0-9' ) ;
 gnop create -S 4096 /dev/gpt/mdisk_${NR} ; done

ls -l /dev/gpt
crw-r-----  1 root  operator    0, 170 Aug  5 16:56 mdboot1
crw-r-----  1 root  operator    0, 176 Aug  5 16:56 mdboot2
crw-r-----  1 root  operator    0, 168 Aug  5 16:56 mdisk_1
crw-r-----  1 root  operator    0, 166 Aug  5 17:38 mdisk_1.nop
crw-r-----  1 root  operator    0, 174 Aug  5 16:56 mdisk_2
crw-r-----  1 root  operator    0, 172 Aug  5 17:38 mdisk_2.nop

gnop list
Geom name: gpt/mdisk_1.nop
WroteBytes: 0
ReadBytes: 131072
Writes: 0
Reads: 21
Error: 5
WriteFailProb: 0
ReadFailProb: 0
Offset: 0
Providers:
1. Name: gpt/mdisk_1.nop
   Mediasize: 2147311616 (2G)
   Sectorsize: 4096
   Mode: r0w0e0
Consumers:
1. Name: gpt/mdisk_1
   Mediasize: 2147315200 (2G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 151552
   Mode: r0w0e0

[snip]

gnop status
           Name  Status  Components
gpt/mdisk_1.nop     N/A  gpt/mdisk_1
gpt/mdisk_2.nop     N/A  gpt/mdisk_2
-------------end of gnop4k --------------------
Note the gnop GEOM provider with a sector size of 4096.
A ZFS storage pool with these freshly created 4K devices is the next step.

Code:
# make pool4k

Creating zpool with 4K gnop devices
if [ -f /tmp/zpool.cache ] ; then mv /tmp/zpool.cache /tmp/zpool.cache.prev ; fi
ls -l /dev/gpt/*nop
crw-r-----  1 root  operator    0, 166 Aug  5 17:38 mdisk_1.nop
crw-r-----  1 root  operator    0, 172 Aug  5 17:38 mdisk_2.nop

# ---- creating a ZFS mirror pool  without automatically mounting it (-m none) .....
zpool create -f -o cachefile=/tmp/zpool.cache -m none  super mirror /dev/gpt/mdisk_*nop

zpool list super
NAME    SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
super  1.98G   560K  1.98G     0%  1.00x  ONLINE  -

zpool status super
  pool: super
 state: ONLINE
  scan: none requested
config:

        NAME                 STATE     READ WRITE CKSUM
        super                ONLINE       0     0     0
          mirror-0           ONLINE       0     0     0
            gpt/mdisk_1.nop  ONLINE       0     0     0
            gpt/mdisk_2.nop  ONLINE       0     0     0

errors: No known data errors

zfs list
NAME    USED  AVAIL  REFER  MOUNTPOINT
super   468K  1.95G   144K  none
-------------end of pool4k --------------------
Export the pool:

Code:
# make export
Exporting super ......
---------------------------------
zpool export super
---------------------------------
Showing import status of super ......
---------------------------------
zpool import
   pool: super
     id: 4218734048000312785
  state: ONLINE
 action: The pool can be imported using its name or numeric identifier.
 config:

        super                ONLINE
          mirror-0           ONLINE
            gpt/mdisk_1.nop  ONLINE
            gpt/mdisk_2.nop  ONLINE
-------------end of export --------------------
Destroy the gnop devices.

Code:
# make gnop_destroy
Destroy the 4K gnop devices
for X in /dev/md1 /dev/md2 ; 
  do  NR=$( echo ${X} | tr -c -d '0-9' ) ;
   gnop destroy /dev/gpt/mdisk_${NR}.nop ;
done

ls -l /dev/gpt
total 0
crw-r-----  1 root  operator    0, 170 Aug  5 16:56 mdboot1
crw-r-----  1 root  operator    0, 176 Aug  5 16:56 mdboot2
crw-r-----  1 root  operator    0, 168 Aug  5 16:56 mdisk_1
crw-r-----  1 root  operator    0, 174 Aug  5 16:56 mdisk_2
[snip]
-------------end of gnop_destroy --------------------
After the gnop devices have gone the way of the dodo, we now import our pool.

Code:
# make import

Import the pool
zpool import -o cachefile=/tmp/zpool.cache super
zpool list super
NAME    SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
super  1.98G   720K  1.98G     0%  1.00x  ONLINE  -
---------------------------------
zpool status super
  pool: super
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        super       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            md1p2   ONLINE       0     0     0
            md2p2   ONLINE       0     0     0

errors: No known data errors
---------------------------------
mount
/dev/ada0s3a on / (ufs, local, noatime, soft-updates)
devfs on /dev (devfs, local, multilabel)
tmpfs on /tmp (tmpfs, local)
-------------end of import --------------------
The pool is ONLINE but not yet mounted.
Note that for some reason the import does not use the GPT labels, but the partition numbers. I only noticed this when I test with memory disks. With real disks the import uses the labels:

Code:
# ls -l /dev/gpt
total 0
crw-r-----  1 root  operator    0, 173 Jul 26 04:41 boot_1
crw-r-----  1 root  operator    0, 182 Jul 26 04:41 boot_2
crw-r-----  1 root  operator    0, 171 Jul 26 04:41 disk_1
crw-r-----  1 root  operator    0, 180 Jul 26 04:41 disk_2

# make import
Import the pool
zpool import -o cachefile=/var/tmp/zpool.cache super
zpool status
  pool: super
 state: ONLINE
  scan: none requested
config:

        NAME            STATE     READ WRITE CKSUM
        super           ONLINE       0     0     0
          mirror-0      ONLINE       0     0     0
            gpt/disk_1  ONLINE       0     0     0
            gpt/disk_2  ONLINE       0     0     0
Make sure that the imported pool although created with 4K size gnop devices, has retained the proper ashift value.

Code:
#  make chk_ashift

Verify that ashift value is 12 (2^12 = 4096 ; 2^9 = 512)
=========================================================
zdb -C -U /tmp/zpool.cache super

MOS Configuration:
        version: 28
        name: 'super'
        state: 0
        txg: 65
        pool_guid: 4218734048000312785
        hostid: 556313802
        hostname: 'althusser.utp.xnet'
        vdev_children: 1
        vdev_tree:
            type: 'root'
            id: 0
            guid: 4218734048000312785
            children[0]:
                type: 'mirror'
                id: 0
                guid: 16715166418299641908
                metaslab_array: 30
                metaslab_shift: 24
                ashift: 12
                asize: 2142502912
                is_log: 0
                create_txg: 4
                children[0]:
                    type: 'disk'
                    id: 0
                    guid: 5669787805994321979
                    path: '/dev/md1p2'
                    phys_path: '/dev/md1p2'
                    whole_disk: 1
                    create_txg: 4
                children[1]:
                    type: 'disk'
                    id: 1
                    guid: 10458960657003691384
                    path: '/dev/md2p2'
                    phys_path: '/dev/md2p2'
                    whole_disk: 1
                    create_txg: 4
====================================
zdb -C -U /tmp/zpool.cache super | grep ashift
                ashift: 12
-------------end of chk_ashift --------------------
That looks fine. With a 4K optimized ZFS pool, consisting of 4K aligned GPT partitions we can resume with the original procedure outlined by Vermaden, by setting options that will be inherited by the child ZFS datasets.

Code:
# make zfs_options
zfs set mountpoint=none super
zfs set checksum=fletcher4 super
zfs set atime=off super

zpool list super
NAME    SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
super  1.98G   780K  1.98G     0%  1.00x  ONLINE  -

zpool status super
  pool: super
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        super       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            md1p2   ONLINE       0     0     0
            md2p2   ONLINE       0     0     0

errors: No known data errors
zfs list
NAME    USED  AVAIL  REFER  MOUNTPOINT
super   504K  1.95G   144K  none
-------------end of zfs_options -------------------
For handling the boot environments with Vermaden's 'beadm' utility we create the following ZFS datasets and set the appropiate options:

Code:
# make zfs_fs
---------------------------------
zfs create                    super/ROOT
zfs create -o mountpoint=/mnt super/ROOT/default
zpool set bootfs=super/ROOT/default  super
zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
super                912K  1.95G   144K  none
super/ROOT           288K  1.95G   144K  none
super/ROOT/default   144K  1.95G   144K  /mnt
---------------------------------
mount
/dev/ada0s3a on / (ufs, local, noatime, soft-updates)
devfs on /dev (devfs, local, multilabel)
tmpfs on /tmp (tmpfs, local)
super/ROOT/default on /mnt (zfs, local, noatime, nfsv4acls)
-------------end of zfs_fs --------------------
As you can see we have now a ZFS dataset mounted on /mnt.

In the original procedure the ZFS swap space is configured after rebooting into the new system, but let us do it now.

Code:
# make zfs_swap

zfs create -V 256m   super/swap
zfs set org.freebsd:swap=on super/swap
zfs set checksum=off        super/swap
zfs set sync=disabled       super/swap
zfs set primarycache=none   super/swap
zfs set secondarycache=none super/swap

zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
super                265M  1.69G   144K  none
super/ROOT           288K  1.69G   144K  none
super/ROOT/default   144K  1.69G   144K  /mnt
super/swap           264M  1.95G    72K  -
-------------end of zfs_swap --------------------
Having a ZFS pool and datasets with the super/ROOT/default mounted on /mnt we are now ready to unpack the FreeBSD installation sets.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 5th August 2013 at 11:13 PM. Reason: Added WARNING
Reply With Quote
  #3   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

Installation

Let us go for it.

Code:
# make install
for THIS in /usr/freebsd-dist/base.txz /usr/freebsd-dist/kernel.txz ;
    do  tar --unlink -xvpJf ${THIS} -C /mnt ; done
x ./
x ./bin/
x ./boot/
x ./dev/
x ./etc/
x ./lib/
x ./libexec/
x ./media/
x ./mnt/
x ./proc/
x ./rescue/
x ./root/
x ./sbin/
[snip]
x ./boot/kernel/zlib.ko
x ./boot/kernel/zlib.ko.symbols
x ./boot/kernel/linker.hints
After all the files have been extracted we are presented with:
  • directory listing of the FreeBSD installation
    Code:
    ls -l /mnt
    total 146
    -rw-r--r--   2 root  wheel  1014 Dec  4  2012 .cshrc
    -rw-r--r--   2 root  wheel   256 Dec  4  2012 .profile
    -r--r--r--   1 root  wheel  6203 Dec  4  2012 COPYRIGHT
    drwxr-xr-x   2 root  wheel    46 Dec  4  2012 bin
    drwxr-xr-x   7 root  wheel    39 Dec  4  2012 boot
    dr-xr-xr-x   2 root  wheel     2 Dec  4  2012 dev
    drwxr-xr-x  20 root  wheel    94 Dec  4  2012 etc
    drwxr-xr-x   3 root  wheel    48 Dec  4  2012 lib
    drwxr-xr-x   3 root  wheel     4 Dec  4  2012 libexec
    drwxr-xr-x   2 root  wheel     2 Dec  4  2012 media
    drwxr-xr-x   2 root  wheel     2 Dec  4  2012 mnt
    dr-xr-xr-x   2 root  wheel     2 Dec  4  2012 proc
    drwxr-xr-x   2 root  wheel   142 Dec  4  2012 rescue
    drwxr-xr-x   2 root  wheel     6 Dec  4  2012 root
    drwxr-xr-x   2 root  wheel   130 Dec  4  2012 sbin
    lrwxr-xr-x   1 root  wheel    11 Dec  4  2012 sys -> usr/src/sys
    drwxrwxrwt   2 root  wheel     2 Dec  4  2012 tmp
    drwxr-xr-x  14 root  wheel    14 Dec  4  2012 usr
    drwxr-xr-x  23 root  wheel    23 Dec  4  2012 var
  • The pool status:
    Code:
    zpool status super
      pool: super
     state: ONLINE
      scan: none requested
    config:
    
            NAME        STATE     READ WRITE CKSUM
            super       ONLINE       0     0     0
              mirror-0  ONLINE       0     0     0
                md1p2   ONLINE       0     0     0
                md2p2   ONLINE       0     0     0
    
    errors: No known data errors
  • The pool input/output statistics:
    Code:
    zpool iostat super
                   capacity     operations    bandwidth
    pool        alloc   free   read  write   read  write
    ----------  -----  -----  -----  -----  -----  -----
    super        616M  1.38G      0      1     64  34.2K
  • listing of the ZFS data sets:
    Code:
    zfs list
    NAME                 USED  AVAIL  REFER  MOUNTPOINT
    super                880M  1.09G   144K  none
    super/ROOT           615M  1.09G   144K  none
    super/ROOT/default   614M  1.09G   614M  /mnt
    super/swap           264M  1.35G    72K  -
    -------------end of install --------------------

Post-installation

Initializing /boot/loader.conf

Code:
# make loader.conf
echo 'zfs_load=YES'                          >> /mnt/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:super/ROOT/default"' >> /mnt/boot/loader.conf
=========== loader.conf =========================
zfs_load=YES
vfs.root.mountfrom="zfs:super/ROOT/default"
-------------end of loader.conf --------------------
To prevent errors at boot time an empty /etc/fstab is needed.
Code:
# make fstab
# ---- create empty /etc/fstab file
touch /mnt/etc/fstab
-------------end of fstab --------------------
Enable ZFS in /etc/rc.conf.

Code:
#  make rc.conf
cat /tmp/template_rc.conf     >> /mnt/etc/rc.conf
cat: /tmp/template_rc.conf: No such file or directory
*** [rc.conf] Error code 1
We fix this error and simply redo.

Code:
# cp template_rc.conf /tmp
# make rc.conf
cat /tmp/template_rc.conf     >> /mnt/etc/rc.conf
echo 'zfs_enable=YES'       >> /mnt/etc/rc.conf
echo '# ------------------' >> /mnt/etc/rc.conf
=========== rc.conf =========================
# ------ rc.conf
#defaultrouter=192.168.222.10
hostname='althusser.utp.xnet'
network_interfaces='bge0'
ifconfig_bge0="SYNCDHCP"

# --- daemons
inetd_enable=NO
sshd_enable=yes
sendmail_enable=yes

# --- time synchronization
ntpd_enable=YES
#openntpd_enable=YES

zfs_enable=YES
# ------------------
Copy the ZFS cache file to /boot/zfs

Code:
#  make zfs_boot
if  [ ! -d /mnt/boot/zfs ] ; then echo Cannot copy /tmp/zpool.cache to /mnt/boot/zfs ! ; exit 2 ; fi
cp -p /tmp/zpool.cache /mnt/boot/zfs/
ls -l /mnt/boot/zfs
total 1
-rw-r--r--  1 root  wheel  1488 Aug  5 17:56 zpool.cache
-------------end of zfs_boot --------------------
Unmount

Code:
# make zfs_umount
zfs unmount -a
zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
super                925M  1.05G   144K  none
super/ROOT           659M  1.05G   144K  none
super/ROOT/default   659M  1.05G   659M  /mnt
super/swap           264M  1.31G    72K  -
------
mount
/dev/ada0s3a on / (ufs, local, noatime, soft-updates)
devfs on /dev (devfs, local, multilabel)
tmpfs on /tmp (tmpfs, local)
-------------end of zfs_umount --------------------
Set the mountpoint to legacy

Code:
# make mountpoint
ls -l /tmp
-rw-r--r--  1 root   wheel   261 Aug  5 23:35 template_rc.conf
-rw-r--r--  1 root   wheel  1488 Aug  5 17:56 zpool.cache
zfs set mountpoint=legacy super/ROOT/default
zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
super                925M  1.05G   144K  none
super/ROOT           659M  1.05G   144K  none
super/ROOT/default   659M  1.05G   659M  legacy
super/swap           264M  1.31G    72K  -
Setup is finished ............
----------------- done ! ---------------------------------
you can now reboot into your new ZFS only system now .....
----------------------------------------------------------
Or you could finish the elementary setup with:

1. Remount your ZFS file system    : mount -t zfs super/ROOT/default /mnt
2. Chroot into /mnt                : chroot /mnt
3. Change the root password        : passwd
4. Create the /etc/mail/aliases.db : newaliases
5. Set up your time zone           : tzsetup

6. Exit the chroot                 : exit
7. Unmount                         : umount /mnt
-------------end of mountpoint --------------------
That's it. A FreeBSD ZFS only system suitable for managing boot environments.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 5th August 2013 at 10:08 PM.
Reply With Quote
  #4   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

One of the first versions of the makefile without all the whistles and bells like the 4K alignment stuff, and variables:

Code:
DISKS   = /dev/ada1 /dev/ada2
DISKS   = /dev/md1 /dev/md2

partition:
.for X in ${DISKS}
        if gpart show ${X} ; then gpart destroy -F ${X} ; fi
        gpart create -s GPT ${X}
        NUMBER=$$( echo ${X} | tr -c -d '0-9' ) ;\
        gpart add -t freebsd-boot -l bootcode$${NUMBER} -s 128k ${X} ;\
        gpart add -t freebsd-zfs -l sys$${NUMBER} ${X}
        gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ${X}
.endfor

pool:
        zpool create -f -o cachefile=/tmp/zpool.cache sys mirror /dev/gpt/sys*
        zpool status
        zpool list

options:
        zfs set mountpoint=none sys
        zfs set checksum=fletcher4 sys
        zfs set atime=off sys
        zfs create sys/ROOT
        zfs create -o mountpoint=/mnt sys/ROOT/default
        zpool set bootfs=sys/ROOT/default sys

# ================================================================================
pre_install: partition pool options

install:
        cd /usr/freebsd-dist ;\
        for X in base.txz kernel.txz; do \
        tar --unlink -xpJf $${X} -C /mnt ;\
        done
        @echo Current directory: $$(pwd)

post_install: zfs_boot loader.conf fstab rc.conf umount mountpoint

all: pre_install install post_install

# ================================================================================

zfs_boot:
        cp /tmp/zpool.cache /mnt/boot/zfs/

loader.conf:
        echo 'zfs_load=YES'                              >> /mnt/boot/loader.conf
        echo 'vfs.root.mountfrom="zfs:sys/ROOT/default"' >> /mnt/boot/loader.conf

fstab:
        # ---- create empty /etc/fstab file
        touch /mnt/etc/fstab

rc.conf:
        echo 'zfs_enable=YES' >> /mnt/etc/rc.conf

umount:
        zfs list
        zfs umount -a

mountpoint:
        zfs set mountpoint=legacy sys/ROOT/default
        zfs list
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 7th August 2013 at 01:43 PM.
Reply With Quote
  #5   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

Placeholder 4
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #6   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default Methods to get the Makefile to the FreeBSD live CD session

  • Download or copy the Makefile to an USB stick and copy from the stick to the /tmp directory of the FreeBSD live CD.
  • Create your own Live USB stick by doing a simple install on USB stick. Configure elementary network connectivity and ssh so you can use scp.

    Then you also have the advantage to add your favourite shell, running script(1) to produce and keep a log of your installation efforts.
  • Use the script from http://mfsbsd.vx.sk to create a more versatile FreeBSD live CD than the official FreeBSD installer.
  • Transfer the Makefile and the rc.conf template with nc(1) :
    • If you are on a network with DHCP you can issue an 'ifconfig' to see which network card you have:
      Code:
      bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
              options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,
                             VLAN_HWTSO,LINKSTATE>
              ether 98:4b:e1:10:08:6b
              nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
              media: Ethernet autoselect (1000baseT <full-duplex>)
              status: active
    • Then run dhclient:
      Code:
      # dhclient bge0
      DHCPREQUEST on bge0 to 255.255.255.255 port 67
      DHCPACK from 192.168.222.10
      bound to 192.168.222.240 -- renewal in 43200 seconds
    • Read the 'nc' man page:

      Code:
      DATA TRANSFER
           The example in the previous section can be expanded to build a basic data
           transfer model.  Any information input into one end of the connection
           will be output to the other end, and input and output can be easily cap-
           tured in order to emulate file transfer.
      
           Start by using nc to listen on a specific port, with output captured into
           a file:
      
                 $ nc -l 1234 > filename.out
      
           Using a second machine, connect to the listening nc process, feeding it
           the file which is to be transferred:
      
                 $ nc host.example.com 1234 < filename.in
      
           After the file has been transferred, the connection will close automati-
           cally.
      So on the machine with the FreeBSD live CD: # nc -l 1234 >Makefile
      On the second machine : $ nc 192.168.222.240 1234 <Makefile

      If you have to move multiple files, it will be easier to pack all these files into a single tar(1) archive first.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 6th August 2013 at 01:05 PM.
Reply With Quote
  #7   (View Single Post)  
Old 5th August 2013
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

The Makefile for downloading. Please rename it to Makefile:

# mv Makefile-ZFS-madness-1.18.txt Makefile
Attached Files
File Type: txt Makefile-ZFS-madness-1.18.txt (12.3 KB, 138 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 8th August 2013 at 12:31 PM.
Reply With Quote
Reply

Tags
4k disk, 4k sector alignment, advanced format disks, freebsd, make, makefile, zfs, zfs madness, zfs root

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
how to remove boot sector viruses? CyberJet OpenBSD General 8 15th July 2011 07:50 PM
HOWTO: Modern FreeBSD Install (vermaden way) vermaden Guides 10 11th July 2011 07:05 AM
${REINPLACE_CMD} on Makefile indiocolifa FreeBSD Ports and Packages 3 26th March 2009 11:16 PM
Makefile Problem BadWolf Programming 4 15th March 2009 01:58 PM
ports config and makefile scripting boincv FreeBSD Ports and Packages 6 1st October 2008 07:57 AM


All times are GMT. The time now is 02:42 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick