View Single Post
  #1   (View Single Post)  
Old 14th January 2010
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,056
Post HOWTO: Modern FreeBSD Install (vermaden way)

All these years sysinstall(8) was helping us to install FreeBSD with most options we needed, today with new filesystems/features like GJournal/ZFS/Geli/GMirror/GStripe its no longer up to the task, because it only supports creating installation on UFS filesystem with SoftUpdates turned ON or OFF.

In this guide you will learn how to setup FreeBSD installation in simple yet flexible setup based on read-only UFS (without SoftUpdates) for 'base system' [1], some SWAP space, /tmp mounted on SWAP and all the other filesystems (/var /usr ...) mounted on ZFS. It will not require rebuilding anything, just simple setup on plain MBR partitions. I should also mention that we would be using AHCI mode for disks. I also provided two versions, for system with one harddisk and with three of them for redundant setup.

Here is the layout of the system with 1 harddisk:
Code:
MBR SLICE 1 |    / | 512 MB | UFS/read-only 
            | SWAP |   2 GB |
            | /tmp | 512 MB | mounted on SWAP with mdmfs(8)
------------+------+---------------------------------------
MBR SLICE 2 | /usr |   REST | ZFS dataset
            | /var |   REST | ZFS dataset
... and here layout for single disk for system with 3 disks:
Code:
MBR SLICE 1 |    / | 512 MB | UFS/read-only 
------------+------+--------+------------------------------
MBR SLICE 2 | SWAP |   1 GB |
            | /tmp | 512 MB | mounted on SWAP with mdmfs(8)
------------+------+--------+------------------------------
MBR SLICE 3 | /usr |   REST | ZFS dataset
            | /var |   REST | ZFS dataset
Redundancy planning for system with 3 disks:
Code:
 [ DISK0 ]           [ DISK1 ]           [ DISK2 ]
 [   /   ] < RAID1 > [   /   ] < RAID1 > [   /   ]
 [ SWAP0 ]           [ SWAP1 ]           [ SWAP2 ]
 [   Z   ] < RAID5 > [   F   ] < RAID5 > [   S   ]
FreeBSD core, the 'base system' [1] should remain almost unchanged/untouched on daily basis while you can mess all other filesystems, this ensures that when things go wrong, you will be able to fix anything still having working 'base system' [1].

You will need *-dvd-* disk or *-memstick-* image for this installation, *-disk1-* will not do since it does not contain livefs system.

Here is the procedude, described as simple as possible.

1.0. I assume that our disk for the installation would be /dev/ad0 (/dev/ad0 /dev/ad1 /dev/ad2 for system with 3 disks)

1.1. Boot *-dvd-* from DVD disk or *-memstick-* image from pendrive
Code:
Country Selection --> United States
Fixit --> CDROM/DVD (*-dvd-*) or USB (*-memstick-*)
1.2. Create your temporary working environment
Code:
fixit# /mnt2/bin/csh
# setenv PATH /mnt2/rescue:/mnt2/usr/bin:/mnt2/sbin
# set filec
# set autolist
# set nobeep
1.3. Load needed modules
Code:
fixit# kldload /mnt2/boot/kernel/geom_mbr.ko
fixit# kldload /mnt2/boot/kernel/opensolaris.ko
fixit# kldload /mnt2/boot/kernel/zfs.ko
1.4. Create/mount needed filesystems
Code:
DISKS: 3                                               | DISKS: 1
# cat > part << __EOF__                                | # cat > part << __EOF__
p 1 165 63  512M                                       | p 1 165 63  2560M
p 2 165  * 1024M                                       | p 2 159  *     *
p 3 159  *     *                                       | p 3   0  0     0
p 4   0  0     0                                       | p 4   0  0     0
a 1                                                    | a 1
__EOF__                                                | __EOF__
                                                       |
# fdisk -f part ad0                                    | # fdisk -f part ad0
# fdisk -f part ad1                                    |
# fdisk -f part ad2                                    |
                                                       |
# kldload /mnt2/boot/kernel/geom_mirror.ko             |
# gmirror label  rootfs ad0s1                          |
# gmirror insert rootfs ad1s1                          |
# gmirror insert rootfs ad2s1                          |
                                                       |
# bsdlabel -B -w /dev/mirror/rootfs                    | # cat > label << __EOF__
                                                       | # /dev/ad0s1:
                                                       | 8 partitions:
                                                       |   a: 512m  0 4.2BSD
                                                       |   b: *     * swap
                                                       | __EOF__
                                                       |
                                                       | # bsdlabel -B -w ad0s1
                                                       | # bsdlabel       ad0s1 | tail -1 >> label
                                                       | # bsdlabel -R    ad0s1 label
                                                       |
# glabel label swap0 ad0s2                             | # glabel label rootfs ad0s1a
# glabel label swap1 ad1s2                             | # glabel label swap   ad0s1b
# glabel label swap2 ad2s2                             |
                                                       |
# newfs /dev/mirror/rootfsa                            | # newfs /dev/label/rootfs
# zpool create basefs raidz ad0s3 ad1s3 ad2s3          | # zpool create basefs ad0s2
# zfs create basefs/usr                                | # zfs create basefs/usr
# zfs create basefs/var                                | # zfs create basefs/var
# mkdir /NEWROOT                                       | # mkdir /NEWROOT
# mount /dev/mirror/rootfsa /NEWROOT                   | # mount /dev/label/rootfs /NEWROOT
# zfs set mountpoint=/NEWROOT/usr basefs/usr           | # zfs set mountpoint=/NEWROOT/usr basefs/usr
# zfs set mountpoint=/NEWROOT/var basefs/var           | # zfs set mountpoint=/NEWROOT/var basefs/var
1.5. Actually install needed FreeBSD sets
Code:
# setenv DESTDIR /NEWROOT
# cd /dist/8.0-RELEASE

# cd base
# ./install.sh (answer 'y' here)
# cd ..

# cd manpages
# ./install.sh
# cd ..

# cd kernels
# ./install.sh generic
# cd ..

# cd /NEWROOT/boot
# rm -r kernel
# mv GENERIC kernel
1.6. Provide basic configuration needed to boot new system
1.6.1.
Code:
DISKS: 3                                          | DISKS: 1
# cat > /NEWROOT/etc/fstab << __EOF__             | # cat > /NEWROOT/etc/fstab << __EOF__
#dev                #mount #fs  #opts #dump #pass | #dev              #mount #fs  #opts #dump #pass
/dev/mirror/rootfsa /      ufs  rw    1     1     | /dev/label/rootfs /      ufs  rw    1     1
/dev/label/swap0    none   swap sw    0     0     | /dev/label/swap   none   swap sw    0     0
/dev/label/swap1    none   swap sw    0     0     | __EOF__
/dev/label/swap2    none   swap sw    0     0     |
__EOF__                                           |
                                                  |
# cat > /NEWROOT/boot/loader.conf << __EOF__      | # cat > /NEWROOT/boot/loader.conf << __EOF__
zfs_load="YES"                                    | zfs_load="YES"
ahci_load="YES"                                   | ahci_load="YES"
geom_mirror_load="YES"                            | __EOF__
__EOF__                                           |
1.6.1.
Code:
# cat > /NEWROOT/etc/rc.conf << __EOF__
zfs_enable="YES"
__EOF__
1.7. Unmount filesystems and reboot
Code:
# cd /
# zfs umount -a
# umount /NEWROOT
# zfs set mountpoint=/usr basefs/usr
# zfs set mountpoint=/var basefs/var
# zpool export basefs
# reboot
Now lets talk things you will need to do after reboot.

2.0. At boot loader select boot into single user mode
4. Boot FreeBSD in single user mode

Code:
Enter full pathname of shell or RETURN for /bin/sh: /bin/csh
% /rescue/mount -w /
% /rescue/zpool import -D || /rescue/zpool import -f basefs
% exit
2.1. Login as root without password
Code:
login: root
password: (just hit ENTER)
2.2. Set root password
Code:
# passwd
2.3. Set hostname
Code:
# echo hostname=\"HOSTNAME\" >> /etc/rc.conf
2.4. Set timezone and date/time
Code:
# tzsetup
# date 201001142240
2.4.1 Generate the newaliases(8) database
Code:
# newaliases
2.5. Tune the ZFS filesystem (only for i386)
Code:
# cat > /boot/loader.conf << __EOF__
vfs.zfs.prefetch_disable=0      # enable prefetch
vfs.zfs.arc_max=134217728       # 128 MB
vm.kmem_size=536870912          # 512 MB
vm.kmem_size_max=536870912      # 512 MB
vfs.zfs.vdev.cache.size=8388608 #   8 MB
__EOF__
2.6. Mount /tmp on SWAP
Code:
# cat >> /etc/rc.conf << __EOF__
tmpmfs="YES"
tmpsize="512m"
tmpmfs_flags="-m 0 -o async,noatime -S -p 1777"
__EOF__
2.7. Move termcap into /etc (instead of useless link on crash)
Code:
# rm /etc/termcap
# mv /usr/share/misc/termcap /etc
# ln -s /etc/termcap /usr/share/misc/termcap
2.8. Add latest security patches
Code:
# freebsd-update fetch
# freebsd-update install
2.9. Make all changes to configuration in /etc, then set / to be mounted read-only in /etc/fstab
Code:
DISKS: 3                                           | DISKS: 1
 #dev                #mount #fs  #opts #dump #pass |  #dev              #mount #fs  #opts #dump #pass
+/dev/mirror/rootfsa /      ufs  ro    1     1     | +/dev/label/rootfs /      ufs  ro    1     1
-/dev/mirror/rootfsa /      ufs  rw    1     1     | -/dev/label/rootfs /      ufs  rw    1     1
 /dev/label/swap0    none   swap sw    0     0     |  /dev/label/swap   none   swap sw    0     0
 /dev/label/swap1    none   swap sw    0     0     |
 /dev/label/swap2    none   swap sw    0     0     |
2.10. Reboot and enjoy modern install of FreeBSD system
Code:
# shutdown -r now
To summarise, this setup provides us these things:
-- bulletproof 'base system' [1] on UFS (w/o SU) mounted read-only
-- /tmp filesystem mounted on SWAP
-- usage of new AHCI mode in FreeBSD
-- flexibility for all other filesystems on ZFS
-- fully working environment on crash (/etc/termcap)
-- disks/filesystems mounted by label, possible device name changes are harmless
-- RAID1 for / and RAID5 for all other systems on setup with 3 disks

[1] base system is / and /usr while this setup, in context of this setup I name 'base system'
the most important core of FreeBSD, the / filesystem and its binaries/libraries/configuration
(thanks to phoenix for reminding me what REAL base system is/means)


CHANGELOG

1.0 / 2010-01-14 / initial version
1.1 / 2010-01-15 / simplified PATH
+fixit# setenv PATH /mnt2/rescue:/mnt2/usr/bin
-fixit# setenv PATH /mnt2/bin:/mnt2/sbin:/mnt2/usr/bin:/mnt2/usr/sbin
1.2 / 2010-01-15 / added link for termcap (instead of duplicate on /etc and /usr) [2.6.]
.# rm /etc/termcap
+# mv /usr/share/misc/termcap /etc
+# ln -s /etc/termcap /usr/share/misc/termcap

-# cp /usr/share/misc/termcap /etc
1.3 / 2010-01-21 / removed unneeded mount commands [2.0.]
-# zfs mount basefs/var
-# zfs mount basefs/usr
1.4 / 2010-03-08 / added setup for 3 disks + cleanup
too much to fit here, we can as well call this new version RELOADED
1.5 / 2011-07-08 / added subsection: 2.4.1 Generate the newaliases(8) database





MIRROR THREAD: http://forums.freebsd.org/showthread.php?t=10334
POLISH VERSION: http://bsdguru.org/dyskusja/viewtopic.php?t=19392
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd

Last edited by vermaden; 8th July 2011 at 10:19 AM.
Reply With Quote