Large MFS filesystems

[jggimi]

MFS filesystems are ramdisk filesystems, and common to all BSDs. They are backed up by swap space, so that high use sections stay in RAM, and receive RAM performance, but low-use sections receive disk drive performance. They are commonly used for /tmp or other temporary uses. They are limited in size to MAXDSIZ, which is 1GB on OpenBSD.

When building OpenBSD-current, I like to mount an MFS filesystem for /usr/obj, to shave some minutes off of build time.

Until this week, I have been able to fit all of /usr/obj within a single 1G MFS filesystem. I can still do so for building userland, but making a release(8) fails, at it requires even more space in /usr/obj than can fit.

I cannot increase MAXDSIZ, as that will break many other components of the OS. So I conceived this workaround:

1. Mount two MFS filesystems
2. Create a large sparse file on each
3. Configure the files as vnodes
4. Concatenate the vnodes as a CCD device
5. Create and mount a partition on the CCD device as /usr/obj

It may seem complicated, but ... it really isn't. Each step is well understood and well documented, and the entire sequence is easily scripted.

The mount_mfs(8) command is used to create the MFS filesystems. The key option is "-s" which is used to set the number of 512-byte sectors.

Vnodes are how filesystems are addressed. The userland tool to manipulate them is vnconfig(8). It is most often used to mount CD or hard disk images. It is used here to make the large (empty) files we created in ramdisk act as devices, for concatenation into a single CCD device.

CCD devices are included in OpenBSD's GENERIC kernel. They can do simple concatenation (RAID0) or mirroring (RAID1-like). In this case, I select simple concatenation. Note that I leave the entire first cylinder (63x255 sectors, about 7.8 MB) empty on each vnode device and the CCD device, for CCD management information.

Here's my build-userland shell script, which shows the sequence:

1. Create mount points
2. Mount MFS filesystems
3. Create files
4. Create vnodes
5. Create partitions in the vnodes for use by CCD
6. Configure the CCD device
7. Create a partition in the CCD device
8. Format it
9. Mount it

#!/bin/sh#
# kernel should be manually built first, then system rebooted, before
# executing this script. 
#

# make sure we're root:
test -O /etc/passwd || echo run as superuser 
test -O /etc/passwd || exit
 
# create ramdisk /usr/obj -- it is bigger than max MFS space: 
mkdir /tmp/j1 /tmp/j2
mount_mfs -o noatime,async -s 1500000 swap /tmp/j1
mount_mfs -o noatime,async -s 1500000 swap /tmp/j2
dd if=/dev/zero of=/tmp/j1/1 bs=1m seek=670 count=0
dd if=/dev/zero of=/tmp/j2/2 bs=1m seek=670 count=0
vnconfig svnd1 /tmp/j1/1
vnconfig svnd2 /tmp/j2/2
printf 'a a\n16065\n\nccd\nw\nq' | disklabel -E svnd1
printf 'a a\n16065\n\nccd\nw\nq' | disklabel -E svnd2
ccdconfig ccd0 0 0 /dev/svnd1a /dev/svnd2a
printf 'a a\n16065\n\n\nw\nq' | disklabel -E ccd0
newfs ccd0a
mount -o async,noatime /dev/ccd0a /usr/obj

# make userland
cd /usr/src
nice make obj
cd /usr/src/etc && env DESTDIR=/ nice make distrib-dirs
cd /usr/src
nice make build

After making the release, a separate step, I can unmount /usr/obj, unconfigure the CCD device, unconfigure the vnodes, unmount the MFS filesystems, and rmdir the two mount points.

[jggimi]

I've updated the guide to eliminate sparse files, which don't always work properly with CCD.

[jggimi]

Guide updated.

Further testing showed sparse files work, but additional space was needed to house CCD metadata.