We're planning on using a pair of 2 GB CF disks in a
gmirror(8) setup for /, with everything else (/usr, /usr/local, /tmp, /var, swap, /home ...) in ZFS. The server we're putting this into has 24 hot-swappable drive bays with 12x 400 GB SATA drives on one 3Ware 9550 RAID controller, and 12x 500 GB SATA drives on another 3Ware 9550 RAID controller (both setup as JBOD).
This is going to be our remote backups box (does rsync backups of all the major servers in the district, with ZFS snapshots done after each run).
The beauty of zpools is that you can add replace drives with larger capacity drives, and the zpool will start using all the extra space. Replace 1 drive per day with 1 TB drives, and two weeks later, we'll have doubled the storage space. What sucks about zpools (at least raidz ones) is that you can't add drives to the pool (so we can't start with 12 drives and expand to 24 down the road).
If it works as well as we think it will, and as well as our test setup has been working, then we're going to put in another identical box at a second remote location, and use the ZFS snapshot streaming feature to replicate the backups. That way, we'll have duplicate backups, in two separate locations, for all the major servers in the district.
All connected via gigabit fibre links!
Some days, I love my job!