DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 9th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default Help getting Jail started...

Hello All,

= = = [ SOLUTION for this issue in thread post 8 & 12 below ] = = =
http://daemonforums.org/showthread.php?t=8660#post51524
-and updated:
https://www.amazon.com/FreeBSD-v10-J.../dp/099719460X

I attempted to create a Jail following the handbook section 15.5. This section is about using jail template and multiple jails. It seems dated.

I followed it putting all my config in /etc/rc.conf but when I tried to start the jail it said I needed to move config to /etc/jail.conf.

I figured out the syntax and created that file to start my jail "www".

Now I am getting the following error:

Code:
root@JAILHOST:/jpool/jails/j/mroot # service -v jail start
jail is located in /etc/rc.d
Starting jails:www: removed
.
root@JAILHOST:/jpool/jails/j/mroot # jls
   JID  IP Address      Hostname                      Path
root@JAILHOST:/jpool/jails/j/mroot #
Thanks for any help. Here is my /etc/fstab and /etc/jail.conf +:

Code:
root@JAILHOST:/jpool/jails/j/mroot # cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/gpt/swap0          none            swap    sw              0       0
/dev/gpt/swap1          none            swap    sw              0       0

/jpool/jails/j/mroot    /jpool/jails/j/www    nullfs  ro        0       0

/jpool/jails/js/www     /jpool/jails/j/www/s  nullfs  rw        0       0
Code:
root@JAILHOST:/jpool/jails/j/mroot # cat /etc/jail.conf
# file: /etc/jail.conf
# Defaults
##exec.prestart = "/sbin/mount -t nullfs -o ro /jpool/jails/j/mroot /jpool/jails/j/$name && /sbin/mount -t nullfs -o rw /jpool/jails/js/$name /jpool/jails/j/$name/s ";
  exec.start += "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
##exec.poststop = "/sbin/umount -f /jpool/jails/j/$name/s && /sbin/umount -f /jpool/jails/j/$name ";
  exec.clean;
  mount.devfs;
#mount.fstab = "/etc/fstab.$name";
  exec.consolelog = "/var/log/jail_"$name"_console.log";
  host.hostname = "Bens$name";
  allow.set_hostname = 0;
  allow.sysvipc = 0;
  allow.nomount;

# Dynamic wildcard parameter:
# Base the path off the jail name.
path = "/jpool/jails/j/$name";

# example {
#  interface = "lo0";
#  ip4.addr = 127.0.0.2;
#  }

 www {
  interface = "em0";
  ip4.addr += 192.168.15.100/24;
 }

# Generated by rc.d/jail at 2014-09-08 20:58:02
##www {
##        host.hostname = "BensWWW";
##        path = "/jpool/jails/j/www";
##        ip4.addr += "192.168.15.100/24";
##        allow.raw_sockets = 0;
##        exec.clean;
##        exec.system_user = "root";
##        exec.jail_user = "root";
##        exec.start += "/bin/sh /etc/rc";
##        exec.stop = "/bin/sh /etc/rc.shutdown";
##        exec.consolelog = "/var/log/jail_www_console.log";
##        mount.devfs;
##        allow.set_hostname = 0;
##        allow.sysvipc = 0;
##}
Code:
root@JAILHOST:/jpool/jails/j/mroot # uname -a
FreeBSD MonitoredSvr 10.0-RELEASE-p7 FreeBSD 10.0-RELEASE-p7 #0:
 Tue Jul  8 06:37:44 UTC 2014
     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
Code:
root@JAILHOST:/jpool/jails/j/mroot # sysctl -a | grep jail
security.jail.jailed: 0
security.jail.vnet: 0
security.jail.jail_max_af_ips: 255
security.jail.set_hostname_allowed: 1
security.jail.socket_unixiproute_only: 1
security.jail.sysvipc_allowed: 0
security.jail.allow_raw_sockets: 0
security.jail.chflags_allowed: 0
security.jail.mount_allowed: 0
security.jail.mount_devfs_allowed: 0
security.jail.mount_nullfs_allowed: 0
security.jail.mount_procfs_allowed: 0
security.jail.mount_tmpfs_allowed: 0
security.jail.mount_zfs_allowed: 0
security.jail.enforce_statfs: 2
security.jail.devfs_ruleset: 0
security.jail.param.jid: 0
security.jail.param.parent: 0
security.jail.param.name: 256
security.jail.param.path: 1024
security.jail.param.securelevel: 0
security.jail.param.enforce_statfs: 0
security.jail.param.devfs_ruleset: 0
security.jail.param.persist: 0
security.jail.param.dying: 0
security.jail.param.children.cur: 0
security.jail.param.children.max: 0
security.jail.param.host.: 0
security.jail.param.host.hostname: 256
security.jail.param.host.domainname: 256
security.jail.param.host.hostuuid: 64
security.jail.param.host.hostid: 0
security.jail.param.cpuset.id: 0
security.jail.param.ip4.: 0
security.jail.param.ip4.saddrsel: 0
security.jail.param.ip6.: 0
security.jail.param.ip6.saddrsel: 0
security.jail.param.allow.set_hostname: 0
security.jail.param.allow.sysvipc: 0
security.jail.param.allow.raw_sockets: 0
security.jail.param.allow.chflags: 0
security.jail.param.allow.quotas: 0
security.jail.param.allow.socket_af: 0
security.jail.param.allow.mount.: 0
security.jail.param.allow.mount.devfs: 0
security.jail.param.allow.mount.nullfs: 0
security.jail.param.allow.mount.procfs: 0
security.jail.param.allow.mount.tmpfs: 0
security.jail.param.allow.mount.zfs: 0

Last edited by bforest; 19th October 2016 at 05:25 PM. Reason: Solution: in thread post 8 & 12 and now workbook reference.
Reply With Quote
  #2   (View Single Post)  
Old 9th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

Also all my jail install directories are ZFS datasets. (and created as closely as I could to match the handbook section 15.5)

Code:
user@JailHost:~ % zfs list | grep jails
NAME                  USED  AVAIL  REFER  MOUNTPOINT
jpool/jails          2.59G  56.0G  45.3K  /jpool/jails
jpool/jails/j        2.59G  56.0G  1.89M  /jpool/jails/j
jpool/jails/j/mroot  2.59G  56.0G  2.59G  /jpool/jails/j/mroot
jpool/jails/js       1.92M  56.0G  41.3K  /jpool/jails/js
jpool/jails/js/www   1.88M  56.0G  1.88M  /jpool/jails/js/www

Last edited by bforest; 12th September 2014 at 01:17 AM.
Reply With Quote
  #3   (View Single Post)  
Old 10th September 2014
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

I never have played with jails, but many people use ezjail to automate the jail creation. Maybe you could give that a try
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #4   (View Single Post)  
Old 10th September 2014
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

I have a page on it that simplifies it a bit.

http://srobb.net/nullfsjail.html

Note, however, that if using FreeBSD-10, devfs needs to be mounted afterwards (this is explained on the page).
Reply With Quote
  #5   (View Single Post)  
Old 10th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

Thank you Scottro,

I think I will blow away my config from following the Handbook and try yours from scratch tomorrow.

-Ben
Reply With Quote
  #6   (View Single Post)  
Old 12th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Thumbs up

Hello Scottro,

Thanks for your assistance! I was able to build my first Jail using basically your instructions.

* I made small changes like changing all "mkdir..." to "zfs create..."
* Entries you asked to go into: /etc/fstab went to: /etc/fstab.www
- - then I just did: mount -aF /etc/fstab.www
- - I also put the "mount /dev" in this file
* Also my /etc/jail.conf includes a line: mount.fstab = "/etc/fstab.www" ;
- - that seemed to mount /dev automatically when starting the jail.

I am not sure but I think the issue with the handbook was in section 15.5.1. I think they missed the step: "make distribution DESTDIR=..."

I kept notes on each step I did. Going to try to write this up for straight v10 install on ZFS. I am on v10 so had to jump around a bit in your instructions.

Thank you so much for writing this up!

-Ben
Reply With Quote
  #7   (View Single Post)  
Old 13th September 2014
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

Trite though it sounds, hearing things like that make it worth the effort to write it up. When you do your own writeup, please either post it here or if it's on a website, post a link.
Reply With Quote
  #8   (View Single Post)  
Old 13th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

To follow is a complete set of instructions to setup Jails.

It is in two sections.

* Configuring the Multi-Jail Infrastructure [instead of HB-15.5.1]
* Installing a single new Jail . . . . . . . [instead of HB-15.5.2]

These instructions have been updated in:
- https://www.amazon.com/FreeBSD-v10-J.../dp/099719460X

As noted in previous posts, I had problems when following the instructions in the Handbook (15.5.1&2). I liked the way the Handbook structured the Jails but some steps are missing and info on /etc/jails.conf are missing.

Much thanks to Scottro for his doc that helped me to understand Jail config and enabling me to get this done.

Next steps will be to learn/document:

* How to back up a Jail.
* How to fine tune jails for specific applications.
* How to restore a jail to a different FBSDhost that is using the same Jail infrastructure.

Code:
Instructions for v10 Multi-JAIL configuration. 
Thanks in part to: http://srobb.net/nullfsjail.html and the FBSD Handbook

ASSUMPTIONS:  

 *  FreeBSD v10.x
 *  BuildWorld on Host is complete 
 *  Host is ZFS based filesystem
 *  cpdup is installed

CREATING MULTI-JAIL INFRASTRUCTURE

= First, create a directory structure for the read-only file system which will contain the FreeBSD 
  binaries for the jails. Then, change directory to the FreeBSD source tree and install the 
  read-only file system to the jail template:

     # su - root
     # zfs create jpool/jails
     # zfs create jpool/jails/j 
     # zfs create jpool/jails/j/mroot

     # setenv D /jpool/jails/j

     # mkdir -p $D/mroot
     # cd /usr/src
     # make installworld DESTDIR=$D/mroot
     # make distribution DESTDIR=$D/mroot

= Next, prepare a FreeBSD Ports Collection for the jails as well as a FreeBSD source tree, 
  which is required for mergemaster:

     # cd $D/mroot

     # mkdir $D/mroot/usr/ports
     # portsnap -p $D/mroot/usr/ports fetch extract
     # cpdup /usr/src $D/mroot/usr/src

= Create a skeleton for the read-write portion of the system:

     # zfs create jpool/jails/js
     # zfs create jpool/jails/j/skel

     # mkdir $D/skel/home $D/skel/usr-X11R6 $D/skel/distfiles
     # mv etc       $D/skel
     # mv usr/local $D/skel/usr-local
     # mv tmp       $D/skel
     # mv var       $D/skel
     # mv root      $D/skel

     # cp -p /etc/resolv.conf  $D/skel/etc  


= Prepare /etc/rc.conf in each new Jail for basics

     # ee $D/skel/etc/rc.conf

           hostname="CHANGEME"
           sshd_enable="YES"
           sendmail_enable="NO"
           dumpdev="NO"


= Use mergemaster to install missing configuration files. Then, remove the extra directories that mergemaster creates:

     # mergemaster -t $D/skel/var/tmp/temproot -D $D/skel -i         #(CHOOSE DEFAULTS)
     # cd $D/skel
     # rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev

= Now, symlink the read-write file system to the read-only file system. 
  Ensure that the symlinks are created in the correct s/ locations as the creation of directories 
  in the wrong locations will cause the installation to fail.

     # cd $D/mroot
     # mkdir s
     # ln -s s/etc etc
     # ln -s s/home home
     # ln -s s/root root
     # ln -s s/usr-local usr/local
     # ln -s s/usr-X11R6 usr/X11R6
     # ln -s s/distfiles usr/ports/distfiles
     # ln -s s/tmp tmp
     # ln -s s/var var

= Create a generic .../skel/etc/make.conf containing these lines:

     # ee $D/skel/etc/make.conf

          # This makes it possible to compile FreeBSD ports inside each jail. 
          # Remember that the ports directory is part of the read-only system. 
          # The custom path for WRKDIRPREFIX allows builds to be done in the read-write portion of every jail.

          WRKDIRPREFIX?=  /s/portbuild

= Enable Jails on Host

     # ee /etc/rc.conf          #(Add the following line)

          jail_enable="YES"


= As a last step, Create a jail.conf config file

     # ee /etc/jail.conf

          # file: /etc/jail.conf
          # Defaults
            exec.start += "/bin/sh /etc/rc";
            exec.stop = "/bin/sh /etc/rc.shutdown";
            exec.clean;
            mount.devfs;

            mount.fstab = "/etc/fstab.$name";
            exec.consolelog = "/var/log/jail_"$name"_console.log";
            host.hostname = "Jail$name";
            allow.set_hostname = 0;
          # allow.nomount;

          # Dynamic wildcard parameter:
          # Base the path off the jail name.
            path = "/jpool/jails/j/$name";

          # example {
          #  interface = "lo0";
          #  ip4.addr = 127.0.0.2;
          #  }
Code:
CREATING A NEW JAIL  (we will assume a jail named "www")


= Create /etc/fstab for the new Jail:

     # su - root
     # ee /etc/fstab.www

          # Device                Mountpoint      FStype  Options         Dump    Pass#
            /jpool/jails/j/mroot  /jpool/jails/j/www    nullfs  ro          0   0
            /jpool/jails/js/www   /jpool/jails/j/www/s  nullfs  rw          0   0


= Modify /etc/rc.conf  to allow alias IP(s) for Jail:

     # ee /etc/rc.conf          #(Add the following line (example IP))

            ifconfig_em0_alias0="inet 192.168.1.51/32"


= Modify /etc/jail.conf with info on the new jail:  

     # ee /etc/jail.conf      #(Add the following lines at bottom)      

            www {
                interface = "em0";
                ip4.addr += 192.168.1.51/24;  # (example IP)
                #allow.sysvipc = 1;
                }

= Create the required mount points for the read-only file system of the new jail:
  And Install the read-write template into each jail using sysutils/cpdup:

      # mkdir /jpool/jails/j/www
      # zfs create jpool/jails/js/www

      # cpdup /jpool/jails/j/skel /jpool/jails/js/www


CONTROLLING THE JAIL

# service jail start www       # (start the jail)
# jls                          # (List running jails and their ID's)
# jexec [ID#] tcsh             # (Log into Jails console on host)

Last edited by bforest; 19th October 2016 at 05:29 PM. Reason: Updated with link to workbook.
Reply With Quote
  #9   (View Single Post)  
Old 13th September 2014
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

Thank you! I will be trying this in the next week or so. (I haven't been using zfs, something else I've been planning to work with in the near future.)
Reply With Quote
Old 15th September 2014
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

By the way, do you have any objection to me linking to this thread on my jail page?
Reply With Quote
Old 15th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

Quote:
Originally Posted by scottro View Post
By the way, do you have any objection to me linking to this thread on my jail page?
Certainly not. I put it here to be referenced.

The reason I did not use ezjail was two reasons.

Quote:
1. Before I use an automated method, I want to do manual so I understand. I wanted to learn Jails.
2. My last crack at ezjail and qjail... it seemed they did not take advantage of ZFS.
I come from a background using Solaris Zones. The zone admin tools are fully engrained using ZFS. ZFS will allow taking snapshots of jails before making configuration changes or snapshots of the template(mroot) before an upgrade. ZFS even makes it easier to delete a jail.

Also I noticed I forgot a step in the second section and edited to correct it yesterday.

Code:
= Modify /etc/rc.conf  to allow alias IP(s) for Jail:
Bottom line, I hope someone will verify this works. I also need to verify that Handbook section 15.5.3 still works with this. If I have to change things there I will post the steps in this thread as well.

Next steps will be to learn/document:

* Verify FHB 15.5.3
* How to back up a Jail.
* How to fine tune jails for specific applications.
* How to restore a jail to a different FBSDhost that is using the same Jail infrastructure.
Reply With Quote
Old 16th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

I was working in my Jails today and I had some issues with accessing the R/W areas. I made some link changes that seemed to have corrected the issues:

Here are the changes (in the infrastructure section)

Code:
= Now, symlink the read-write file system to the read-only file system. 

     # cd $D/mroot
     # mkdir s
     # ln -s /s/etc       etc
     # ln -s /s/home    home
     # ln -s /s/root      root
     # ln -s /s/usr-local    usr/local
     # ln -s /s/usr-X11R6 usr/X11R6
     # ln -s /s/distfiles     usr/ports/distfiles
     # ln -s /s/tmp     tmp
     # ln -s /s/var      var
Basically I just changed the "s/..." to "/s/..." when creating the links.

- - - - - - -
Someone wrote me regarding this thread and suggested the following link:

http://jail-primer.sourceforge.net

I believe this is related to the: "qjail" port. There is some good info in this document, especially about /etc/jail.conf options. My only issue with the doc is that there is no reference to ZFS and having a jail configuration built on it.

I think Jails+ZFS has great advantages but I understand jails don't "need" ZFS.
Reply With Quote
Old 16th September 2014
gpatrick gpatrick is offline
Spam Deminer
 
Join Date: Nov 2009
Posts: 245
Default

You're going to a lot of work. This http://savagedlight.me/2014/03/07/fr...ocal-networks/ is a very succinct method to create Jails without having to install third party software.
Reply With Quote
Old 16th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

gpatrick:

The only thirdparty tool is "cpdup". It is what they used in the Handbook 15.5. I would rather us just "cp -p" or use tar to copy from one location to the other but not sure what cpdup does or why they used it in the handbook. Guess I didn't feel like fighting that battle at the time.

The method I lay out above is actually pretty simple. Initial configuration takes about 15 - 30 min. Adding an additional jail takes about 5 min. The part I didn't discuss that takes longer is the initial buildworld.

The link you provided does not seem to have any instructions for initial jail configuration. Just multiple local networks.

Last edited by bforest; 27th September 2014 at 06:51 PM.
Reply With Quote
Old 27th September 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default Backup's and Snapshots

Currently I have three jails running:

dns
mail
www

I have been working in www mostly and once I got nginx and drupal(including DB) working I needed to take a snapshot so if I goofed it up I could get back to this starting point.

I used the following command to backup the r/w portion of the jail:

Code:
su root -c "zfs snapshot jpool/jails/js/www@v7Drupal`date "+%Y%m%d-%H%M" ` "
you can get a list of your zfs snapshots using:

Code:
zfs list -t snapshot
I took this particular snapshot while the jail was running or not but probably best if you shut it down. Only takes a second and 0 space initially.

After loading a Theme and trying to run a Drupal helper called: Drush I ended up with a www jail that was broken. It took some work to get my drupal updated to v7 so rather than go thru all that again I just restored the snapshot.

Before restoring I needed to stop the jail. I have had some issues with my r/o mount not un-mounting automatically so I check to see if it is mounted and if so I force it to unmount:

Code:
   #   su - root -c "service jail stop www "
   #   mount | grep www
   #   su - root -c "umount -f /jpool/jails/j/www  "
   #   mount
Once that was done I just restored the snapshot to the last good point that I had available:
Code:
   #   zfs list -t snapshot
   #   su - root -c "zfs rollback jpool/jails/js/www@v7Drupal20140920-1841 "
   #   zfs list -t snapshot
   #   su - root -c "service jail start www "
   #   jls
When I started the jail it came up just like I had just updated to Drupal v7. Site was up and working like nothing happened, database and all.

Also, ZFS has "send/receive" commands that allow you to save a snapshot to an archive file or restore a snapshot from an archive file.

This facility should be good for not only archiving snapshots/backups but transferring them from one system to another.

I also snapshotted my "jpool/jails/j" which gets me my master and skeleton read only infrastructure.

Code:
user@JailHost:~ % zfs list -t snapshot
NAME                                       USED  AVAIL  REFER  MOUNTPOINT
jpool/jails/j@20140926-2015                   0      -  46.6K  -
jpool/jails/js/dns@20140919-1602           276K      -  2.04M  -
jpool/jails/js/mail@new20140924-1745       239K      -  2.22M  -
jpool/jails/js/www@20140916-1855          4.12M      -  32.5M  -
jpool/jails/js/www@20140919-1321          22.3M      -   701M  -
jpool/jails/js/www@v6Drupal20140922-1812  29.7M      -   780M  -
jpool/jails/js/www@v7Drupal20140922-2041  13.3M      -   808M  -
rpool/usr/src@20140904                     545M      -   545M  -
-Ben
Reply With Quote
Old 5th December 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default UPDATING

To follow is a complete set of instructions to update the ZFS based Jails we previously created.

It is in one section.

* Upgrading the Multi-Jail Infrastructure [instead of HB-15.5.3]

These instructions have been updated in:
- https://www.amazon.com/FreeBSD-v10-J.../dp/099719460X

As noted in earlier, I had problems when following the instructions in the Handbook (15.5.1,2&3). I liked the way the Handbook structured the Jails but some steps are missing. The same applies to this section. (mostly about Mergemaster)

I have performed this update about 3 times in the past few months.

Code:
Instructions for v10 UPDATING a Multi-FreeBSD-JAIL configuration.
Thanks in part to: http://srobb.net/nullfsjail.html and the FBSD Handbook

ASSUMPTIONS:

 *  FreeBSD v10.x
 *  BuildWorld on Host is complete (because the Jails are using same kernel as host) (Handbook 24.5; 24.6)
 *  Host is ZFS based filesystem
 *  cpdup is installed

UPDATING A MULTI-JAIL INFRASTRUCTURE

= First, create a directory structure for the read-only file system which will
  contain the FreeBSD binaries for the jails. Then, change directory to the FreeBSD
  source tree and install the read-only file system to the jail template:

     Log into Jail Host

     # su - root

     # setenv D /jpool/jails/j

     # zfs create jpool/jails/j/mroot2

     # mkdir -p $D/mroot2
     # cd /usr/src
     # make installworld DESTDIR=$D/mroot2
     # make distribution DESTDIR=$D/mroot2

= Next, prepare a FreeBSD Ports Collection for the jails as well as a FreeBSD source
  tree, which is required for mergemaster:

     # cd $D/mroot2
     # cpdup /usr/src $D/mroot2/usr/src

= Create a skeleton for the read-write portion of the system:  
   {Of course use your own date: skel.YYYYMMDD on the backup of old "skel"}

     # zfs rename jpool/jails/j/skel jpool/jails/j/skel.20141120
     # zfs create jpool/jails/j/skel

     # mkdir $D/skel/home $D/skel/usr-X11R6 $D/skel/distfiles
     # mv etc       $D/skel
     # mv usr/local $D/skel/usr-local
     # mv tmp       $D/skel
     # mv var       $D/skel
     # mv root      $D/skel



= Prepare /etc/rc.conf in each new Jail for basics

     # cp -p $D/skel.20141120/etc/rc.conf      $D/skel/etc
     # cp -p $D/skel.20141120/etc/make.conf    $D/skel/etc
     # cp -p $D/skel.20141120/etc/resolv.conf  $D/skel/etc



= Use mergemaster to install missing configuration files. Then, remove the extra
  directories that mergemaster creates:

     # mergemaster -t $D/skel/var/tmp/temproot -D $D/skel -i         #(CHOOSE DEFAULTS)
     # cd $D/skel
     # rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev

= Now, symlink the read-write file system to the read-only file system.
  Ensure that the symlinks are created in the correct /s/ locations as the
  creation of directories in the wrong locations will cause the installation to fail.

     # cd $D/mroot2
     # mkdir s
     # ln -s /s/etc             etc
     # ln -s /s/home            home
     # ln -s /s/root            root
     # ln -s /s/usr-local       usr/local
     # ln -s /s/usr-X11R6       usr/X11R6
     # ln -s /s/tmp             tmp
     # ln -s /s/var             var


= Now stop all running jails. Update Ports. Backup old mroot. Install new mroot.

     # service jail stop
     NOTE: make sure all jail mounts are unmounted.  

     # mv $D/mroot/usr/ports $D/mroot2/usr
     # portsnap -p $D/mroot2/usr/ports fetch update

   {Of course use your own date: mroot.YYYYMMDD}

     # cd $D
     # zfs rename jpool/jails/j/mroot  jpool/jails/j/mroot.20141120
     # zfs rename jpool/jails/j/mroot2 jpool/jails/j/mroot
     # exit
   Reboot the Host


= Adding temproot to each existing jail  {just hold enter key to leave everything for later}
  Log into Jail Host
     # su - root

     # mergemaster -t /jpool/jails/js/www/var/tmp/temproot  -D /tmp/ignore
     # mergemaster -t /jpool/jails/js/dns/var/tmp/temproot  -D /tmp/ignore
     # mergemaster -t /jpool/jails/js/mail/var/tmp/temproot -D /tmp/ignore


= TESTING
   Use jls to check if the jails started correctly.
   ssh into the jails (or jexec JID tcsh)
   Run "mergemaster -r" as root in each jail to update the configuration files.
    ( answer all with "i" )

Last edited by bforest; 19th October 2016 at 05:31 PM. Reason: Corrected MergeMaster command. Added link to workbook.
Reply With Quote
Old 5th December 2014
Oko's Avatar
Oko Oko is offline
Rc.conf Instructor
 
Join Date: May 2008
Location: Kosovo, Serbia
Posts: 1,102
Default

Quote:
Originally Posted by J65nko View Post
I never have played with jails, but many people use ezjail to automate the jail creation. Maybe you could give that a try
I would strongly suggest installing PC-BSD utilities sysutils/pcbsd-utils which contains Warden jail management tool. I am using it in production to manage dozen of Jails. I have heard good things about sysutils/cbsd. The second tool is interesting because the stated objective is unified interface to both Jails and Bhyve. Bhyve for now is very difficult to install. By the way both above ports require 10.xxx or above. I am running it on 10.1 in production.

Edit: I just learned from CBSD website about vmrc http://vmrc.bsd.lv/ and the name of its author Michael Dexter appears on many Bhyve threads. There is apparently another tool http://petitecloud.org/ which is mentioned in some Bhyve threads but I am not sure in what contexts.

Another interesting thing is that theoretically one could port Bhyve to other OS and I have seen few people daydreaming of porting it to OpenBSD

I would be interested about the state of Xen on NetBSD. It seems to be rotting at the moment and it is not nearly as useful as its Linux counterpart.

Last edited by Oko; 7th December 2014 at 08:58 PM. Reason: Adding additional information.
Reply With Quote
Old 5th December 2014
scottro's Avatar
scottro scottro is offline
Real Name: Scott Robbins
ISO Quartermaster
 
Join Date: Apr 2008
Location: NYC
Posts: 652
Default

Hrm, where's the thanks button? I wanted to thank Oko for that one. I haven't tried it and probably won't for awhile, but it's an interesting idea for us that I will definitely have to try soon.

Aha--a forum search shows me that the thanks button has been disabled, so I'll just say, Thanks Oko.
I'm going to read up on Warden and see if it would fit our needs. Ironically, at work, as I'm kind of the jail guy, a few people find it amusing to call me Warden.
Reply With Quote
Old 9th December 2014
bforest bforest is offline
Port Guard
 
Join Date: Aug 2008
Location: East Coast, USA
Posts: 32
Default

Quote:
Originally Posted by gpatrick View Post
You're going to a lot of work. This http://savagedlight.me/2014/03/07/fr...ocal-networks/ is a very succinct method to ...
. . . "have a FreeBSD jail host with multiple local networks"

Using the info at the one page link that GPatrick provided I was able to configure my jail host so that the Host is on my "Admin" network and the Jails are on the "General" network.

This is done by creating multiple routing tables and attaching them to specific network interfaces. These are also known as FIB, and are manipulated with the setfib utility.

Three files need to be configured: (Please visit the link for details)
- /boot/loader.conf
- /etc/rc.local
- /etc/jail.conf

I have two physical NIC's on my JailHost:
- em0 is my Admin network which the host communicates on.
- em1 is my General network which the Jails communicate on.

In my /etc/rc.conf I only has to reference the proper NIC em1:
Code:
ifconfig_em1_alias0="inet 192.168.25.100/32"  #www
ifconfig_em1_alias1="inet 192.168.25.120/32"  #mail
ifconfig_em1_alias2="inet 192.168.25.5/32"    #dns

My current Jail.conf looks something like this:
Code:
user@JailHost:~ % cat /etc/jail.conf

# file: /etc/jail.conf
# Defaults

  exec.start += "/bin/sh /etc/rc";
  exec.stop = "/bin/sh /etc/rc.shutdown";
  exec.clean;
  mount.devfs;

  mount.fstab = "/etc/fstab.$name";
  exec.consolelog = "/var/log/jail_"$name"_console.log";
  host.hostname = "Jail_$name";
  allow.set_hostname = 0;
# allow.nomount;

# Dynamic wildcard parameter:
# Base the path off the jail name.
  path = "/jpool/jails/j/$name";

  # example {
          #  interface = "lo0";
          #  ip4.addr = 127.0.0.2;
          #  }

    dns {
        exec.fib=1;  # Set to the GENERAL routing table
        interface = "em1";
        ip4.addr  = 192.168.25.5/24;
        #allow.sysvipc = 1;
        #allow.raw_sockets = 1;  # Debugging purposes
        }

    www {
        exec.fib=1;  # Set to the GENERAL routing table
        interface = "em1";
        ip4.addr  = 192.168.25.100/24;
        #allow.sysvipc = 1;
        #allow.raw_sockets = 1;  # Debugging purposes
        }

   mail {
        exec.fib=1;  # Set to the GENERAL routing table
        interface = "em1";
        ip4.addr  = 192.168.25.120/24;
        #allow.sysvipc = 1;
        #allow.raw_sockets = 1;  # Debugging purposes
        }

Last edited by bforest; 9th December 2014 at 01:03 AM.
Reply With Quote
Old 9th December 2014
Oko's Avatar
Oko Oko is offline
Rc.conf Instructor
 
Join Date: May 2008
Location: Kosovo, Serbia
Posts: 1,102
Default

Quote:
Originally Posted by bforest View Post
. . . "have a FreeBSD jail host with multiple local networks"
Where is the damn THANK YOU button?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
thinkpad x200 or similar owners pleas help getting started gosha General Hardware 1 14th July 2009 04:06 AM
Trying to get started translating OpenBSD Documentaion qmemo OpenBSD General 6 12th July 2009 12:50 PM
Pf Nat getting started ?? neurosis FreeBSD Security 11 16th November 2008 08:58 PM
Apache : httpd could not be started lalebarde General software and network 13 13th November 2008 11:51 PM
Getting started with DTrace tanked FreeBSD General 2 25th June 2008 09:21 AM


All times are GMT. The time now is 11:47 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick