View Single Post
  #3   (View Single Post)  
Old 2nd December 2009
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

3 Installation and configuration

Installing the OpenBSD snapshot tracking utilities

For the installation 'root' privileges are not needed. To prevent a possible name clash with other utilities, installation will be done in a subdirectory of your home directory, and not in any of the '*bin*' file hierarchies.
  1. Download the attached tarball 'OBSDsnapshot_tracking-1.05.tgz' and place it in your home directory.
  2. Create a directory, for instance SNAP.

    Code:
    $ mkdir SNAP
  3. Unpack the 'tgz' file in the newly created directory.

    Code:
    $ tar xvzf OBSDsnapshot_tracking-1.05.tgz -C SNAP
    backup
    chk
    chk-latest
    differ
    fnice
    mk-netrc
    pkg-differ
    pkg-mirror-complete
    show
    Makefile
    rc.machines
  4. Change directory to SNAP.

    Code:
    $ cd SNAP ; ls -l
      1131 Dec  2 01:54 Makefile
       732 Oct 25 00:06 backup
       392 Dec 28  2008 chk
       710 Oct 29 01:33 chk-latest
      1409 Oct 23 03:20 differ
       174 Dec 29  2008 fnice
      2350 Oct 19 03:59 mk-netrc
      1214 Oct 19 04:00 pkg-differ
      1889 Oct 19 04:05 pkg-mirror-complete
      2645 Nov 29 04:01 rc.machines
      1822 Oct 19 04:20 show
  5. Run 'make' to setup a directory for the architecture(s), you are interested in.

    You specify subdirectories for each architecture you are interested in. There is no need to have these exactly match the official OpenBSD names, so for 'i386' you could use INTEL or I386. I prefer the upper-case version of the official names, which makes it easy to use the shell directory name completion feature.

    For the 'i386' architecture:

    Code:
    $ make ARCH=I386 install
    For 'i386', and 'amd64':

    Code:
    $ make ARCH='I386 AMD64' install
    
    mkdir -p I386 I386/NOW
    ln -sf NOW I386/PREV
    cp rc.machines I386
    ln -s  ../backup I386/backup
    ln -s  ../chk I386/chk
    ln -s  ../chk-latest I386/chk-latest
    ln -s  ../differ I386/differ
    ln -s  ../fnice I386/fnice
    ln -s  ../mk-netrc I386/mk-netrc
    ln -s  ../pkg-differ I386/pkg-differ
    ln -s  ../pkg-mirror-complete I386/pkg-mirror-complete
    ln -s  ../show I386/show
    mkdir -p AMD64 AMD64/NOW
    ln -sf NOW AMD64/PREV
    cp rc.machines AMD64
    ln -s  ../backup AMD64/backup
    ln -s  ../chk AMD64/chk
    ln -s  ../chk-latest AMD64/chk-latest
    ln -s  ../differ AMD64/differ
    ln -s  ../fnice AMD64/fnice
    ln -s  ../mk-netrc AMD64/mk-netrc
    ln -s  ../pkg-differ AMD64/pkg-differ
    ln -s  ../pkg-mirror-complete AMD64/pkg-mirror-complete
    ln -s  ../show AMD64/show
  6. Change directory to the architecture, that you wish to configure.

    After changing to the directory I386, you will see the following files and symbolic links:

    Code:
    $ cd I386 ; ls -l                                                                                                      
    drwxr-xr-x  2 j65  j65   512 Dec  2 02:03 NOW
    lrwxr-xr-x  1 j65  j65     3 Dec  2 02:03 PREV -> NOW
    lrwxr-xr-x  1 j65  j65     9 Dec  2 02:03 backup -> ../backup
    lrwxr-xr-x  1 j65  j65     6 Dec  2 02:03 chk -> ../chk
    lrwxr-xr-x  1 j65  j65    13 Dec  2 02:03 chk-latest -> ../chk-latest
    lrwxr-xr-x  1 j65  j65     9 Dec  2 02:03 differ -> ../differ
    lrwxr-xr-x  1 j65  j65     8 Dec  2 02:03 fnice -> ../fnice
    lrwxr-xr-x  1 j65  j65    11 Dec  2 02:03 mk-netrc -> ../mk-netrc
    lrwxr-xr-x  1 j65  j65    13 Dec  2 02:03 pkg-differ -> ../pkg-differ
    lrwxr-xr-x  1 j65  j65    22 Dec  2 02:03 pkg-mirror-complete -> ../pkg-mirror-complete
    -rw-r--r--  1 j65  j65  2645 Dec  2 02:03 rc.machines
    lrwxr-xr-x  1 j65  j65     7 Dec  2 02:03 show -> ../show
    • NOW : directory for storing most recent ftp mirror listings
    • PREV : symbolic link to the previous results. Because the first time there is no data from an earlier listings retrieval, it is simply linked to the NOW directory.
    • Symbolic links to the architecture independent scripts in the parent directory : 'backup', 'chk', 'chk-latest', 'differ', 'fnice', 'mk-netrc', 'pkg-differ', 'pkg-mirror-complete', 'show'.
    • The configuration file 'rc.machines'.
  7. Edit 'rc.machines'

    This file contains configuration settings for 'mk-netrc', the '.netrc' generator.

    Code:
    USER=anonymous
    PASSWORD='assimilated@borg.net'
    VERSION=snapshots
    
    DIR=/pub/OpenBSD        # standard directory shortcut
    
    MACHINES="
    ftp.openbsd.org::${DIR}
    ftp.eu.openbsd.org:stockholm:${DIR}
    ftp.wu-wien.ac.at:vienna:${DIR}
    "
    The 'USER' and 'PASSWORD' variables are used to create the 'login' and 'password' tokens. 'VERSION' and 'DIR' assist in setting up ftp directory paths like '/pub/OpenBSD/snapshots/i386/'.

    Variable 'MACHINES' defines records with three fields. The first is the ftp site name, needed for the 'machine' token of '.netrc'. Field number two is an identifier to name the listings retrieved from that particular site. The third and last field specifies the directory path leading to the OpenBSD section.

    The comments in 'rc.machines' also describe and explain the format.

    Code:
    # Separate fields with ':'
    #
    # - field 1: name of ftp site (without 'ftp://')
    #
    # - field 2: name of identifier to be appended to files.
    #            e.g. id 'vienna' will create: SHA256-vienna, latest-vienna AND latest_pkg-vienna
    #            EXCEPTION: 'ftp.openbsd.org' has no identifier!
    #
    #            Some examples:
    #
    #                   2162 Nov 28 06:43 SHA256
    #                   2162 Nov 27 00:33 SHA256-nluug
    #                   2162 Nov 28 06:43 SHA256-stockholm
    #                   2162 Nov 28 06:43 SHA256-vienna
    #                    981 Nov 29 00:58 latest
    #                    981 Nov 29 00:58 latest-nluug
    #                    981 Nov 29 00:58 latest-stockholm
    #                    981 Nov 29 00:58 latest-vienna
    #                 286617 Nov 29 00:58 latest_pkg
    #                 286617 Nov 29 00:58 latest_pkg-nluug
    #                 286617 Nov 29 00:58 latest_pkg-stockholm
    #                 300676 Nov 29 00:58 latest_pkg-vienna
    #
    # - field 3: path to OpenBSD directory on ftp site (normally /pub/OpenBSD) 
    
    # Some exceptions to the standard ftp directory layout: 
    #
    # ftp.freenet.de:freenet:/pub/ftp.openbsd.org/pub/OpenBSD
    # ftp.belnet.be:belnet:/packages/openbsd
    Edit the 'MACHINES' variable to select mirrors that are geographically or connection wise close to you. Use at least a couple of the second-level mirrors. For some architecture more then one snapshot a day is been created. Some 3rd level ftp mirrors are continuously lagging behind one or snapshots for the installation files.

    It is still useful to track these lagging mirrors for the packages, as these are not generated daily.
  8. Run 'mk-netrc' to create a '.netrc' file.

    After you have edited 'rc.machines' it is time to generate the '.netrc' file. 'ftp(1)' describes how this file can automate the ftp login and the retrieval of directory listings and files.

    In invoking 'mk-netrc' you have to use the official OpenBSD architecture names, using the proper case, as these are used as directory names in the ftp file lay-out.

    Code:
    $ ./make-netrc i386
    ./mk-netrc : sourcing rc.machines...
    machine ftp.openbsd.org login anonymous password assimilated@borg.net
    
    macdef init
    prompt off
    epsv4 off
    preserve on
    get /pub/OpenBSD/snapshots/i386/SHA256  SHA256
    ls  /pub/OpenBSD/snapshots/i386/ "| ./fnice  >latest"
    ls  /pub/OpenBSD/snapshots/packages/386/ "| ./fnice  >latest_pkg"
    quit
    
    machine ftp.eu.openbsd.org login anonymous password assimilated@borg.net
    
    macdef init
    prompt off
    epsv4 off
    preserve on
    get /pub/OpenBSD/snapshots/i386/SHA256  SHA256-stockholm
    ls  /pub/OpenBSD/snapshots/i386/ "| ./fnice  >latest-stockholm"
    ls  /pub/OpenBSD/snapshots/packages/i386/ "| ./fnice  >latest_pkg-stockholm"
    quit
    Compare the paths of the ftp get and ls commands, with the ones you see in your browser or ftp program. If it looks goods, redirect the output to '.netrc'. You do not have to worry about the 'sourcing rc.machines...' message. Because it is printed on standard error and not on standard out it will not redirected to '.netrc'.

    Code:
    $  ./make-netrc i386 >.netrc
    ./mk-netrc : sourcing rc.machines...
    We now pause for a security warning from the ftp man page:

    Code:
    Note that if this token (the password) is present in the .netrc
    file for any user other than anonymous, ftp will abort the auto-login
    process if the .netrc is readable by anyone besides the user.
    In our case the newly generated '.netrc' file only contains passwords for anonymous users, so ftp will not mind if '.netrc' is group and world readable. But as it is always good to acquire good habits:

    Code:
    $ chmod g=,o= .netrc
    Or:

    Code:
    $ chmod 600 .netrc ; ls -l .netrc
    -rw-------  1 j65  j65  1248 Dec  2 02:55 .netrc

At this point we have concluded our configuration of the snapshot tracking utilities for the 'i386' architecture. If do not intend to track snapshots for other architectures you are now ready to run 'chk'.

If you use other architectures, you still have to configure these. Each processor architecture needs it own '.netrc'. Assuming you used make ARCH="I386 AMD64 SPARC64" install, you will have to repeat steps 6 to 8, but this time in the directories AMD64 and SPARC64.



$Id: OBSDsnapshot-tracking.xml,v 1.14 2009/12/02 04:26:05 j65nko Exp $
$Id: book-vbul-html.xsl,v 1.3 2008/12/24 02:59:45 j65nko Exp $
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote