DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

 
 
Thread Tools Display Modes
Prev Previous Post   Next Post Next
  #1   (View Single Post)  
Old 7th January 2010
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default Automating OpenBSD snapshot downloads with a .netrc file

While in http://www.daemonforums.org/showthread.php?t=670, another approach is used, this script uses a .netrc generated on the fly to do the complete download in a single login session.

The .netrc syntax is described in the ftp(1) man page.

Some examples
Code:
$ snapget
snapget
Destination directory: Snapshots
./snapget: Please specify one or more sets
The sets are: base x mini pxe floppy man misc iso
Selecting the 'mini' install sets as defined in the snapget script
Code:
$ snapget mini
Destination directory: Snapshots
-------------- Created .netrc-----------------------
     1  
     2  machine ftp.nluug.nl login anonymous password thanks@puffy.org
     3  
     4  macdef init
     5  prompt off
     6  epsv4 off
     7  preserve on
     8  get /pub/OpenBSD/snapshots/i386/SHA256 Snapshots/SHA256
     9  get /pub/OpenBSD/snapshots/i386/base46.tgz Snapshots/base46.tgz
    10  get /pub/OpenBSD/snapshots/i386/bsd Snapshots/bsd
    11  get /pub/OpenBSD/snapshots/i386/bsd.rd Snapshots/bsd.rd
    12  get /pub/OpenBSD/snapshots/i386/etc46.tgz Snapshots/etc46.tgz
    13  quit
    14  
------------------------------------------------------
Do you want to start 'ftp' with this '.netrc' ? (Y/N)
n
./snapget: I take this for a 'NO', aborting ...
The second line defines the ftp site, login name and password.

After an empty line a macro definition named 'init' is created. It contains some ftp commands. See the ftp man page for the details. Line 8 to 12 perform a ftp 'get' of the minimal number of filesets to install OpenBSD. By leaving out 'bsd.rd' it could be still be made smaller.

The 'init' macro will be executed immediately after the ftp server has authorized the ftp login. Because the macro ends with a 'quit' command the ftp session will terminate automatically.

Code:
$  snapget pxe flopp
Destination directory: Snapshots
./snapget : undefined set flopp
./snapget : aborting
The name of the file sets will be checked, and if an unknown file set is detected, the script will terminate with a message informing the user.

Normally a ".netrc' file is located in the home directory. Actually ftp uses the environment variable HOME to decide in which directory to look.
Code:
$ env | grep HOME
HOME=/home/j65nko
In order not to overwrite a possible existing .netrc the script stores the .netrc file in the directory defined in the ${DESTDIR{ variable.

By using env(1) utility, the value of HOME is changed into $DESTDIR}, and thus coaches ftp to locate and execute the right .netrc:
Code:
env HOME=${DESTDIR} ftp -4 ${SITE} 2>&1 | tee ${DESTDIR}/Logfile
A very simple illustration of using 'env', is to print the current date and time from another time zone by setting the TZ environment variable
Code:
# env  TZ=CET date    
Thu Jan  7 03:24:55 CET 2010

$env  TZ=EST date   
Wed Jan  6 21:25:09 EST 2010
A last example :
Code:
$snapget mini floppy
Destination directory: Snapshots
-------------- Created .netrc-----------------------
     1  
     2  machine ftp.nluug.nl login anonymous password thanks@puffy.org
     3  
     4  macdef init
     5  prompt off
     6  epsv4 off
     7  preserve on
     8  get /pub/OpenBSD/snapshots/i386/SHA256 Snapshots/SHA256
     9  get /pub/OpenBSD/snapshots/i386/base46.tgz Snapshots/base46.tgz
    10  get /pub/OpenBSD/snapshots/i386/bsd Snapshots/bsd
    11  get /pub/OpenBSD/snapshots/i386/bsd.rd Snapshots/bsd.rd
    12  get /pub/OpenBSD/snapshots/i386/etc46.tgz Snapshots/etc46.tgz
    13  get /pub/OpenBSD/snapshots/i386/floppy46.fs Snapshots/floppy46.fs
    14  get /pub/OpenBSD/snapshots/i386/floppyB46.fs Snapshots/floppyB46.fs
    15  get /pub/OpenBSD/snapshots/i386/floppyC46.fs Snapshots/floppyC46.fs
    16  quit
    17  
------------------------------------------------------
Do you want to start 'ftp' with this '.netrc' ? (Y/N)
y
Trying 192.87.102.43...
Connected to ftp.nluug.nl.
220-Welcome to the FTP archive of SURFnet BV and
220-The Netherlands Unix Users Group (NLUUG).
[snip]
331 Please specify the password.
230 Login successful.
prompt off
Interactive mode off.
epsv4 off
EPSV/EPRT on IPv4 off.
preserve on
Preserve modification times on.
get /pub/OpenBSD/snapshots/i386/SHA256 Snapshots/SHA256
local: Snapshots/SHA256 remote: /pub/OpenBSD/snapshots/i386/SHA256
227 Entering Passive Mode (192,87,102,43,199,230)
150 Opening BINARY mode data connection for /pub/OpenBSD/snapshots/i386/SHA256 (2162 bytes).
226 File send OK.
2162 bytes received in 0.19 seconds (10.90 KB/s)
get /pub/OpenBSD/snapshots/i386/base46.tgz Snapshots/base46.tgz
local: Snapshots/base46.tgz remote: /pub/OpenBSD/snapshots/i386/base46.tgz
227 Entering Passive Mode (192,87,102,43,239,73)
150 Opening BINARY mode data connection for /pub/OpenBSD/snapshots/i386/base46.tgz (50281346 bytes).
226 File send OK.
50281346 bytes received in 64.90 seconds (756.64 KB/s)
[snip]
get /pub/OpenBSD/snapshots/i386/floppyC46.fs Snapshots/floppyC46.fs
local: Snapshots/floppyC46.fs remote: /pub/OpenBSD/snapshots/i386/floppyC46.fs
227 Entering Passive Mode (192,87,102,43,222,187)
150 Opening BINARY mode data connection for /pub/OpenBSD/snapshots/i386/floppyC46.fs (1474560 bytes).
226 File send OK.
1474560 bytes received in 1.99 seconds (723.88 KB/s)
quit
221 Goodbye.
The retrieved files:
Code:
$ ls -l Snapshots/ 
total 134992
-rw-r--r--  1 j65nko  j65nko      3599 Jan  7 03:42 Logfile
-rw-r--r--  1 j65nko  j65nko      2162 Jan  5 22:59 SHA256
-rw-r--r--  1 j65nko  j65nko  50281346 Jan  5 22:58 base46.tgz
-rw-r--r--  1 j65nko  j65nko   7505333 Jan  5 22:58 bsd
-rw-r--r--  1 j65nko  j65nko   6239716 Jan  5 22:58 bsd.rd
-rw-r--r--  1 j65nko  j65nko    522035 Jan  5 22:58 etc46.tgz
-rw-r--r--  1 j65nko  j65nko   1474560 Jan  5 22:58 floppy46.fs
-rw-r--r--  1 j65nko  j65nko   1474560 Jan  5 22:58 floppyB46.fs
-rw-r--r--  1 j65nko  j65nko   1474560 Jan  5 22:58 floppyC46.fs
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
 

Tags
.netrc, ftp, openbsd snapshots

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
PHP read file contents - Maximum file size cksraj Programming 1 21st September 2009 11:38 AM
Best way to upgrade from -release to snapshot Carpetsmoker OpenBSD General 5 26th July 2009 08:51 PM
file:/// mfaridi FreeBSD Security 3 27th July 2008 02:18 PM
Automating ports update with portmaster jaymax FreeBSD Ports and Packages 3 12th June 2008 06:56 AM
Customizeable FTP auto-fetch script (OpenBSD snapshot download as example) J65nko Guides 2 1st June 2008 03:29 AM


All times are GMT. The time now is 01:27 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick