![]() |
|
Guides All Guides and HOWTO's. |
![]() |
|
Thread Tools | Display Modes |
|
|||
![]()
1. Automating FreeBSD release downloads with a '.netrc' file
1.1 Introduction Many years ago for the first time, I did a ftp install of FreeBSD 4.x. using the two floppies. I instructed 'sysinstall' to retrieve the installation files from a FreeBSD ftp mirror on the internet. Previously I had installed FreeBSD a few times from CD. At that time I was using a new ISDN modem, which was fast with 64Kb, at least compared with the 14k4 old modem. Because the system did not boot, I must have made a mistake somewhere. Not looking forward to a couple of repeated ftp downloads with the ISDN modem, I decided to download the release files once and for all on a FreeBSD Pentium-Pro box. Some simple configuration steps turned the Pentium-Pro into a local ftp server. This method has become my favorite way to install FreeBSD as well as OpenBSD. First download the files making up the FreeBSD release to a ftp server in my local network. Boot 'sysinstall' with the install floppies, and install FreeBSD by fetching the installation files from the local ftp server. A nice side effect is speed. Transfer over the network turned out to be much faster than reading from a 10x speed CD on a Pentium I box. 1.2 File hierarchy of a FreeBSD release In the standardized ftp structure, we find the release files in '/pub/FreeBSD/releases/<arch>/<release>/', where <arch> stands for the hardware architecture like 'i386' or 'amd64', and <release> is a release tag, e.g. '7.2-RELEASE' or '8.0-RELEASE'. Code:
lrwxr-xr-x 1 1006 3000 1 May 1 2009 7.2-RELEASE -> . -r--r--r-- 1 1006 3000 4877 May 1 2009 ERRATA.HTM -r--r--r-- 1 1006 3000 3514 May 1 2009 ERRATA.TXT -r--r--r-- 1 1006 3000 186567 May 1 2009 HARDWARE.HTM -r--r--r-- 1 1006 3000 114717 May 1 2009 HARDWARE.TXT -r--r--r-- 1 1006 3000 19736 May 1 2009 README.HTM -r--r--r-- 1 1006 3000 14311 May 1 2009 README.TXT -r--r--r-- 1 1006 3000 60037 May 1 2009 RELNOTES.HTM -r--r--r-- 1 1006 3000 31407 May 1 2009 RELNOTES.TXT drwxrwxr-x 2 1006 3000 1024 May 1 2009 base drwxrwxr-x 2 1006 3000 512 May 1 2009 catpages -rw-r--r-- 1 1006 3000 25 May 1 2009 cdrom.inf drwxrwxr-x 2 1006 3000 512 May 1 2009 dict drwxrwxr-x 2 1006 3000 1536 May 1 2009 doc -r--r--r-- 1 1006 3000 3704 May 1 2009 docbook.css drwxrwxr-x 2 1006 3000 512 May 1 2009 floppies drwxrwxr-x 2 1006 3000 512 May 1 2009 games drwxrwxr-x 2 1006 3000 512 May 1 2009 info drwxrwxr-x 2 1006 3000 1024 May 1 2009 kernels drwxrwxr-x 2 1006 3000 512 May 1 2009 manpages lrwxr-xr-x 1 1006 3000 40 May 2 2009 packages -> ../../../ports/i386/packages-7.2-release drwxrwxr-x 2 1006 3000 512 May 1 2009 ports drwxrwxr-x 2 1006 3000 512 May 1 2009 proflibs drwxrwxr-x 2 1006 3000 2560 May 1 2009 src
Downloading all these files with the command line 'ftp' would be rather tedious. For a long time I used 'wget' to fetch the files. A couple of years later I discovered how to use a '.netrc' file to script ftp commands. 1.3 A '.netrc' file to download a FreeBSD release A full explanation of the '.netrc' tokens can be found in the ftp(1) man page. Code:
machine ftp2.dk.FreeBSD.org macdef init prompt off cd /pub/FreeBSD/releases/amd64/8.0-RELEASE mget ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf docbook.css $ getdir base catpages dict doc games info kernels manpages ports proflibs src ! ln -s . 8.0-RELEASE quit macdef getdir ! mkdir $i mget $i/* In the following invocation of the ftp command, the '-4' instructs the ftp client to only use IPv4 and not try IPv6, the 'a' option prescribes an 'anonymous' login. Code:
$ ftp -4a ftp://ftp2.dk.FreeBSD.org
1.4 Transcript of '.netrc' processing Code:
$ ftp -4a ftp://ftp2.dk.FreeBSD.org Connected to heset.dkuug.dk. 220 FTP server ready. 331 Guest login ok, send your email address as password. 230- Velcome to the DKUUG software archive. 230- If you encounter any problems please send mail to ftpadmin@dkuug.dk [snip] 230 Guest login ok, access restrictions apply. prompt off Interactive mode off. cd /pub/FreeBSD/releases/amd64/8.0-RELEASE 250 CWD command successful. mget ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf docbook.css local: ERRATA.HTM remote: ERRATA.HTM 150 Opening BINARY mode data connection for 'ERRATA.HTM' (4932 bytes). 226 Transfer complete. 4932 bytes received in 0.19 seconds (25.12 KB/s) [snip] local: docbook.css remote: docbook.css 150 Opening BINARY mode data connection for 'docbook.css' (3704 bytes). 226 Transfer complete. 3704 bytes received in 0.19 seconds (18.88 KB/s) Code:
$ getdir base catpages dict doc games info kernels lib32 manpages ports proflibs src ! mkdir base mget base/* local: base/CHECKSUM.MD5 remote: base/CHECKSUM.MD5 150 Opening BINARY mode data connection for 'base/CHECKSUM.MD5' (2163 bytes). 226 Transfer complete. 2163 bytes received in 0.19 seconds (11.04 KB/s) local: base/CHECKSUM.SHA256 remote: base/CHECKSUM.SHA256 150 Opening BINARY mode data connection for 'base/CHECKSUM.SHA256' (3703 bytes). 226 Transfer complete. 3703 bytes received in 0.19 seconds (18.93 KB/s) local: base/base.aa remote: base/base.aa 150 Opening BINARY mode data connection for 'base/base.aa' (1425408 bytes). 226 Transfer complete. 1425408 bytes received in 1.95 seconds (713.33 KB/s) [snip] local: base/install.sh remote: base/install.sh 150 Opening BINARY mode data connection for 'base/install.sh' (442 bytes). 226 Transfer complete. 442 bytes received in 0.20 seconds (2.20 KB/s) Code:
! mkdir src mget src/* local: src/CHECKSUM.MD5 remote: src/CHECKSUM.MD5 150 Opening BINARY mode data connection for 'src/CHECKSUM.MD5' (5758 bytes). 226 Transfer complete. 5758 bytes received in 0.20 seconds (28.05 KB/s) local: src/CHECKSUM.SHA256 remote: src/CHECKSUM.SHA256 150 Opening BINARY mode data connection for 'src/CHECKSUM.SHA256' (9678 bytes). 226 Transfer complete. 9678 bytes received in 0.19 seconds (49.59 KB/s) local: src/install.sh remote: src/install.sh 150 Opening BINARY mode data connection for 'src/install.sh' (942 bytes). 226 Transfer complete. 942 bytes received in 0.19 seconds (4.81 KB/s) local: src/sbase.aa remote: src/sbase.aa 150 Opening BINARY mode data connection for 'src/sbase.aa' (75919 bytes). 226 Transfer complete. 75919 bytes received in 0.21 seconds (349.05 KB/s) [snip] local: src/susbin.ac remote: src/susbin.ac 150 Opening BINARY mode data connection for 'src/susbin.ac' (120619 bytes). 226 Transfer complete. 120619 bytes received in 0.28 seconds (427.11 KB/s) local: src/susbin.inf remote: src/susbin.inf 150 Opening BINARY mode data connection for 'src/susbin.inf' (99 bytes). 226 Transfer complete. 99 bytes received in 0.20 seconds (0.48 KB/s) ! ln -s . 8.0-RELEASE quit 221 Goodbye $ 1.5 '.netrc' commands to download boot-only-iso image Code:
! mkdir iso lcd iso cd /pub/FreeBSD/ISO-IMAGES-amd64/8.0 mget 8.0-RELEASE-amd64-bootonly.iso CHECKSUM.MD5 CHECKSUM.SHA256 Code:
mget remote-files Expand the remote-files on the remote machine and do a get for each file name thus produced. ...... Files are transferred into the local working directory, which can be changed with `lcd directory'; new local directories can be created with `! mkdir directory' After having set the correct directory for both local and remote, the mget downloads the iso, as well as the two checksum files. A transcript: Code:
! mkdir iso lcd iso Local directory now /home/j65nko/FREEBSD/amd64_80/iso cd /pub/FreeBSD/ISO-IMAGES-amd64/8.0 250 CWD command successful. mget 8.0-RELEASE-amd64-bootonly.iso CHECKSUM.MD5 CHECKSUM.SHA256 local: 8.0-RELEASE-amd64-bootonly.iso remote: 8.0-RELEASE-amd64-bootonly.iso 150 Opening BINARY mode data connection for '8.0-RELEASE-amd64-bootonly.iso' (47366144 bytes). 226 Transfer complete. 47366144 bytes received in 60.97 seconds (758.67 KB/s) local: CHECKSUM.MD5 remote: CHECKSUM.MD5 150 Opening BINARY mode data connection for 'CHECKSUM.MD5' (351 bytes). 226 Transfer complete. 351 bytes received in 0.20 seconds (1.75 KB/s) local: CHECKSUM.SHA256 remote: CHECKSUM.SHA256 150 Opening BINARY mode data connection for 'CHECKSUM.SHA256' (526 bytes). 226 Transfer complete. 526 bytes received in 0.20 seconds (2.57 KB/s 1.6 A merge of the two '.netrc' files The boot-only-iso is fetched first, so the impatient already can burn the image file to CD, while the release files are still being downloaded. Code:
machine ftp2.dk.FreeBSD.org macdef init prompt off ! mkdir iso lcd iso cd /pub/FreeBSD/ISO-IMAGES-amd64/8.0 mget 8.0-RELEASE-amd64-bootonly.iso CHECKSUM.MD5 CHECKSUM.SHA256 lcd .. cd /pub/FreeBSD/releases/amd64/8.0-RELEASE mget ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf docbook.css $ getdir base catpages dict doc games info kernels lib32 manpages ports proflibs src ! ln -s . 8.0-RELEASE quit macdef getdir ! mkdir $i mget $i/* 1.7 The '.netrc' file generator script By only editing the shell variables SITE, RN, and ARCH, a different ftp mirror site, release or processor architecture can be chosen. The MY_ISO_DIR variable allows customization of the destination directory of the boot-only-iso, and it's checksum files. Code:
# -- start of user configurable settings #SITE=ftp4.freebsd.org #SITE=ftp2.dk.FreeBSD.org SITE=ftp.nl.freebsd.org RN='8.0' #ARCH=i386 ARCH=amd64 MY_ISO_DIR="ISO" # -- end of user configurable settings Code:
RELEASE="${RN}-RELEASE" DIR=/pub/FreeBSD/releases/${ARCH}/${RELEASE} FILE=${ARCH}_${RELEASE}files.txt # -------- bootonly.iso ----------------------- # ftp://ftp.nl.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/8.0/ # 8.0-RELEASE-i386-bootonly.iso ISO_DIR="/pub/FreeBSD/ISO-IMAGES-${ARCH}/${RN}" BOI="${RELEASE}-${ARCH}-bootonly.iso" MD5="CHECKSUM.MD5" SHA256="CHECKSUM.SHA256" Code:
$ sh -vx release-fetch # -- start of user configurable settings #SITE=ftp4.freebsd.org #SITE=ftp2.dk.FreeBSD.org SITE=ftp.nl.freebsd.org + SITE=ftp.nl.freebsd.org RN='8.0' + RN=8.0 #ARCH=i386 ARCH=amd64 + ARCH=amd64 MY_ISO_DIR="ISO" + MY_ISO_DIR=ISO # -- end of user configurable settings RELEASE="${RN}-RELEASE" + RELEASE=8.0-RELEASE DIR=/pub/FreeBSD/releases/${ARCH}/${RELEASE} + DIR=/pub/FreeBSD/releases/amd64/8.0-RELEASE FILE=${ARCH}_${RELEASE}files.txt + FILE=amd64_8.0-RELEASEfiles.txt # -------- bootonly.iso ----------------------- # ftp://ftp.nl.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/8.0/ # 8.0-RELEASE-i386-bootonly.iso ISO_DIR="/pub/FreeBSD/ISO-IMAGES-${ARCH}/${RN}" + ISO_DIR=/pub/FreeBSD/ISO-IMAGES-amd64/8.0 BOI="${RELEASE}-${ARCH}-bootonly.iso" + BOI=8.0-RELEASE-amd64-bootonly.iso MD5="CHECKSUM.MD5" + MD5=CHECKSUM.MD5 SHA256="CHECKSUM.SHA256" + SHA256=CHECKSUM.SHA256 The method used is not very well known, but is simple. Echo a ftp ls command to stdout. Have ftp receive this command through a pipe as standard input to execute. Code:
get_main_listing() { echo "ls ${DIR} ${FILE}" | ftp -4ai ${SITE} 1>&2 } get_main_listing Code:
echo "ls /pub/FreeBSD/releases/amd64/8.0-RELEASE \ amd64_8.0-RELEASEfiles.txt" | ftp -4ai ftp.nl.freebsd.org 1>&2 Code:
lrwxrwxrwx 1 500 450 1 Nov 23 07:53 8.0-RELEASE -> . -r--r--r-- 1 500 450 4932 Nov 21 15:10 ERRATA.HTM -r--r--r-- 1 500 450 3563 Nov 21 15:10 ERRATA.TXT -r--r--r-- 1 500 450 190670 Nov 21 15:10 HARDWARE.HTM -r--r--r-- 1 500 450 115402 Nov 21 15:10 HARDWARE.TXT -r--r--r-- 1 500 450 19867 Nov 21 15:10 README.HTM -r--r--r-- 1 500 450 14336 Nov 21 15:10 README.TXT -r--r--r-- 1 500 450 10512 Nov 21 15:10 RELNOTES.HTM -r--r--r-- 1 500 450 7545 Nov 21 15:10 RELNOTES.TXT drwxrwxr-x 2 500 450 4096 Nov 22 20:04 base drwxrwxr-x 2 500 450 4096 Nov 22 20:06 catpages -rw-r--r-- 1 500 450 25 Nov 21 15:10 cdrom.inf drwxrwxr-x 2 500 450 4096 Nov 22 20:06 dict drwxrwxr-x 2 500 450 4096 Nov 22 20:06 doc -r--r--r-- 1 500 450 3704 Nov 21 15:10 docbook.css drwxrwxr-x 2 500 450 4096 Nov 22 20:06 games drwxrwxr-x 2 500 450 4096 Nov 22 20:07 info drwxrwxr-x 2 500 450 4096 Nov 22 20:19 kernels drwxrwxr-x 2 500 450 4096 Nov 22 20:21 lib32 drwxrwxr-x 2 500 450 4096 Nov 22 20:23 manpages lrwxrwxrwx 1 500 450 41 Nov 23 07:53 packages -> ../../../ports/am d64/packages-8.0-release drwxrwxr-x 2 500 450 4096 Nov 22 20:32 ports drwxrwxr-x 2 500 450 4096 Nov 22 20:33 proflibs drwxrwxr-x 2 500 450 4096 Nov 22 20:59 src Code:
# select the release file directories SETS=$(awk '/^d/ { printf " %s", $9 }' ${FILE} ) # select the release documents DOCS=$(awk '/^-r/ { printf " %s" , $9 }' ${FILE} ) Code:
# select the release file directories SETS=$(awk '/^d/ { printf " %s", $9 }' ${FILE} ) + awk /^d/ { printf " %s", $9 } amd64_8.0-RELEASEfiles.txt + SETS= base catpages dict doc games info kernels lib32 manpages \ ports proflibs src # select the release documents DOCS=$(awk '/^-r/ { printf " %s" , $9 }' ${FILE} ) + awk /^-r/ { printf " %s" , $9 } amd64_8.0-RELEASEfiles.txt + DOCS= ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM \ README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf docbook.css Finally after having gathered all necessary information, the script can generate the '.netrc' file. Code:
# -- Create .netrc using 'here document' cat <<END_OF_NETRC machine ${SITE} macdef init prompt off ! mkdir ${MY_ISO_DIR} lcd ${MY_ISO_DIR} cd ${ISO_DIR} mget ${BOI} ${MD5} ${SHA256} lcd .. cd ${DIR} mget ${DOCS} $ getdir ${SETS} ! ln -s . ${RELEASE} quit macdef getdir ! mkdir \$i mget \$i/* END_OF_NETRC Code:
+ cat + << END_OF_NETRC machine ftp.nl.freebsd.org macdef init prompt off ! mkdir ISO lcd ISO cd /pub/FreeBSD/ISO-IMAGES-amd64/8.0 mget 8.0-RELEASE-amd64-bootonly.iso CHECKSUM.MD5 CHECKSUM.SHA256 lcd .. cd /pub/FreeBSD/releases/amd64/8.0-RELEASE mget ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf do cbook.css $ getdir base catpages dict doc games info kernels lib32 manpages ports proflibs src ! ln -s . 8.0-RELEASE quit macdef getdir ! mkdir $i mget $i/* Code:
# print Usage message on stderr cat <<END >&2 ------------------------------------------------------------- Remember that ftp(1) uses the HOME environment variable to locate .netrc Assuming you saved and/or edited .netrc in the current directory or '.', use it as follows: env HOME=. ftp -4a ftp://${SITE} 2>&1 | tee Logfile END + cat + << END + >&2 ------------------------------------------------------------- Remember that ftp(1) uses the HOME environment variable to locate .netrc Assuming you saved and/or edited .netrc in the current directory or '.', use it as follows: env HOME=. ftp -4a ftp://ftp.nl.freebsd.org 2>&1 | tee Logfile An excerpt from env(1): Code:
NAME env - set and print environment SYNOPSIS env [-i] [name=value ...] [utility [argument ...]] DESCRIPTION env executes utility after modifying the environment as specified on the command line. The option name=value specifies an environment variable, name, with a value of value. 1.8 Example of usage Code:
$ mkdir FBSD80 ; release-fetch >FBSD80/.netrc Trying 192.87.102.42... ------------------------------------------------------------- Remember that ftp(1) uses the HOME environment variable to locate .netrc Assuming you saved and/or edited .netrc in the current directory or '.', use it as follows: env HOME=. ftp -4a ftp://ftp.nl.freebsd.org 2>&1 | tee Logfile $ cd FBSD80 ; ls -l .netrc -rw-r--r-- 1 j65nko j65nko 501 Jan 16 20:45 .netrc Code:
$ env HOME=. ftp -4a ftp://ftp.nl.freebsd.org 2>&1 | tee Logfile Trying 192.87.102.42... Connected to ftp.nluug.nl. 220-Welcome to the FTP archive of SURFnet BV and 220-The Netherlands Unix Users Group (NLUUG). [snip] 331 Please specify the password. 230 Login successful. prompt off Interactive mode off. ! mkdir ISO lcd ISO Local directory now /home/j65nko/FREEBSD/FBSD80/ISO cd /pub/FreeBSD/ISO-IMAGES-amd64/8.0 250 Directory successfully changed. mget 8.0-RELEASE-amd64-bootonly.iso CHECKSUM.MD5 CHECKSUM.SHA256 local: 8.0-RELEASE-amd64-bootonly.iso remote: 8.0-RELEASE-amd64-bootonly.iso 150 Opening BINARY mode data connection for 8.0-RELEASE-amd64-bootonly.iso (47366144 bytes). 226 File send OK. 47366144 bytes received in 61.47 seconds (752.48 KB/s) [snip] 150 Opening BINARY mode data connection for src/susbin.inf (99 bytes). 226 File send OK. 99 bytes received in 0.20 seconds (0.49 KB/s) ! ln -s . 8.0-RELEASE quit 221 Goodbye. $Id: Ftp_download_FBSD_with_netrc.xml,v 1.6 2010/01/17 03:11:32 j65nko Exp $
$Id: vbul-html.xsl,v 1.15 2010/01/16 00:58:03 j65nko Exp $
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
|
|||
![]() Code:
#!/bin/sh # $Id: release-fetch,v 1.5 2010/01/16 20:13:08 j65nko Exp $ # fetch FreeBSD release files and boot-only-iso with ftp and .netrc # Copyright (c) 2006,2007,2008,2009 J65nko <Administrator daemonforums.org> # # Permission to use, copy, modify, and distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # # THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES # WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF # MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR # ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES # WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN # ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. # # -- start of user configurable settings #SITE=ftp4.freebsd.org #SITE=ftp2.dk.FreeBSD.org SITE=ftp.nl.freebsd.org RN='8.0' #ARCH=i386 ARCH=amd64 MY_ISO_DIR="ISO" # -- end of user configurable settings RELEASE="${RN}-RELEASE" DIR=/pub/FreeBSD/releases/${ARCH}/${RELEASE} FILE=${ARCH}_${RELEASE}files.txt # -------- bootonly.iso ----------------------- # ftp://ftp.nl.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/8.0/ # 8.0-RELEASE-i386-bootonly.iso ISO_DIR="/pub/FreeBSD/ISO-IMAGES-${ARCH}/${RN}" BOI="${RELEASE}-${ARCH}-bootonly.iso" MD5="CHECKSUM.MD5" SHA256="CHECKSUM.SHA256" # ----------------------------------------- # get_main_listing() { echo "ls ${DIR} ${FILE}" | ftp -4ai ${SITE} 1>&2 } get_main_listing # select the release file directories SETS=$(awk '/^d/ { printf " %s", $9 }' ${FILE} ) # select the release documents DOCS=$(awk '/^-r/ { printf " %s" , $9 }' ${FILE} ) # -- Create .netrc using 'here document' cat <<END_OF_NETRC machine ${SITE} macdef init prompt off ! mkdir ${MY_ISO_DIR} lcd ${MY_ISO_DIR} cd ${ISO_DIR} mget ${BOI} ${MD5} ${SHA256} lcd .. cd ${DIR} mget ${DOCS} $ getdir ${SETS} ! ln -s . ${RELEASE} quit macdef getdir ! mkdir \$i mget \$i/* END_OF_NETRC # print Usage message on stderr cat <<END >&2 ------------------------------------------------------------- Remember that ftp(1) uses the HOME environment variable to locate .netrc Assuming you saved and/or edited .netrc in the current directory or '.', use it as follows: env HOME=. ftp -4a ftp://${SITE} 2>&1 | tee Logfile END
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
|
|||
![]()
Downloads of the 'release-fetch' script and the guide in XHTML format
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
|
|||
![]()
When accessing one of the FreeBSD FTP services it is very common to get timed out before downloading of all the files is completed. Re-running the above shown .netrc will just start downloading from the beginning again.
The use of the mget command is what causes this. Changing mget to mreget will cause FTP to check the local host for the file and only download it if it's missing from the local host or it's date or size is different. This way you can use the same .netrc for the first execution and or to restart downloading files where your task was forced to end because the remote host timed you out. I issue this command on the user root console command line for the first execution and all restarts. ftp -v ftp2.jp.FreeBSD.org This is the corrected .netrc file I have in /root directory Code:
machine ftp2.jp.FreeBSD.org login anonymous password FBSD@home.com macdef init prompt off cd /pub/FreeBSD/releases/i386/8.0-RELEASE epsv4 off mreget ERRATA.HTM ERRATA.TXT HARDWARE.HTM HARDWARE.TXT README.HTM mreget README.TXT RELNOTES.HTM RELNOTES.TXT cdrom.inf docbook.css $ getdir base catpages dict doc games info kernels manpages ports proflibs src quit macdef getdir ! mkdir $i mreget $i/* The use of the “epsv4 off” statement. FTP defaults to using extended passive mode. I could not get FTP to function through my ipfilter firewall until I forced FTP to use native passive mode by adding the "epsv4 off" which turns off extended passive mode. Secondly I am using the mirror ftp2.jp.FreeBSD.org site. I tried a few other mirror FTP sites and was timeout after 3 to 5 minutes. This mirror has very little traffic and I got 80% done before timing out. Only one restart was necessary to complete the download. Recommend you try different mirrors located near your geographical location until you find one with low traffic. Last edited by J65nko; 11th February 2010 at 09:04 PM. Reason: Added [code] and [/code] tags to .netrc file |
|
|||
![]()
Re: epsv4 off
I also use epsv4 off this in the .netrc file of Automating OpenBSD snapshot downloads with a .netrc file and as well in Tracking OpenBSD snapshots with some simple shell scripts I just wonder why I did not use it in this particular one ![]() Re: interrrupted ftp transfers I cannot remember to ever had an interrupted ftp transfer from my favorite ftp.nluug.nl mirror. My rule for selecting ftp sites is: if there isn't a suitable in your own country, use one in a not so big country with a good infrastructure, like Denmark, Switzerland, Sweden, Norway etc. If I would have an interrupted ftp transfer using a .netrc I would simply remove the lines with the files already downloaded. But using mreget, as you suggested, is of course is a more pro-active solution. Unfortunately the OpenBSD ftp client only supports reget. And in case you wonder what the OpenBSD ftp client has to do with this. Well, I use a local OpenBSD ftp server to install both FreeBSD and OpenBSD from, so that is why. RE: location of .netrc Although ftp uses the HOME environment variable to locate .netrc, this can easily be overridden by using env as I pointed out in my guide. This works on both Open- and FreeBSD. Besides manipulating the HOME variable with env, the FreeBSD ftp client has two additional ways to select a .netrc
BTW I wonder why in your example you need to be root to ftp. Seen from a security viewpoint, a bad example ![]()
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
![]() |
Tags |
.netrc, .netrc macdef, .netrc macro, ftp scripting, mreget, reget |
Thread Tools | |
Display Modes | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Automating OpenBSD snapshot downloads with a .netrc file | J65nko | Guides | 1 | 7th January 2010 03:09 AM |
FreeBSD 8.0-RELEASE | Nightweaver | FreeBSD Installation and Upgrading | 10 | 10th December 2009 12:12 AM |
FreeBSD 7.2-RELEASE | vermaden | FreeBSD General | 6 | 12th May 2009 08:33 PM |
FreeBSD 6.2-RELEASE > 7.0-RELEASE Upgrade | Marci | FreeBSD Installation and Upgrading | 2 | 23rd July 2008 02:10 PM |
Automating ports update with portmaster | jaymax | FreeBSD Ports and Packages | 3 | 12th June 2008 06:56 AM |