Script to update NetBSD using snapshots

[s0xxx]

Script could be helpful to those that are using snapshots to update their NetBSD systems. Script originally used 'coprocess' feature of ksh, but then the NetBSD folks changed the releng server from ftp to http. As I'm just too lazy to convert it to Bourne sh script, is still in ksh. Anyway, I expect opinions and recomendations to improve the script.

#!/bin/ksh 
#    getsets.ksh v.0.2 
#    dalibor Dot gudzic At gmail Dot com 
#    www.bsdserbia.org 
#    11/2009 
# 

DWNLD_DIR="/home/soxxx/netbsd_sets" 
FTP_SITE="http://nyftp.netbsd.org" 
FTP_OPTIONS="/usr/bin/ftp -o -" 
FTP_RELENG="pub/NetBSD-daily" 
TAG="netbsd-5"

VERSION="$(awk -F\' '/BUILDID/ { print $2 }' /etc/release)" 

set -A SETS "base.tgz comp.tgz etc.tgz kern-GENERIC.tgz man.tgz misc.tgz tests.tgz \ 
            text.tgz xbase.tgz xcomp.tgz xetc.tgz xfont.tgz xserver.tgz" 

f_using() { 
cat <<EOF 

Script execution: ${0##*/} <getsets | installkernel | installsets | etcupdate | help> 

    getsets          -    Downloading of sets from ftp server
    installkernel    -    Extract and install GENERIC kernel 
    installsets      -    Installing of sets 
    etcupdate        -    Update, merge of files in /etc dirctory 
    help             -    Show help on using script 

Apart from downloading of sets, to correctly execute the script start it as: 

    sudo ${0##*/} parameter 

EOF 
} 

f_error_check() { 
    err=$? 
    print "" 
    test "err" -eq "$1" && print "$2" && exit $1 
} 

if [[ $# -ne 1 ]] 
then 
    print "Error: (only) one parameter needed for execution of script." 
    f_using 
    exit 1 
fi 

case $1 in 
    getsets) 
        NEW_VERSION=$(ftp -o - $FTP_SITE/$FTP_RELENG/$TAG/ | \ 
                            awk '
                            { if ( match ($0, "[0-9]+Z")) { 
                                    print substr($0, RSTART, RLENGTH) 
                                } 
                            }' | sort -r | head -1) 

        if [[ $NEW_VERSION == $VERSION ]]; then 
            print "Yo, no new sets for update, ending the script." 
            exit 0 
        else 
            VERSION=$NEW_VERSION 
            print "New version is: $VERSION" 
            FTP_PATH="$FTP_RELENG/$TAG/$VERSION/i386/binary/sets" 

            print -n "Checking for presence of 'i386' directory ..." 
            
            $FTP_OPTIONS $FTP_SITE/$FTP_RELENG/$TAG/$VERSION | \ 
            awk '/i386/ {yeah_baby=1} 
                    END{ if (yeah_baby) { 
                              print " present at the class, continue." 
                          } else { 
                             print " not present. Unexcused absence." 
                              exit 33; 
                          } 
                     }' 
            f_error_check 33 "Aborting script." 
        
            print "Checking the entries ..." 

            $FTP_OPTIONS $FTP_SITE/$FTP_PATH/ | \ 
            awk -v S="$SETS" '
                   BEGIN { 
                        split(S, sets_arr) 
                    } 
                    { 
                        for (i in sets_arr) { 
                            if ($0 ~ sets_arr[i]) { 
                                found_arr[sets_arr[i]]++ 
                            } 
                        } 
                    } 
                    END { 
                        for (j in sets_arr) { 
                            if (sets_arr[j] in found_arr) { 
                                print "Set " sets_arr[j] " is on the server." 
                            } else { 
                                print "Set " sets_arr[j] " was not found on the server." 
                                n++ 
                            } 
                        } if(n>0) 
                            exit 44; 
                    }' 
            f_error_check 44 "Some sets were not found. Aborting script." 

            if [[ $(pwd) != $DWNLD_DIR ]]; then 
                print "Moving to $DWNLD_DIR" 
                cd $DWNLD_DIR 
                f_error_check 1 "Error when moving to $DWNLD_DIR." 
            fi 

            for SET in ${SETS[@]}; do 
                    print "Downloading set ==> $SET..." 
                ftp $FTP_SITE/$FTP_PATH/$SET 
                f_error_check 1 "Error while downloading set $SET." 
            done 
        fi 
    ;; 
    installkernel) 
        if [ -f $DWNLD_DIR/kern-GENERIC.tgz ]; then 
            print "We backup old kernel as netbsd.old ..." 
                /bin/mv /netbsd /netbsd.old 
                f_error_check 1 "Error while backuping the kernel." 

            print "Extracting GENERIC kernel to / ..." 
                /usr/bin/progress -zf $DWNLD_DIR/kern-GENERIC.tgz /bin/tar -C / -xpf - 
                f_error_check 1 "Error while extracting the kernel." 

            print "" 
            print -n "New kernel extracted, reboot to boot the new kernel ( yes/no ): " 

            while read ANSWER; do 
            case $ANSWER in 
                [Yy]|[Yy][Ee][Ss]) 
                    sync; sync; shutdown -r now ;; 
                [Nn]|[Nn][Oo]) 
                    print "New kernel will boot on next reboot." 
                    exit ;; 
                *) 
                    print "Answer with [y]es or [n]o" 
                    continue ;; 
            esac 
        done 
        else 
            print "kern-GENERIC doesn’t exist" && exit 1 
        fi 
    ;; 
    installsets) 
        print "Checking the validity of archives:"; print "" 
        for SET in ${SETS[@]}; do 
            [[ $SET == "kern-GENERIC.tgz" ]] && continue 
            print -n "Testing archive ==> $SET ..." 
                /bin/tar -tzvf $DWNLD_DIR/$SET >/dev/null 2>&1 && print " this one is fine." 
                f_error_check 1 " damn, something smells bad in set $SET. Aborting the script" 
        done 
        print 
        print "All archives (hopefully) valid. :-) We continue ..." 

        for SET in ${SETS[@]}; do 
            [[ ($SET == "etc.tgz") || ($SET == "xetc.tgz") || ($SET == "kern-GENERIC.tgz") ]] && continue 
            print "Extracting set ==> $SET..." 
                /usr/bin/progress -zf $DWNLD_DIR/$SET /rescue/tar -C / -xpf - 
                f_error_check 1 "Error while extracting set $SET." 
        done 
    ;; 
    etcupdate) 
        /usr/sbin/etcupdate -s $DWNLD_DIR/etc.tgz -s $DWNLD_DIR/xetc.tgz 
    ;; 
    help) 
        f_using 
        ;; 
    *) 
        print "" 
        print "Error: unknown parameter $1" 
        print "" 
        f_using 
        exit 1 
    ;; 
esac

[s0xxx]

Skript "in action":

Downloading of sets:

nbsd$ ./getset.ksh getsets                                                                                
New version is: 200909140000Z 
Moving to /home/soxxx/netbsd_sets 
Downloading set ==> base..tgz.. 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/base.tgz 
100% |*****************************************************************************| 26241 KiB  189.47 KiB/s    00:00 ETA 
26870857 bytes retrieved in 02:18 (189.47 KiB/s) 
Downloading set ==> comp.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/comp.tgz 
100% |*****************************************************************************| 37822 KiB  189.44 KiB/s    00:00 ETA 
38730622 bytes retrieved in 03:19 (189.44 KiB/s) 
Downloading set ==> etc.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/etc.tgz 
100% |*****************************************************************************|   367 KiB  157.78 KiB/s    00:00 ETA 
376257 bytes retrieved in 00:02 (157.75 KiB/s) 
Downloading set ==> kern-GENERIC.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/kern-GENERIC.tgz 
 32% |************************

Kernel install:

nbsd$ sudo ./getset.ksh installkernel 
We backup old kernel as netbsd.old... 
Extracting GENERIC kernel to / ... 
100% |*****************************************************************************| 11470 KiB   41.90 MiB/s    00:00 ETA 

New kernel extracted, reboot to boot the new kernel ( yes/no ): n 
New kernel will boot on next reboot.
nbsd$

Installing sets:

nbsd$ sudo ./getset.ksh installsets 
Extracting set ==> base.tgz... 
100% |**************************************************************************************| 77550 KiB    9.82 MiB/s    00:00 ETA 
Extracting set ==> comp.tgz... 
100% |**************************************************************************************|   130 MiB    9.22 MiB/s    00:00 ETA 
Extracting set ==> kern-GENERIC.tgz... 
100% |**************************************************************************************| 11470 KiB   36.95 MiB/s    00:00 ETA 
Extracting set ==> man.tgz... 
100% |**************************************************************************************| 50790 KiB   13.91 MiB/s    00:00 ETA 
Extracting set ==> misc.tgz... 
100% |**************************************************************************************| 12310 KiB   23.33 MiB/s    00:00 ETA 
Extracting set ==> tests.tgz... 
100% |**************************************************************************************|  8410 KiB    5.73 MiB/s    00:00 ETA 
Extracting set ==> text.tgz... 
100% |**************************************************************************************|  9830 KiB   18.97 MiB/s    00:00 ETA 
Extracting set ==> xbase.tgz... 
 82% |**********************************************************************

[IdOp]

Originally Posted by s0xxx

As I'm just too lazy to convert it to Bourne sh script, is still in ksh. Anyway, I expect opinions and recomendations to improve the script.

I've mucked around with it a bit and converted to bash (not sh). Haven't been able to test anything significant due to poor connectivity. Use at your own peril.

[s0xxx]

The only thing that needs to be changed are probably the arrays and small other things, in order to convert it to sh. I'm not so eager to change it as NetBSD, by default, comes with ksh. In bash case, you'd have to install it first.

While reading the script I spoted one thing though, script was checking for the SHA512 and MD5 later in the script, even though the two were not mentioned in SETS array. In case they were not found on server script would fail. So I just removed it; if one needs it it could be added at the beginning of the script in SETS array.

[s0xxx]

Few more things, there are two more servers that could be used for downloading of releng sets:

ftp://ftp.jaist.ac.jp/pub/NetBSD-daily/ (via ftp/http/rsync) (info)
ftp://ftp.fr.netbsd.org/pub/NetBSD-daily/ (via ftp/http) (info)

I believe Europe folks would prefer the closer site.
At the time when I was writing script I did not know of ftp site that provided releng builds. That's why I used awk to parse the output that comes when using http in stead of ftp. Using ftp is much simpler.

At the end, a warning/useful info why/who should (not) use the daily builds:

http://mail-index.netbsd.org/tech-mi...msg000153.html

[IdOp]

Originally Posted by s0xxx

The only thing that needs to be changed are probably the arrays and small other things, in order to convert it to sh. I'm not so eager to change it as NetBSD, by default, comes with ksh. In bash case, you'd have to install it first.

Agreed. It was partly just that bash is the most convenient for me and I figured if someone wants sh then this is at least half-way there. The other part was that re-writing the script forced me to look at it closely enough to learn about the interesting underlying facts and principles, like

(a) there are snapshots out there!

(b) how to upgrade by hand without booting an install iso.

Thanks for this and the additional info.