DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 19th November 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default Script to update NetBSD using snapshots

Script could be helpful to those that are using snapshots to update their NetBSD systems. Script originally used 'coprocess' feature of ksh, but then the NetBSD folks changed the releng server from ftp to http. As I'm just too lazy to convert it to Bourne sh script, is still in ksh. Anyway, I expect opinions and recomendations to improve the script.

Code:
#!/bin/ksh 
#    getsets.ksh v.0.2 
#    dalibor Dot gudzic At gmail Dot com 
#    www.bsdserbia.org 
#    11/2009 
# 

DWNLD_DIR="/home/soxxx/netbsd_sets" 
FTP_SITE="http://nyftp.netbsd.org" 
FTP_OPTIONS="/usr/bin/ftp -o -" 
FTP_RELENG="pub/NetBSD-daily" 
TAG="netbsd-5"

VERSION="$(awk -F\' '/BUILDID/ { print $2 }' /etc/release)" 

set -A SETS "base.tgz comp.tgz etc.tgz kern-GENERIC.tgz man.tgz misc.tgz tests.tgz \ 
            text.tgz xbase.tgz xcomp.tgz xetc.tgz xfont.tgz xserver.tgz" 

f_using() { 
cat <<EOF 

Script execution: ${0##*/} <getsets | installkernel | installsets | etcupdate | help> 

    getsets          -    Downloading of sets from ftp server
    installkernel    -    Extract and install GENERIC kernel 
    installsets      -    Installing of sets 
    etcupdate        -    Update, merge of files in /etc dirctory 
    help             -    Show help on using script 

Apart from downloading of sets, to correctly execute the script start it as: 

    sudo ${0##*/} parameter 

EOF 
} 

f_error_check() { 
    err=$? 
    print "" 
    test "err" -eq "$1" && print "$2" && exit $1 
} 

if [[ $# -ne 1 ]] 
then 
    print "Error: (only) one parameter needed for execution of script." 
    f_using 
    exit 1 
fi 

case $1 in 
    getsets) 
        NEW_VERSION=$(ftp -o - $FTP_SITE/$FTP_RELENG/$TAG/ | \ 
                            awk '
                            { if ( match ($0, "[0-9]+Z")) { 
                                    print substr($0, RSTART, RLENGTH) 
                                } 
                            }' | sort -r | head -1) 

        if [[ $NEW_VERSION == $VERSION ]]; then 
            print "Yo, no new sets for update, ending the script." 
            exit 0 
        else 
            VERSION=$NEW_VERSION 
            print "New version is: $VERSION" 
            FTP_PATH="$FTP_RELENG/$TAG/$VERSION/i386/binary/sets" 

            print -n "Checking for presence of 'i386' directory ..." 
            
            $FTP_OPTIONS $FTP_SITE/$FTP_RELENG/$TAG/$VERSION | \ 
            awk '/i386/ {yeah_baby=1} 
                    END{ if (yeah_baby) { 
                              print " present at the class, continue." 
                          } else { 
                             print " not present. Unexcused absence." 
                              exit 33; 
                          } 
                     }' 
            f_error_check 33 "Aborting script." 
        
            print "Checking the entries ..." 

            $FTP_OPTIONS $FTP_SITE/$FTP_PATH/ | \ 
            awk -v S="$SETS" '
                   BEGIN { 
                        split(S, sets_arr) 
                    } 
                    { 
                        for (i in sets_arr) { 
                            if ($0 ~ sets_arr[i]) { 
                                found_arr[sets_arr[i]]++ 
                            } 
                        } 
                    } 
                    END { 
                        for (j in sets_arr) { 
                            if (sets_arr[j] in found_arr) { 
                                print "Set " sets_arr[j] " is on the server." 
                            } else { 
                                print "Set " sets_arr[j] " was not found on the server." 
                                n++ 
                            } 
                        } if(n>0) 
                            exit 44; 
                    }' 
            f_error_check 44 "Some sets were not found. Aborting script." 

            if [[ $(pwd) != $DWNLD_DIR ]]; then 
                print "Moving to $DWNLD_DIR" 
                cd $DWNLD_DIR 
                f_error_check 1 "Error when moving to $DWNLD_DIR." 
            fi 

            for SET in ${SETS[@]}; do 
                    print "Downloading set ==> $SET..." 
                ftp $FTP_SITE/$FTP_PATH/$SET 
                f_error_check 1 "Error while downloading set $SET." 
            done 
        fi 
    ;; 
    installkernel) 
        if [ -f $DWNLD_DIR/kern-GENERIC.tgz ]; then 
            print "We backup old kernel as netbsd.old ..." 
                /bin/mv /netbsd /netbsd.old 
                f_error_check 1 "Error while backuping the kernel." 

            print "Extracting GENERIC kernel to / ..." 
                /usr/bin/progress -zf $DWNLD_DIR/kern-GENERIC.tgz /bin/tar -C / -xpf - 
                f_error_check 1 "Error while extracting the kernel." 

            print "" 
            print -n "New kernel extracted, reboot to boot the new kernel ( yes/no ): " 

            while read ANSWER; do 
            case $ANSWER in 
                [Yy]|[Yy][Ee][Ss]) 
                    sync; sync; shutdown -r now ;; 
                [Nn]|[Nn][Oo]) 
                    print "New kernel will boot on next reboot." 
                    exit ;; 
                *) 
                    print "Answer with [y]es or [n]o" 
                    continue ;; 
            esac 
        done 
        else 
            print "kern-GENERIC doesn’t exist" && exit 1 
        fi 
    ;; 
    installsets) 
        print "Checking the validity of archives:"; print "" 
        for SET in ${SETS[@]}; do 
            [[ $SET == "kern-GENERIC.tgz" ]] && continue 
            print -n "Testing archive ==> $SET ..." 
                /bin/tar -tzvf $DWNLD_DIR/$SET >/dev/null 2>&1 && print " this one is fine." 
                f_error_check 1 " damn, something smells bad in set $SET. Aborting the script" 
        done 
        print 
        print "All archives (hopefully) valid. :-) We continue ..." 

        for SET in ${SETS[@]}; do 
            [[ ($SET == "etc.tgz") || ($SET == "xetc.tgz") || ($SET == "kern-GENERIC.tgz") ]] && continue 
            print "Extracting set ==> $SET..." 
                /usr/bin/progress -zf $DWNLD_DIR/$SET /rescue/tar -C / -xpf - 
                f_error_check 1 "Error while extracting set $SET." 
        done 
    ;; 
    etcupdate) 
        /usr/sbin/etcupdate -s $DWNLD_DIR/etc.tgz -s $DWNLD_DIR/xetc.tgz 
    ;; 
    help) 
        f_using 
        ;; 
    *) 
        print "" 
        print "Error: unknown parameter $1" 
        print "" 
        f_using 
        exit 1 
    ;; 
esac
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD

Last edited by s0xxx; 23rd November 2009 at 12:50 PM. Reason: minor bug
Reply With Quote
  #2   (View Single Post)  
Old 19th November 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default

Skript "in action":

Downloading of sets:

Code:
nbsd$ ./getset.ksh getsets                                                                                
New version is: 200909140000Z 
Moving to /home/soxxx/netbsd_sets 
Downloading set ==> base..tgz.. 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/base.tgz 
100% |*****************************************************************************| 26241 KiB  189.47 KiB/s    00:00 ETA 
26870857 bytes retrieved in 02:18 (189.47 KiB/s) 
Downloading set ==> comp.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/comp.tgz 
100% |*****************************************************************************| 37822 KiB  189.44 KiB/s    00:00 ETA 
38730622 bytes retrieved in 03:19 (189.44 KiB/s) 
Downloading set ==> etc.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/etc.tgz 
100% |*****************************************************************************|   367 KiB  157.78 KiB/s    00:00 ETA 
376257 bytes retrieved in 00:02 (157.75 KiB/s) 
Downloading set ==> kern-GENERIC.tgz... 
Requesting http://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/200909140000Z/i386/binary/sets/kern-GENERIC.tgz 
 32% |************************
Kernel install:

Code:
nbsd$ sudo ./getset.ksh installkernel 
We backup old kernel as netbsd.old... 
Extracting GENERIC kernel to / ... 
100% |*****************************************************************************| 11470 KiB   41.90 MiB/s    00:00 ETA 

New kernel extracted, reboot to boot the new kernel ( yes/no ): n 
New kernel will boot on next reboot.
nbsd$
Installing sets:

Code:
nbsd$ sudo ./getset.ksh installsets 
Extracting set ==> base.tgz... 
100% |**************************************************************************************| 77550 KiB    9.82 MiB/s    00:00 ETA 
Extracting set ==> comp.tgz... 
100% |**************************************************************************************|   130 MiB    9.22 MiB/s    00:00 ETA 
Extracting set ==> kern-GENERIC.tgz... 
100% |**************************************************************************************| 11470 KiB   36.95 MiB/s    00:00 ETA 
Extracting set ==> man.tgz... 
100% |**************************************************************************************| 50790 KiB   13.91 MiB/s    00:00 ETA 
Extracting set ==> misc.tgz... 
100% |**************************************************************************************| 12310 KiB   23.33 MiB/s    00:00 ETA 
Extracting set ==> tests.tgz... 
100% |**************************************************************************************|  8410 KiB    5.73 MiB/s    00:00 ETA 
Extracting set ==> text.tgz... 
100% |**************************************************************************************|  9830 KiB   18.97 MiB/s    00:00 ETA 
Extracting set ==> xbase.tgz... 
 82% |**********************************************************************
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD
Reply With Quote
  #3   (View Single Post)  
Old 23rd November 2009
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 552
Thanked 14 Times in 13 Posts
Default

Quote:
Originally Posted by s0xxx View Post
As I'm just too lazy to convert it to Bourne sh script, is still in ksh. Anyway, I expect opinions and recomendations to improve the script.
I've mucked around with it a bit and converted to bash (not sh). Haven't been able to test anything significant due to poor connectivity. Use at your own peril.
Attached Files
File Type: gz getset.sh.gz (2.1 KB, 21 views)
Reply With Quote
  #4   (View Single Post)  
Old 23rd November 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default

The only thing that needs to be changed are probably the arrays and small other things, in order to convert it to sh. I'm not so eager to change it as NetBSD, by default, comes with ksh. In bash case, you'd have to install it first.

While reading the script I spoted one thing though, script was checking for the SHA512 and MD5 later in the script, even though the two were not mentioned in SETS array. In case they were not found on server script would fail. So I just removed it; if one needs it it could be added at the beginning of the script in SETS array.
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD
Reply With Quote
  #5   (View Single Post)  
Old 23rd November 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default

Few more things, there are two more servers that could be used for downloading of releng sets:

ftp://ftp.jaist.ac.jp/pub/NetBSD-daily/ (via ftp/http/rsync) (info)
ftp://ftp.fr.netbsd.org/pub/NetBSD-daily/ (via ftp/http) (info)

I believe Europe folks would prefer the closer site.
At the time when I was writing script I did not know of ftp site that provided releng builds. That's why I used awk to parse the output that comes when using http in stead of ftp. Using ftp is much simpler.

At the end, a warning/useful info why/who should (not) use the daily builds:

http://mail-index.netbsd.org/tech-mi...msg000153.html
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD
Reply With Quote
  #6   (View Single Post)  
Old 23rd November 2009
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 552
Thanked 14 Times in 13 Posts
Default

Quote:
Originally Posted by s0xxx
The only thing that needs to be changed are probably the arrays and small other things, in order to convert it to sh. I'm not so eager to change it as NetBSD, by default, comes with ksh. In bash case, you'd have to install it first.
Agreed. It was partly just that bash is the most convenient for me and I figured if someone wants sh then this is at least half-way there. The other part was that re-writing the script forced me to look at it closely enough to learn about the interesting underlying facts and principles, like

(a) there are snapshots out there!

(b) how to upgrade by hand without booting an install iso.

Thanks for this and the additional info.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
A little confused. Do "snapshots" (vs dump=image) have any correlation, non-unix? jb_daefo FreeBSD General 9 21st November 2009 04:41 AM
Cannot update with CVS guitarscn OpenBSD Installation and Upgrading 3 7th September 2009 11:12 PM
SHA256 replaces MD5 in OpenBSD snapshots J65nko OpenBSD General 3 6th May 2009 04:36 PM
Need to update my FreeBSD...help jedispy FreeBSD Ports and Packages 4 9th June 2008 05:34 PM
Update from 6.1 to 6.3 did nothing? alanthing FreeBSD Installation and Upgrading 4 8th June 2008 02:28 PM


All times are GMT. The time now is 06:40 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick