DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 24th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default Automating patch generation and application for configuration files

1. Automating patch generation and application for configuration files
  • 1.1 Introduction
  • 1.2 Patching '/etc/mail/aliases'manually
  • 1.3 Automating patch creation
  • 1.4 Using 'patchcreate' to create a 'sshd_config' patch
  • 1.5 The 'patchcreate' script
  • 1.6 Comparison of 'sshd' before and after patching


1.1 Introduction

Anyone following mailing lists will have seem messages where a developer asks an user to apply a source code patch for solving a problem or enabling support for a new piece of hardware. This is only one of the multiple uses of patches, they also can be used to customize configuration files.

What does the man pages for patch(1) tell us?

Code:
NAME
     patch - apply a diff file to an original
       
DESCRIPTION
     patch will take a patch file containing any of the four forms of differ-
     ence listing produced by the diff(1) program and apply those differences
     to an original file, producing a patched version.  If patchfile is omit-
     ted, or is a hyphen, the patch will be read from the standard input.
Because 'diff' is the program used to create a patch, an excerpt of it's man page

Code:
NAME
     diff - differential file and directory comparator
       
DESCRIPTION
     The diff utility compares the contents of file1 and file2 and writes to
     the standard output the list of changes necessary to convert one file in-
     to the other.  No output is produced if the files are identical.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #2   (View Single Post)  
Old 24th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

1.2 Patching '/etc/mail/aliases'manually

The OpenBSD afterboot(8) man page lists a configuration task we will automate with a patch.

Code:
Edit /etc/mail/aliases and set the three standard aliases to go to either
a mailing list, or the system administrator.
      
      # Well-known aliases -- these should be filled in!
      root:           sysadm
      manager:        root
      dumper:         root
       
Run newaliases(8) after changes.
We will use directory ORIG to save a copy of the original '/etc/mail/aliases' file. The edited version will be in directory NEW.

Code:
$ mkdir WORK ; cd WORK
$ mkdir NEW ORIG
$ cp /etc/mail/aliases . 
$ cp aliases ORIG
$ cp aliases NEW
$ vi NEW/aliases
After editing we create the patch with diff

Code:
$ diff -u ORIG/aliases NEW/aliases

--- ORIG/aliases        Sat Jan 23 03:00:16 2010
+++ NEW/aliases Sat Jan 23 03:03:54 2010
@@ -64,9 +64,9 @@
 sshd:   /dev/null
 
 # Well-known aliases -- these should be filled in!
-# root:
-# manager:
-# dumper:
+root: j65nko
+manager: j65nko
+dumper: j65nko
 
 # RFC 2142: NETWORK OPERATIONS MAILBOX NAMES
 abuse:         root
@@ -74,11 +74,11 @@
 security:      root
 
 # RFC 2142: SUPPORT MAILBOX NAMES FOR SPECIFIC INTERNET SERVICES
-# hostmaster:  root
-# usenet:      root
-# news:                usenet
-# webmaster:   root
-# ftp:         root
+hostmaster:    root
+usenet:        root
+news:          usenet
+webmaster:     root
+ftp:           root
 
 # uncomment this for msgs:
 # msgs: "|/usr/bin/msgs -s"
As you can see it is very easy to figure out the changes. The deleted lines from ORIG/aliases are prefixed with "---", the added lines in NEW/aliases with "+++".

A simple redirect will save the patch to file:

Code:
$ diff -u ORIG/aliases NEW/aliases >patch
Applying the patch is simple. We will use the following options:

Code:
     -b, --backup
             Save a backup copy of the file before it is modified.  By default
             the original file is saved with a backup extension of ".orig" un-
             less the file already has a numbered backup, in which case a num-
             bered backup is made.  This is equivalent to specifying "-V
             existing".  This option is currently the default, unless --posix
             is specified.

     -p strip-count, --strip strip-count
             Sets the pathname strip count, which controls how pathnames found
             in the patch file are treated, in case you keep your files in a
             different directory than the person who sent out the patch.  The
             strip count specifies how many slashes are to be stripped from
             the front of the pathname.  (Any intervening directory names also
             go away.)  For example, supposing the file name in the patch file
             was /u/howard/src/blurfl/blurfl.c:

             Setting -p0 gives the entire pathname unmodified.
Code:
$ patch -b -p0 aliases <patch

Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- ORIG/aliases       Sat Jan 23 03:00:16 2010
|+++ NEW/aliases        Sat Jan 23 03:03:54 2010
--------------------------
Patching file aliases using Plan A...
Hunk #1 succeeded at 64.
Hunk #2 succeeded at 74.
done
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #3   (View Single Post)  
Old 24th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

1.3 Automating patch creation

In this section we will discuss the script 'patchcreate'. It will generate a patch, which subsequently is embedded in a shell script using a shell here document. Running the shell script will apply the patch.

The unmodified script patches the file in the current directory, just like we did manually earlier on. After a test confirms the proper working, only a simple edit is needed to have the patch script work on the real actual file. In our case "/etc/mail/aliases".

For the impatient we will look at the generated patch script first.

Code:
$ cat -n _patcher
     1  # ----------------------------------------------------------
     2  echo
     3  echo --- patch script for: aliases --- BEGIN
     4  
     5  # ---  edit the following line if needed
     6  FILE=./aliases
     7  
     8  patch -b -p0 ${FILE} <<END_OF_PATCH
     9  --- ORIG/aliases        Sat Jan 23 02:16:03 2010
    10  +++ NEW/aliases Sat Jan 23 02:17:39 2010
    11  @@ -64,9 +64,9 @@
    12   sshd:   /dev/null
    13   
    14   # Well-known aliases -- these should be filled in!
    15  -# root:
    16  -# manager:
    17  -# dumper:
    18  +root: j65nko
    19  +manager: j65nko
    20  +dumper: j65nko
    21   
    22   # RFC 2142: NETWORK OPERATIONS MAILBOX NAMES
    23   abuse:         root
    24  @@ -74,11 +74,11 @@
    25   security:      root
    26   
    27   # RFC 2142: SUPPORT MAILBOX NAMES FOR SPECIFIC INTERNET SERVICES
    28  -# hostmaster:  root
    29  -# usenet:      root
    30  -# news:                usenet
    31  -# webmaster:   root
    32  -# ftp:         root
    33  +hostmaster:    root
    34  +usenet:        root
    35  +news:          usenet
    36  +webmaster:     root
    37  +ftp:           root
    38   
    39   # uncomment this for msgs:
    40   # msgs: "|/usr/bin/msgs -s"
    41  END_OF_PATCH
    42  
    43  echo  --- patch script for: aliases --- END
The FILE variable in line 6 defines the target of the patch. By changing it to '/etc/mail/aliases' this file will be patched.

The patch command is on line 8, the end marker END_OF_PATCH of the here document at line 41.

An attentive reader may wonder why the typic shell shebang "#!/bin/sh" line is missing. The patch script is meant as component or a building block of a larger script. I use these kind of self-contained patch scripts in 'install.site' scripts used to customize an OpenBSD install. For details see Customizing the install process.

Of course you can run it stand-alone with a simple sh _patcher, e.g. as a post-install script for a FreeBSD or NetBSD system.


1.4 Using 'patchcreate' to create a 'sshd_config' patch

We will now see how the script automates the creation of a patch for the '/etc/sshd_config' file. First we will extract this ssh configuration file from a 'etc46.tgz' file.

Code:
$ tar tvzf /home/j65nko/Snapshots/etc46.tgz | grep ssh
 
drwx------  2 root     wheel            0 Jan 19 23:44 ./etc/skel/.ssh
-rw-------  1 root     wheel            0 Jan 19 23:44 ./etc/skel/.ssh/authorized_keys
drwxr-xr-x  2 root     wheel            0 Jan 19 23:44 ./etc/ssh
-rw-r--r--  1 root     wheel         1555 Jan 19 23:44 ./etc/ssh/ssh_config
-rw-r--r--  1 root     wheel         2524 Jan 19 23:44 ./etc/ssh/sshd_config

$ tar xvzf /home/j65nko/Snapshots.old/etc46.tgz ./etc/ssh/sshd_config 
./etc/ssh/sshd_config

$ ls -l etc/ssh/sshd_config
-rw-r--r--  1 j65nko  j65nko  2524 Jan 19 23:44 etc/ssh/sshd_config

$ cp etc/ssh/sshd_config .
$ ls -l sshd_config
-rw-r--r--  1 j65nko  wheel  2524 Jan 24 05:15 sshd_config
Having the file in our current directory, we can run 'patchcreate'

Code:
$ patchcreate sshd_config

Ok, found file sshd_config
Saving copy of sshd_config in directory ORIG
-rw-r--r--  1 j65nko  j65nko  2524 Jan 24 05:16 ORIG/sshd_config
Copy original sshd_config to directory NEW for editing
-rw-r--r--  1 j65nko  j65nko  2524 Jan 24 05:16 NEW/sshd_config

Press Enter to edit sshd_config 

[snip vi session]

Copying original ORIG/sshd_config back to current dir to test <_patcher>
-rw-r--r--  1 j65nko  j65nko   630 Jan 24 05:20 _patcher
-rw-r--r--  1 j65nko  j65nko  2524 Jan 24 05:16 sshd_config
The generated patch script:

Code:
$ cat -n _patcher
     1  # ----------------------------------------------------------
     2  echo
     3  echo --- patch script for: sshd_config --- BEGIN
     4  
     5  # ---  edit the following line if needed
     6  FILE=./sshd_config
     7  
     8  patch -b -p0 ${FILE} <<END_OF_PATCH
     9  --- ORIG/sshd_config    Sun Jan 24 05:16:30 2010
    10  +++ NEW/sshd_config     Sun Jan 24 05:20:26 2010
    11  @@ -9,9 +9,11 @@
    12   # default value.
    13   
    14   #Port 22
    15  -#AddressFamily any
    16  -#ListenAddress 0.0.0.0
    17  +AddressFamily inet 
    18  +ListenAddress 192.168.222.244 
    19   #ListenAddress ::
    20  +
    21  +AllowUsers j65nko robert
    22   
    23   # The default requires explicit activation of protocol 1
    24   #Protocol 2
    25  END_OF_PATCH
    26  
    27  echo  --- patch script for: sshd_config --- END
Executing the script:

Code:
$ sh _patcher
--- patch script for: sshd_config --- BEGIN
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- ORIG/sshd_config   Sun Jan 24 05:16:30 2010
|+++ NEW/sshd_config    Sun Jan 24 05:20:26 2010
--------------------------
Patching file ./sshd_config using Plan A...
Hunk #1 succeeded at 9.
done
--- patch script for: sshd_config --- END
Code:
$ cat -n sshd_config
[snip]
     9  # default value
    10  
    11  #Port 22
    12  AddressFamily inet 
    13  ListenAddress 192.168.222.244 
    14  #ListenAddress ::
    15  
    16  AllowUsers j65nko robert
[snip]  
    17
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #4   (View Single Post)  
Old 24th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

1.5 The 'patchcreate' script

Code:
     1  #!/bin/sh
     2  # $Id: Patchcreate.xml,v 1.6 2010/01/24 14:11:53 j65nko Exp $
     3  
     4  EDIT=/usr/bin/vi
     5  PATCHSCRIPT=_patcher
     6  
     7  mkdir -p ORIG 
     8  mkdir -p NEW
     9  
    10  # -- file name specified?
    11  if [ $# -ne 1 ] ; then
    12     echo $0 ERROR: No file name specified!
    13     exit 1
    14  fi 
    15  
    16  # -- specified file exists?
    17  if [ -f $1 -o -f ORIG/$1 ] ; then 
    18      echo Ok, found file $1
    19  else
    20      echo $0 ERROR file "$1" does not exist!
    21      exit 1
    22  fi    
    23  
    24  if [ -f ORIG/$1 ]; then
    25      echo Good, found original $1 in directory ORIG
    26  else
    27       echo Saving copy of $1 in directory ORIG 
    28       cp -p $1 ORIG/$1
    29  fi
    30  
    31  ls -l ORIG/$1
    32  
    33  if [ -f NEW/$1 ]; then
    34      echo Good, found  $1 in directory NEW
    35      ls -l NEW/$1
    36  else
    37      echo Copy original $1 to directory NEW for editing 
    38      cp -p ORIG/$1 NEW/$1
    39      ls -l NEW/$1
    40  fi
    41  
    42  # exit
    43  
    44  printf "\nPress Enter to edit $1" ; read X
    45  
    46  $EDIT NEW/$1
    47  
    48  # ------- create patch script with the patch in-line
    49  
    50  cat <<END > ${PATCHSCRIPT}
    51  # ----------------------------------------------------------
    52  echo
    53  echo --- patch script for: $1 --- BEGIN
    54  
    55  # ---  edit the following line if needed
    56  FILE=./$1
    57  
    58  patch -b -p0 \${FILE} <<END_OF_PATCH
    59  $(diff -u ORIG/${1} NEW/${1})
    60  $(echo 'END_OF_PATCH')
    61  
    62  echo  --- patch script for: $1 --- END
    63  END
    64  
    65  echo "Copying original ORIG/$1 back to current dir to test <${PATCHSCRIPT}>" 
    66  cp -p ORIG/$1 .
    67  ls -l $1 ${PATCHSCRIPT}
    68
Keep the unmodified file in 'ORIG', the file to be changed, or already modified in 'NEW'. At the end (lines 65-67), copy the original to the current directory for testing.

The actual work, creating a here document to be used by 'patch', is done in 58-60. The complete script generation, starts on line 50 and ends on line 63 and is done with another here document.

As dessert, the next section shows the effect of this 'sshd_config' patch.


1.6 Comparison of 'sshd' before and after patching

Before the patch:

Code:
$ netstat -a -f inet
Active Internet connections (including servers)
Proto   Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp          0     48  vintrax.ssh            hercules.47094         ESTABLISHED
tcp          0      0  localhost.submissi     *.*                    LISTEN
tcp          0      0  localhost.smtp         *.*                    LISTEN
tcp          0      0  *.ssh                  *.*                    LISTEN
Active Internet connections (including servers)
Proto   Recv-Q Send-Q  Local Address          Foreign Address        (state)
udp          0      0  vintrax.4599           virtueledoos.nl.ntp   
udp          0      0  vintrax.13835          ntp2.hro.nl.ntp       
udp          0      0  vintrax.43599          ntp.mediamatic.n.ntp  
udp          0      0  *.syslog               *.*                   

$ netstat -a -f inet6
Active Internet connections (including servers)
Proto   Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp6         0      0  localhost.submissi     *.*                    LISTEN
tcp6         0      0  localhost.smtp         *.*                    LISTEN
tcp6         0      0  *.ssh                  *.*                    LISTEN
After the patch and stopping and restarting 'sshd':

Code:
$ netstat -af inet
Active Internet connections (including servers)
Proto   Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp          0     48  vintrax.ssh            hercules.33562         ESTABLISHED
tcp          0      0  vintrax.ssh            *.*                    LISTEN
tcp          0      0  localhost.submissi     *.*                    LISTEN
tcp          0      0  localhost.smtp         *.*                    LISTEN

$ netstat -af inet6
Active Internet connections (including servers)
Proto   Recv-Q Send-Q  Local Address          Foreign Address        (state)
tcp6         0      0  localhost.submissi     *.*                    LISTEN
tcp6         0      0  localhost.smtp         *.*                    LISTEN
Instead of LISTENing on all IP4 addresses (*.ssh), 'sshd' will now restrict itself to the 192.168.222.244 'vintrax' IP address. It also stopped LISTENing on all IPv6 addresses (*.ssh).

$Id: Patchcreate.xml,v 1.6 2010/01/24 14:11:53 j65nko Exp $
$Id: vbul-html.xsl,v 1.15 2010/01/16 00:58:03 j65nko Exp $
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #5   (View Single Post)  
Old 24th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

Download of the patchcreate script
Attached Files
File Type: txt patchcreate.txt (1.3 KB, 136 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #6   (View Single Post)  
Old 25th January 2010
J65nko J65nko is online now
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

In the following Makefile _aliases is a patch script for the /etc/mail/aliases file created and discussed in the previous post.

The file _newaliases:
Code:
# -----------------
cat <<END
Running NEWALIASES to rebuild the aliases database ....
$(newaliases)

END
# --
The _existing-root-mail installs or moves the existing /var/mail/root to /var/mail/${USER}. If done as first thing after an install, this will usually do no harm, because the user's /var/mail file will not exist yet or will be empty.

If you don't want this , read the install(1) man page for the correct option to create a backup of an existing /var/mail/${USER} first.

Code:
# -------------------------------------------
USER=j65nko
echo --- move existing root email to $USER
install -o ${USER} -g ${USER} -m u=rw /var/mail/root /var/mail/${USER}
The Makefile itself:

Code:
# Makefile to divert root mail to user account

FILE    = _rootmail-to-j65nko

${FILE}:    _aliases _newaliases _existing-root-mail
        cat ${.ALLSRC} >${.TARGET}

clean:
        rm -f ${FILE}
You run the Makefile with the make command:
Code:
$ make
cat _aliases _newaliases _existing-root-mail >_rootmail-to-j65nko
All components are concatenated to a single file. The resulting file name can be changed easily by editing the FILE variable.
After editing one of the components run make again to create an updated version.

A commonly encountered issue with Makefiles:

Code:
"Makefile", line 6: Need an operator
Fatal errors encountered -- cannot continue
You probably got this error by copying and pasting the Makefile instead of downloading and unpacking the attached tgz file.

The Makefile requires a tab before the commands, which (re)create the target. Here the cat command in line 6. By pasting, you lost this tab and it was replaced by a couple of spaces.

You can diagnose this easily with:
Code:
 cat -ten Makefile                                 
     1  # Makefile to divert root mail to user account$
     2  $
     3  FILE     = _rootmail-to-j65nko$
     4  $
     5  ${FILE}:    _aliases _newaliases _existing-root-mail$
     6          cat ${.ALLSRC} >${.TARGET}$
The "operator" line 6, has no tab, a ^I, but a series of spaces.

The correct version
Code:
$ cat -ten Makefile

     1  # Makefile to divert root mail to user account$
     2  $
     3  FILE^I= _rootmail-to-j65nko$
     4  $
     5  ${FILE}:    _aliases _newaliases _existing-root-mail$
     6  ^Icat ${.ALLSRC} >${.TARGET}$
Here line 6 has a tab character
Attached Files
File Type: tgz Makefile.tgz (949 Bytes, 120 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 11th February 2010 at 10:14 PM. Reason: More detailed explanation of the missing tab/operator
Reply With Quote
  #7   (View Single Post)  
Old 27th October 2014
persistent_ignoramus persistent_ignoramus is offline
Real Name: Branislav
New User
 
Join Date: Sep 2009
Location: Belgrade,Kosovo,Serbia
Posts: 4
Default sticky

This thread should be made sticky. I've spent quite a while searching for it.
Reply With Quote
Reply

Tags
custom configuration, diff(1), install.site, makefile, need an operator, patch(1), sitexx.tgz

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Automating FreeBSD release downloads with a .netrc file J65nko Guides 4 11th February 2010 09:02 PM
managing configuration files JMJ_coder General software and network 1 21st January 2010 06:57 AM
Automating OpenBSD snapshot downloads with a .netrc file J65nko Guides 1 7th January 2010 03:09 AM
patch application for usb mouse? aesop FreeBSD Installation and Upgrading 1 17th January 2009 11:15 PM
Automating ports update with portmaster jaymax FreeBSD Ports and Packages 3 12th June 2008 06:56 AM


All times are GMT. The time now is 11:26 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick