DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 24th June 2009
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default matts: a shell script to mail attachments from the command line

matts: a shell script to mail files as attachments from the command line

  • 1.1 Introduction
  • 1.2 Investigating a MIME mail message
  • 1.3 The mailing attachment(s) script 'matts'
  • 1.4 Man page


1.1 Introduction

Mailing output of commands or contents of files in-line is not difficult to do from within the shell. Some examples:
  • A listing of all files with the '*.txt' extension:

    Code:
    $ ls -l *.txt | mail -s "My text files" xyz@gmail.com
  • Sending "dmesg" output:

    Code:
    $ dmesg | mail -s "dmesg of host zeno" t.blair@xyzz.com
  • Contents of a file in-line

    Code:
    $ mail -s "The costs of Vista" abc@gmail.com < vista_cost.txt

However, sending an attachment using the MIME standard cannot be done easily from the command line. In the following section we will take a detailed look at a MIME mail message containing an attachment and then you will understand why.


1.2 Investigating a MIME mail message

The MIME acronym stands for Multipurpose Internet Mail Extensions. These extensions add support for single or multiple non-text attachments. See http://en.wikipedia.org/wiki/MIME.

The MIME part of a mail message:

Code:
 1  MIME-Version: 1.0
 2  Content-Type: multipart/mixed;
 3      boundary="===========N=E=X=T===5facb5052688c84e499c086252c902db========" 
 4  Content-Transfer-Encoding: 7bit 
 5  
 6    This message is in MIME format. If you see this,
 7    then your mail reader is not MIME-aware 
 8  
 9  --===========N=E=X=T===5facb5052688c84e499c086252c902db========
10  Content-Type: text/plain; charset=ISO-8859-1 
11  Content-Transfer-Encoding: 7bit 
12  
13  Find attached the following file: backup.tgz
14  -rw-r--r--  1 root  wheel  2022 Feb 27 04:13 backup.tgz
15  
16  The MD5 checksum: MD5 (backup.tgz) = 1c03908b6ba8414a86f98410e4a9cbdf
17  
18  --===========N=E=X=T===5facb5052688c84e499c086252c902db========
19  Content-Type: application/octet-stream; name="backup.tgz"
20  Content-Transfer-Encoding: base64
21  Content-Disposition: attachment; filename="backup.tgz"
22  
23  H4sIAAAAAAAAA+1XbXfaVhLO1+pXTDE+mFPLhsQvKQ7dyEa2lQLyIjltsmdPjpAuoEYvVC+22Tb/
24  fZ+5kjD28XrTD+1+WM0HkLh3npm588zcwckyx12EIsrUTNxlL/4M6XQ7nePD7otOIY+/jw8Pjl50
25  [snip away many lines]
26  l2yeELKMa9XBzBXxnDLj668sTmgX6W+tG2qLp1207NzNqoRUY3BxDSnK//pffi211FJLLbXUUkst
27  tdRSSy211FJLLbXUUkst/2/yb0AcgmMAKAAA
28  
29  --===========N=E=X=T===5facb5052688c84e499c086252c902db========--
Let us have a look at the different parts.

Code:
 1  MIME-Version: 1.0
 2  Content-Type: multipart/mixed;
 3      boundary="===========N=E=X=T===5facb5052688c84e499c086252c902db========" 
 4  Content-Transfer-Encoding: 7bit 
 5  
 6    This message is in MIME format. If you see this,
 7    then your mail reader is not MIME-aware 
 8  
 9  --===========N=E=X=T===5facb5052688c84e499c086252c902db========
In line 1, we see that the MIME-version is "1.0". This version is the only one, that has been defined up to now.

Lines 2-3 define the "Content-Type" as "multipart/mixed" and specify an unique boundary, "===========N=E=X=T===5facb5052688c84e499c086252c9 02db========", as end of part marker.

The "Content-Transfer-Encoding" is set for 7 bits.

Line 5-8 contain a message in case the mail reader program does not know anything of MIME. A MIME-aware mailer will just skip this message.

In line 9, for the first time, we meet the unique boundary. Please note, and this is important, the '--' in front.

Code:
 10  Content-Type: text/plain; charset=ISO-8859-1 
 11  Content-Transfer-Encoding: 7bit 
 12  
 13  Find attached the following file: backup.tgz
 14  -rw-r--r--  1 root  wheel  2022 Feb 27 04:13 backup.tgz
 15  
 16  The MD5 checksum: MD5 (backup.tgz) = 1c03908b6ba8414a86f98410e4a9cbdf
 17  
 18  --===========N=E=X=T===5facb5052688c84e499c086252c902db========
Lines 10-11 form the lead-in of the first real useful MIME section, with a text part in lines 12-17 and again followed by the unique separator, prefixed with a '--'.

Code:
 19  Content-Type: application/octet-stream; name="backup.tgz"
 20  Content-Transfer-Encoding: base64
 21  Content-Disposition: attachment; filename="backup.tgz"
 22  
 23  H4sIAAAAAAAAA+1XbXfaVhLO1+pXTDE+mFPLhsQvKQ7dyEa2lQLyIjltsmdPjpAuoEYvVC+22Tb/
 24  fZ+5kjD28XrTD+1+WM0HkLh3npm588zcwckyx12EIsrUTNxlL/4M6XQ7nePD7otOIY+/jw8Pjl50
 25  [snip away many lines]
 26  l2yeELKMa9XBzBXxnDLj668sTmgX6W+tG2qLp1207NzNqoRUY3BxDSnK//pffi211FJLLbXUUkst
 27  tdRSSy211FJLLbXUUkst/2/yb0AcgmMAKAAA
 28  
 29  --===========N=E=X=T===5facb5052688c84e499c086252c902db========--
Line 19 specifies "application/octet-stream" as "Content-Type" and the filename as "backup.tgz". Because a gzipped (compressed) tar file, is a binary stream of data, it needs to be encoded, or mapped to the normal range of readable characters. The "base64" encoding takes care of this. The "Content-Transfer-Encoding" of line 20 specifies this encoding.

Line 21, the "Content-Disposition" defines it as an attachment with "backup.tgz" as filename. The base64 encoded data, not completely shown here, is in line 23-27, and is followed by the last boundary. To signal this boundary as the last one, it not only has an "--" at the start, but also at the end.

As mentioned before the structure of a MIME message with a normal text part and an attachment part, is not really too complicated.


1.3 The mailing attachment(s) script 'matts'

Features:
  • Command line options allow specification of the following:

    • "From:" (required)
    • Recipient (required) using at least one of the following "To:", "Cc:" or "Bcc"
    • "Subject:" (required)
    • Plain message text (optional)
  • Supports multiple files to be attached.
  • Automatic generation of a MD5 checksum file for all attached files.
  • Preview of the generated mail message
  • An ISC style license, as used by the OpenBSD project.

Examples of usage:
  • Getting help

    Code:
    $ matts -h
    ---- Usage of ./matts ------------
    -f      : set From:
    -t      : set To:
    -s      : set Subject:
    -c      : set Cc:
    -b      : set Bcc:
    -m      : message text 
    -n      : do NOT send, just show 
    -h      : help, show this message
  • Using simple mail addresses:

    Code:
    matts -f "john@a89-226-158-99.adsl.xs4all.nl" -t john884@gmail.com \
         -s "Backup of my XML articles" RCSarticles.tgz
  • Full mail addresses.

    Code:
    matts -f "John van Mierlo <john@a89-226-158-99.adsl.xs4all.nl>" \
     -b "John van Mierlo <john884@gmail.com>, J. van Mierlo <john1024@yahoo.com>" \
     -s "Xorg configuration Backup" /etc/X11/xorg.conf /var/log/Xorg.0.log
  • Using the "-n" option:

    Code:
    $ matts -f j65nko -t j65nko@xxxx.xxx -s 'The latest matts' -n matts
    Message-Id: <20090624003430-archiver@hercules.utp.xnet>
    Date: Wed Jun 24 02:34:30 CEST 2009
    From: j65nko
    Subject: The latest matts
    To: j65nko@xxxx.xxx
    X-Generator:  matts,v 1.10 2009/06/23 22:26:38 j65nko Exp 
    MIME-Version: 1.0
    Content-Type: multipart/mixed;
        boundary="=======N=E=X=T===e7c27321ce16ef2e13c9c7b435d3a845==" 
    Content-Transfer-Encoding: 7bit 
    
      This message is in MIME format. If you see this,
      then either your mail reader is not MIME-aware or
      the goondo shell scripter did something wrong ;)
    
    --=======N=E=X=T===e7c27321ce16ef2e13c9c7b435d3a845==
    Content-Type: application/octet-stream; name="MD5"
    Content-Transfer-Encoding: base64
    Content-Disposition: attachment; filename="MD5"
    
    TUQ1IChtYXR0cykgPSBkYTIyNjU1YmIzZDI5NThiNzNiY2RkNjA0MzA1MDBmNgo=
    
    --=======N=E=X=T===e7c27321ce16ef2e13c9c7b435d3a845==
    Content-Type: application/octet-stream; name="matts"
    Content-Transfer-Encoding: base64
    Content-Disposition: attachment; filename="matts"
    
    IyEvYmluL3NoCiMgJElkOiBtYXR0cyx2IDEuMTAgMjAwOS8wNi8yMyAyMjoyNjoz
    OCBqNjVua28gRXhwICQKIyBtYXR0czogIE1haWwgQVRUYWNobWVudFMKCiMgQ29w
    [snip]
    ZTY0IGVuY29kZWQgZGF0YSBjYW4gYmUgZm91bmQgaW4gdGhlCiMgIE1JTUUgJ2Zp
    bGVuYW1lPScgY29uc3RydWN0CiMgLS0tIEVORAoK
    
    --=======N=E=X=T===e7c27321ce16ef2e13c9c7b435d3a845==--


1.4 Man page

The man page can be copied to the man directory. For example on OpenBSD:

Code:
# cp man.1 /usr/local/man/man1
$Id: mail-attachments.xml,v 1.3 2009/06/24 01:36:24 j65nko Exp $
$Id: vbul-html.xsl,v 1.14 2008/09/12 03:44:16 j65nko Exp $
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 1st September 2022 at 07:17 AM. Reason: simplify the title
Reply With Quote
  #2   (View Single Post)  
Old 24th June 2009
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

A tar archive containing the matts script and man page.

To unpack and install:

Code:
$ cp  /home/xxx/matts.tgz /tmp
$ cd /tmp
$ tar xvzf matts.tgz
Become root and copy the matts shell script:
Code:
# cp matts  /usr/local/bin
chown root:wheel /usr/local/bin/matts
Make sure permissions are OK else change with "chmod'

Copy the man page:
Code:
# cp matts.1 /usr/local/man/man1
Again make sure permissions are OK else change them with "chmod'.
It is nearly 4 o'clock in the morning and I really have to go to bed now:
Attached Files
File Type: tgz matts.tgz (4.6 KB, 155 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #3   (View Single Post)  
Old 11th October 2009
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Tcpdump Spy
 
Join Date: Apr 2008
Location: Netherlands
Posts: 2,243
Default

Quote:
It is nearly 4 o'clock in the morning and I really have to go to bed now
4AM in the morning Et tu j65nko ?!
http://www.ted.com/talks/rives_on_4_a_m.html
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #4   (View Single Post)  
Old 12th October 2009
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,056
Default

@J65nko

I used my script for that:

Code:
#! /bin/sh

# subject
SUBJ="$( hostname -s ) | $( basename ${0} ) | report"

__usage() {
  echo "usage: $( basename ${0} ) email@domain.com message.txt [file1] [file2]"
  exit 1
}

__check_file() {
  if [ ! -f "${1}" ]
  then
    echo "error: file '${1}' does not exist"
    exit 1
  fi
}

__check_mail() {
  if [ ${?} -ne 0 ]
  then
    echo "error: mail(1) failed to send message"
    exit 1
  fi
}

[ ! ${#} -ge 2 ] && __usage
__check_file "${2}"

MAIL="${1}"
MESG="${2}"

case ${#} in 
  (2)
    mail -s "${SUBJ}" "${MAIL}" < "${MESG}"
    __check_mail
    ;;

  (*)
    shift
    shift
    BODY="$( basename ${0} ).body"

    cat "${MESG}" > "${BODY}"
    for FILE in "${@}"
    do
      __check_file "${FILE}"
      uuencode "${FILE}" "${FILE}" >> "${BODY}"
    done
    mail -s "${SUBJ}" "${MAIL}" < "${BODY}" 
    __check_mail
    ;;

esac
Of course it can be extended to support subject/cc/bcc ... but nice to know other alternatives as matts.
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd
Reply With Quote
  #5   (View Single Post)  
Old 12th October 2009
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

@Carpetsmoker
What? You only read my contributions 3 months after posting?

@Vermaden
Mail messages created by matts are Mime 1.0 compliant, meaning that other mail clients, e.g. gmail or Yahoo mail or alpine, will recognize the attachments and will offer a simple way to download or retrieve such an attachment.

A script which only uses uuencode requires a lot of manual intervention by the user to convert the uuencoded mail message into a file.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #6   (View Single Post)  
Old 12th October 2009
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,056
Default

Quote:
Originally Posted by J65nko View Post
@Vermaden
Mail messages created by matts are Mime 1.0 compliant, meaning that other mail clients, e.g. gmail or Yahoo mail or alpine, will recognize the attachments and will offer a simple way to download or retrieve such an attachment.

A script which only uses uuencode requires a lot of manual intervention by the user to convert the uuencoded mail message into a file.
I received messages generated by this script in Microsoft Entourage @ Mac OSX and each file was ready to download as usual, no other work required.

Havent checked if it created MIME 1.0 compilant messages, but Entourage recognized each attachement properly as separate attached file + message content.
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd
Reply With Quote
  #7   (View Single Post)  
Old 12th October 2009
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Tcpdump Spy
 
Join Date: Apr 2008
Location: Netherlands
Posts: 2,243
Default

For FreeBSD, I had to change line 200:
mktemp -p .
to
mktemp MATTS

Because FreeBSD doesn't support the -p option...

Also, it would be nice use EDITOR for filling in headers and the message, I really like the way mutt does this for example:

Code:
From: Martin Tournoij <carpetsmoker@rwxrwxrwx.net>
To: test@example.com
Cc: 
Bcc: 
Subject: some subject
Reply-To: 

A little message with text
... More text
... And even more ...
--

Vermaden, your script doesn't make MIME compliant messages. Entourage must do some magic/parsing on the message to sniff suspected attachments because all your script is sending is a bunch of good ol' MIME-less RFC822 plain text.
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #8   (View Single Post)  
Old 3rd August 2022
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

A new version that instead of the older MD5 now generates a SHA256 checksum file of the attachments.

Multiple file attachments are now dealt with correctly.

The attached tarball includes a sample script 'ezmatts' for repeated attachment mailing:
Code:
#!/bin/sh

# For repeated mails, for example sending backups to mail
# You only need to specify the file(s) to attach from the command line
# Or specify them here 

# If matts lives in '/usr/local/bin' or '~/bin', you hav to  remove './'i
#  from './matts' 
 
# $ ./matts-them -n *c

# remember that the options will be parsed by the shell!
# so you need matching single or double quotes to delimit the options

#  ---- Usage of ./matts ------------
#  -f      : set From:
#  -t      : set To:
#  -s      : set Subject:
#  -c      : set Cc:
#  -b      : set Bcc:
#  -m      : message text 
#  -n      : do NOT send, just show 
#  -h      : help, show this message
#  

# For inline 

#./matts \
#-f johndoe@lenap.utp.xnet \
#-t johndoe+BUP@gmail.com \
#-s 'The newest matts with SHA256 instead of MD5' \
#-m 'The old version as well as the latest one ....' \
#-n \
#matts matts.prev

# --- Make backup of 'zeno's /etc/wireguard directory
# maybe not too smart to mail wireguard stuff  it to gmail ....

BACKUP='zeno-bup-wireguard.tgz'
doas tar cvzf $BACKUP /etc/wireguard

./matts \
-f johndoe@lenap.utp.xnet \
-t johndoe+BUP@gmail.com \
-s 'Host zeno backup of /etc/wireguard' \
-n \
$BACKUP


#-m 'The files you requested. Regards' \
#-n \

# Specifing the files on the command line
# $ ./ezmatts  *txt
# shell variable "$@" contains whatever you specified on the command
# line

# ./matts \
# -f johndoe@lenap.utp.xnet \
# -t johndoe+BUP@gmail.com \
# -s 'Project homework assignment' \
# -n \
# "$@"
Attached Files
File Type: tgz matts-sha256.tgz (3.7 KB, 11 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 3rd August 2022 at 03:21 AM.
Reply With Quote
Reply

Tags
attachments, mail, mail attachments command line, mail command line, matts, mime 1.0

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
command-line-fu (digg like site with UNIX commands) vermaden Guides 3 13th March 2009 03:56 PM
avidemux2 command line roddierod FreeBSD General 1 10th November 2008 08:27 PM
OSS , playing a sound via command line.. scotsman FreeBSD General 7 29th August 2008 08:01 PM
Script to mail me resources usage? bigb89 Programming 5 20th July 2008 06:58 AM
How launch script or command from icewm? aleunix General software and network 8 8th June 2008 09:12 PM


All times are GMT. The time now is 11:06 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick