DaemonForums  

Go Back   DaemonForums > Miscellaneous > General software and network

General software and network General OS-independent software and network questions, X11, MTA, routing, etc.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 3 Weeks Ago
shep shep is offline
Real Name: Scott
Arp Constable
 
Join Date: May 2008
Location: Dry and Dusty
Posts: 1,441
Default [SOLVED]imap backup questions

The 21st century has caught me and I'm looking at adapting my mail backup system to
include multiple devices (cellphone) on the same account.

My Workstation uses fetchmail to pull pop3 email, without deleting them on the server,
and Mail Delivery Agent (MDA) stanza to pipe them to nmh. nmh processes them to
#mh format and delivers to +inbox I back these up yearly but they would not include an
email sent from my laptop or cellphone. Once backup'd, I delete all emails from the
server for several reasons: server has storage limits, mail sorting time becomes
noticeable and emails over 6 months old have no privacy protections in the U.S.

My laptop uses mutt w/ imap4, set mbox_type=mh and cellphone Fairmail/imap4.

I'm thinking about switching all devices to imap4 and then running a backup
script every 6 months.

@j65nko uses fetchmail as a backup solution but I would like to pull mails from
inbox and sent.
https://daemonforums.org/showpost.ph...79&postcount=1

I prefer mbox types where each email is a spearate file; eg #mh or maildir
http://mutt.postle.net/storage/index.html

mail/offlineimap and mail/isync both look to
handle several folders although I'll be using Spectrum rather than Gmail
Both utilize maildir format for storage.
http://www.offlineimap.org/doc/use_cases.html http://isync.sourceforge.net/

I've not used mutt mbox_type = maildir and alpine does not support maildir
without a patch - I'm leaning toward staying with mh format but could not find
documentation to use nmh with offlineimap or isync.

Anyone used mutt w/ mbox_type = maildir? Offlineimap/isync with a MDA, specifically nmh?
Also open to alternative suggestions.

Solved: Fetchmail allows multiple email servers to be specified with an mda entry for each server. One minor annoyance is that passwords for each server are asked in sequence at the start of fetchmail. The first time I ran it, the SentMail folder had an authorization failure which I wondered was due to a timeout. I commented out the Inbox entry and retrieved the SentMail box. I tried again w/ Inbox and SentMail enabled and it downloaded both folders.

Fetchmail does not have a "keep alive" function like Mutt. It may be prudent to split the Inbox and SentMail boxes into 2 separate fetchmailrc configuration files and run each separately when backing up. The mda entry worked perfectly to put both folders into #mh format.

I'll post my files in @j65nko howto as a F/U.

Last edited by shep; 3 Weeks Ago at 08:33 PM.
Reply With Quote
  #2   (View Single Post)  
Old 3 Weeks Ago
Sehnsucht94's Avatar
Sehnsucht94 Sehnsucht94 is offline
Real Name: Paolo Vincenzo Olivo
Package Pilot
 
Join Date: Oct 2017
Location: Rome
Posts: 160
Default

Quote:
Originally Posted by shep View Post
Anyone used mutt w/ mbox_type = maildir? Offlineimap/isync with a MDA, specifically nmh?
Also open to alternative suggestions.
I use mutt with mbox_type = maildir, relying on offlineimap to sync mailboxes from my account on the tildeverse. I use msmtp as MDA (though I've also used the portable version of DragonFly BSD's DMA). offlineimap and msmtp invoke password-store to decrypt my password.
Th only downside of OfflineIMAP is that it's still written in python2.

~/.offlineimaprc:
Code:
[general]
# List of accounts to be synced, separated by a comma.
accounts = Tilde 
metadata = ~/.offlineimap
#ui = basic   
pythonfile = ~/.bin/offlineimap.py
maxsyncaccounts = 1
             
[Account Tilde]
# Identifier for the local repository; e.g. the maildir to be synced via IMAP.
localrepository = tilde-local
# Identifier for the remote repository; i.e. the actual IMAP, usually non-local.
remoterepository = tilde-remote
# Minutes between syncs
autorefresh = 1
quick = 10     
               
[Repository tilde-local]
# OfflineIMAP supports Maildir, GmailMaildir, and IMAP for local repositories.
type = Maildir 
# Where should the mail be placed?
localfolders = ~/Mail
               
[Repository tilde-remote]
# Remote repos can be IMAP or Gmail, the latter being a preconfigured IMAP.
type = IMAP    
remotehost = tilde.pink
remoteuser = <username>
# Synchronize folders 
folderfilter = lambda foldername: foldername in ["INBOX", "Drafts", "Sent", "Archive", "Trash"]
# Decrypt and read the encrypted password
remotepasseval = get_pass("")
ssl=yes        
sslcacertfile = /etc/openssl/certs/ca-certificates.crt
ssl_version = tls1_2
keepalive = 60 
holdconnectionopen = yes
~/.bin/offlineimap.py
Code:
#! /usr/bin/env python2.7
from subprocess import check_output

def get_pass():
    return check_output("pass mail/tilde",
     shell=True).strip("\n")
~/.msmstprc
Code:
# Set default values for all following accounts.
defaults
auth           on
tls            on
tls_starttls   on
tls_trust_file /etc/openssl/certs/ca-certificates.crt
tls_cert_file  ~/.mutt/certificates/mutt.crt
tls_key_file   ~/.mutt/private/mutt.key
syslog         on
tls_certcheck  on
logfile        ~/.msmtp.log

# home server 
account      tilde
host           smtp.tilde.pink
port           587
user           <username>
from           <username>@tilde.pink
passwordeval   "pass mail/tilde" 

account  default : tilde
~/.mutt/accounts/mdir.pink :
Code:
set realname= "<MY Name>"
set mbox_type=Maildir
set folder=~/Mail
set mask=".*" 
mailboxes ! + `\
for file in ~/Mail/*; do \
   box=$(basename "$file"); \
   if [ ! "$box" = '.' -a ! "$box" = '..' -a ! "$box" = '.customflags' \
       -a ! "$box" = '.subscriptions' ]; then \
       echo -n "\"+$box\" "; \
   fi; \
done`
set spoolfile=+INBOX
set postponed= +Drafts
set record = +Sent
set maildir_trash = yes
set sendmail="/usr/pkg/bin/msmtp -a tilde -t"
set use_from=yes
set envelope_from=yes
set from= "My Name <username@tilde.pink>"
set mail_check = 90
__________________
“Mi casa tendrá dos piernas y mis sueños no tendrán fronteras„
Reply With Quote
  #3   (View Single Post)  
Old 3 Weeks Ago
shep shep is offline
Real Name: Scott
Arp Constable
 
Join Date: May 2008
Location: Dry and Dusty
Posts: 1,441
Default

Quote:
Originally Posted by Sehnsucht94 View Post
---clip
Th only downside of OfflineIMAP is that it's still written in python2.
---clip
Upstream offlineimap has been forked to Python3 based offlineimap3 and is recommended by the original project:

https://github.com/OfflineIMAP/offlineimap
https://github.com/OfflineIMAP/offlineimap3

I ended up with a fetchmail/nmh solution but updating offlineimap -> offlineimap3 would be a worthy port upgrade for OpenBSD.

In my research, I also found a performance comparison between offlineimap and isync. For large backup's, isync (coded in C) was said to perform about 35% faster.

https://irreal.org/blog/?p=8184

I'm on the fence as far as Maildir vs #mh storage formats. Out of the box, Evolution/Thunderbird use Maildir and Sylpheed/Claws-Mail use #mh.

Last edited by shep; 3 Weeks Ago at 02:29 PM.
Reply With Quote
  #4   (View Single Post)  
Old 2 Weeks Ago
Sehnsucht94's Avatar
Sehnsucht94 Sehnsucht94 is offline
Real Name: Paolo Vincenzo Olivo
Package Pilot
 
Join Date: Oct 2017
Location: Rome
Posts: 160
Default

Quote:
Originally Posted by shep View Post
Upstream offlineimap has been forked to Python3 based offlineimap3 and is recommended by the original project.
Thanks! Didn't know that, and was really hoping for somebody to port it to python3. Last time I checked, this hasn't happened yet, but it was a long time ago.
Quote:
I ended up with a fetchmail/nmh solution but updating offlineimap -> offlineimap3 would be a worthy port upgrade for OpenBSD.
Guess I could do the same for pkgsrc.
Quote:
In my research, I also found a performance comparison between offlineimap and isync. For large backup's, isync (coded in C) was said to perform about 35% faster.

https://irreal.org/blog/?p=8184

I'm on the fence as far as Maildir vs #mh storage formats. Out of the box, Evolution/Thunderbird use Maildir and Sylpheed/Claws-Mail use #mh.
Good to know; I remember using Claws-Mail with MH for years, but for some reason -which now I fail to remember- at some point I migrated to Thunderbird and then never looked back.
__________________
“Mi casa tendrá dos piernas y mis sueños no tendrán fronteras„
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Simple backup questions unixjingleman FreeBSD General 15 27th February 2011 11:21 PM
uw-imap connection timeout andrewm OpenBSD Packages and Ports 4 22nd October 2009 11:03 AM
backup freeBSD 7.0 using Backup Exec ccc FreeBSD General 2 25th April 2009 09:23 PM
Courier IMAP Different MD5 version jrs665 OpenBSD Packages and Ports 3 30th October 2008 12:34 PM
Recommendation for Imap and POP server Pher FreeBSD Installation and Upgrading 5 20th July 2008 07:43 PM


All times are GMT. The time now is 11:25 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick