DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 19th November 2009
wimwauters wimwauters is offline
Port Guard
 
Join Date: Aug 2008
Posts: 36
Thanked 1 Time in 1 Post
Default encryption and volume split of folder backups

Hi all,

is anyone doing encrypted backups of folders or files, and splitting the backup into volumes?

Tar (or the file system?) seems to have a limit at 8GB (too big to be practical anyway, I prefer chunks of 700MB or 2048MB), and using it with encryption I haven't tried to figure out yet

p7zip does not store Unix file access rights by itself, so it needs piping from tar, but then you run into a limit on the path name of 256 characters (try saving a webpage): "tar: File name too long for ustar "

I'm trying all this on a box with OpenBSD 4.5 i386, but I've got 2 boxes with 4.6 to play with too.

I use the RAR package on my old OpenBSD 3.9 fileservers, but i's time to install 4.6 on them, me thinks

Any thoughts are appreciated.

Cheers!
Reply With Quote
  #2   (View Single Post)  
Old 19th November 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

The pax tool can use cpio or tar formats and can manage multiple volumes, but it has the limitations of the underlying format. On the other hand, dump/restore has none of the other formats' restrictions, but it cannot handle multiple volumeswith media that does not have end-of-volume marks, so it is limited to tape drives. For the same reason, it cannot manage multivolume via named pipes.

I use dump and restore to avoid the limitations of tar, gtar, cpio, or pax. The output is piped through gzip and piped to a storage server via ssh. On the storage server, offsite backup is done to DVD+RW, with dump again, and the multivolume is managed through through two tools: flyisofs and shunt, from sysutils/shunt. This breaks the streams into 4.3GB chunks piped into growisofs, from sysutils/dvd+rw-tools.

Here's an example of using shunt and flyisofs for backup, then shunt for restore. The files to be backed up are already gzipped dump files, but you could add piping to openssl enc, gzip, what ever you want. .
Code:
Backup:

# shunt -c 'dump -b 64 -0af - <files>' + \
    'flyisofs fbc=2090000 mbc=2295104 | growisofs -Z /dev/rcd0c=/dev/fd/0'

Restore:

       # shunt 'mount /dev/cd0c /mnt;cat mnt/*;umount /mnt' + -c 'restore -rf -'
Avoid using sudo on shunted pipes -- this complicates administration in the event you must stop operations and kill related processes.
Reply With Quote
  #3   (View Single Post)  
Old 20th November 2009
wimwauters wimwauters is offline
Port Guard
 
Join Date: Aug 2008
Posts: 36
Thanked 1 Time in 1 Post
Default

Interesting, I learnt about shunt to extend piping capabilities, but it looks like I'll be compiling RAR from ports and continue to use that until I am confident with/about Bacula.

Is there really no UNIX command line way of backing up files into encrypted chunks, with the ability to restore individual files
Reply With Quote
  #4   (View Single Post)  
Old 20th November 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

I just gave it to you.

The restore example concatenates files from multiple mount sequences of DVDs, and concatenates them into a single pipe into restore(1). To restore individual files, use restore(1) in interactive mode. Change the restore operands from "-rf -" to "-if -" to interactively restore from standard input. To add decryption or decryption, pipe the stream through openssl(1) with the enc operand.

The restore(1) man page and the openssl(1) man page may be helpful.
Reply With Quote
  #5   (View Single Post)  
Old 22nd November 2009
DraconianTimes's Avatar
DraconianTimes DraconianTimes is offline
Security Geek
 
Join Date: May 2008
Location: United Kingdom
Posts: 37
Thanked 2 Times in 2 Posts
Default

This won't get you CD/DVD/tape backups, but you might find this interesting:

http://www.boxbackup.org/

Rgs,
Nick
Reply With Quote
  #6   (View Single Post)  
Old 24th November 2009
wimwauters wimwauters is offline
Port Guard
 
Join Date: Aug 2008
Posts: 36
Thanked 1 Time in 1 Post
Default

Quote:
Originally Posted by jggimi View Post
I just gave it to you.

The restore example concatenates files from multiple mount sequences of DVDs, and concatenates them into a single pipe into restore(1). To restore individual files, use restore(1) in interactive mode. Change the restore operands from "-rf -" to "-if -" to interactively restore from standard input. To add decryption or decryption, pipe the stream through openssl(1) with the enc operand.

The restore(1) man page and the openssl(1) man page may be helpful.
Excellent, thanks for persevering, jggimi, I had no idea that restore could be interactive and pick out files or folders, for some reason I had it in my mind that it was all or nothing. I will study the man page.

Much obliged
Reply With Quote
  #7   (View Single Post)  
Old 24th November 2009
wimwauters wimwauters is offline
Port Guard
 
Join Date: Aug 2008
Posts: 36
Thanked 1 Time in 1 Post
Default

Quote:
Originally Posted by DraconianTimes View Post
This won't get you CD/DVD/tape backups, but you might find this interesting:

http://www.boxbackup.org/

Rgs,
Nick
Thanks for the info, that's an exciting piece of software, and I will be playing with it. I have many endusers who need this type of thing

However, I like to keep my server setups as simple/vanilla/minimalistic/KISS as possible, especially since my clients (small businesses) can not afford to run much of a serverfarm. By keeping the installs minimalistic, I speed up bare metal installs/repairs.

And having a command line way of restoring files, or whole backups, on a fresh install with the same tool as on a full install, is ace
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MS DOS volume labels? bigearsbilly FreeBSD General 2 1st April 2009 10:50 PM
dd and split usage problem paul-lkw FreeBSD General 4 31st January 2009 08:13 PM
zfs backups terryd FreeBSD General 2 2nd December 2008 10:25 PM
mplayer volume cannot increase. bsdnewbie999 OpenBSD General 3 19th June 2008 03:16 PM


All times are GMT. The time now is 05:22 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick