30th June 2013
|
Administrator
|
|
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,132
|
|
Using rsync to synchronize or mirror your home directory files to an USB stick
Using rsync to synchronize or mirror your home directory files to an USB stick
Before you modify and start using the script please consider the following:
- OS dependencies
The script has been developed on OpenBSD and uses an USB stick formatted with OpenBSD UFS filesystem.
On my system with an IDE drive the hard disk is 'wd0' and the USB stick is 'sd0'.
When you use SATA disks 'sd0' will be your first hard disk. So please make sure you will use the correct device name for your USB stick.
For speed purpose the USB stick is mounted with what is called softdep in OpenBSD.
Code:
$ mount
/dev/wd0a on / type ffs (local)
/dev/sd0a on /mnt type ffs (local, noatime, softdep)
In FreeBSD this is called soft updates.
In OpenBSD this is a option for mount(8):
Code:
softdep
(FFS only.) Mount the file system using soft
dependencies. Instead of metadata being written
immediately, it is written in an ordered fashion to keep
the on-disk state of the file system consistent. This
results in significant speedups for file create/delete
operations. This option is ignored when using the -u
flag and a file system is already mounted read/write.
The options async and softdep are mutually exclusive.
For FreeBSD you have to use the -U option when you create the file system with newfs(8):
Code:
-U Enable soft updates on the new file system.
- The rsync '--delete' option makes this script unsuitable for backing up purposes.
If you accidentally delete the picture of your boy/girl friend in his/her new swimsuit from your home directory, it will also be deleted from the USB stick upon the next run of this script.
Or in Jggimi's immortal words, a script that uses plain 'rsync' actually is a "backup data loss" script. This is one of them
If you really need a backup script you will have to wait for an next installment, that will use rsnapshot
From rsnapshot.org
Quote:
rsnapshot is a filesystem snapshot utility for making backups of local and remote systems.
Using rsync and hard links, it is possible to keep multiple, full backups instantly available. The disk space required is just a little more than the space of one full backup, plus incrementals.
|
- Security
In the current state, the script will also sync your private SSH keys in the .ssh directory, the passwords saved by your browser etc. So please keek the USB stick in a physical secure place or bad things may happen.
My USB stick details:
- dmesg:
Code:
umass0 at uhub0 port 5 configuration 1 interface 0 "SanDisk Cruzer Edge" rev 2.00/1.26 addr 2
umass0: using SCSI over Bulk-Only
scsibus3 at umass0: 2 targets, initiator 0
sd0 at scsibus3 targ 1 lun 0: <SanDisk, Cruzer Edge, 1.26> SCSI3 0/direct removable serial.0781556b501E0E819601
sd0: 15267MB, 512 bytes/sector, 31266816 sectors
- fdisk:
Code:
Disk: sd0 geometry: 1946/255/63 [31266816 Sectors]
Offset: 0 Signature: 0xAA55
Starting Ending LBA Info:
#: id C H S - C H S [ start: size ]
-------------------------------------------------------------------------------
0: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
1: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
2: 00 0 0 0 - 0 0 0 [ 0: 0 ] unused
*3: A6 0 1 2 - 1945 254 63 [ 64: 31262426 ] OpenBSD
- disklabel:
Code:
# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: Cruzer Edge
duid: 624ba8481eba7c02
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 1946
total sectors: 31266816
boundstart: 64
boundend: 31262490
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 31262400 64 4.2BSD 2048 16384 1
c: 31266816 0 unused
Things that you will have to modify - Name of your home directory
- Device name and label of your USB stick
- The browser cache name in 'BROWSER_CACHE'
- The rsync command to exclude more files and/or directories
Script structure
- Initialize variables
- Check for root privileges to mount the USB stick and create the log files
Code:
./USB-rsync.sh : you are not ROOT !!
./USB-rsync.sh : aborting .......
- Delete '.serverauth*' files older than 1 day
Code:
deleting old .serverauth files
286167 4 -rw------- 1 adriaan adriaan 186 Jun 28 01:38 /home/adriaan/.serverauth.29764
286207 4 -rw------- 1 adriaan adriaan 186 Jun 28 05:53 /home/adriaan/.serverauth.9425
286211 4 -rw------- 1 adriaan adriaan 186 Jun 28 22:06 /home/adriaan/.serverauth.12955
286216 4 -rw------- 1 adriaan adriaan 186 Jun 29 08:27 /home/adriaan/.serverauth.30490
286208 4 -rw------- 1 adriaan adriaan 186 Jun 29 17:38 /home/adriaan/.serverauth.24636
- Mount USB stick
Code:
Mounting USB stick .......
USB has been been mounted
Filesystem Size Used Avail Capacity Mounted on
/dev/wd0a 27.6G 12.4G 13.8G 47% /
/dev/sd0a 14.7G 5.1G 9.4G 35% /mnt
- Using the less pager perform a dry run of rsync to show the files that will be affected.
Code:
sending incremental file list
adriaan/
adriaan/Guide-usb-rynsc.txt
adriaan/USB-rsync.sh
adriaan/.mozilla/firefox/chmj5t7f.default/
adriaan/.mozilla/firefox/chmj5t7f.default/.parentlock
adriaan/.mozilla/firefox/chmj5t7f.default/_CACHE_CLEAN_
[snip]
adriaan/.mozilla/firefox/chmj5t7f.default/thumbnails/
adriaan/.mozilla/firefox/chmj5t7f.default/thumbnails/14b8d30faa111187e7ff165b59b4a956.png
adriaan/.mozilla/firefox/chmj5t7f.default/thumbnails/f297b1b7fbb3972f2e4c5bb3ed0b6aec.png
adriaan/RCS/
adriaan/RCS/USB-rsync.sh,v
sent 41309 bytes received 381 bytes 27793.33 bytes/sec
total size is 5475203761 speedup is 131331.34 (DRY RUN)
- Query the user whether the actual rsync has to be done
Code:
Do you want to perform the actual, real rsync now? (Y/N):
- After the rsync has ended, show an informational message about the disk space used and the location of the log files.
Code:
sent 33540722 bytes received 3884 bytes 919030.30 bytes/sec
total size is 5475796134 speedup is 163.24
./USB-rsync.sh : has finished
=======================================================
Filesystem Size Used Avail Capacity Mounted on
/dev/wd0a 27.6G 12.4G 13.8G 47% /
/dev/sd0a 14.7G 5.1G 9.4G 35% /mnt
The log file of the dry run : /var/log/USB-rsync_dry_run.log
The log file of the actual run : /var/log/USB-rsync.log
Please do not forget to umount your USB stick .........
=======================================================
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Last edited by J65nko; 15th April 2021 at 02:14 AM.
Reason: Fixed rsnapshot,org link
|