DaemonForums  

Go Back   DaemonForums > Miscellaneous > Off-Topic

Off-Topic Everything else.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 4th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default Data Back Up Schemes

Hi Folks:

Two Questions:
(1) I have been fumbling around for years when it comes to backing data up. I ran into the typical problems of making sure the files I save are the latest and most up to date manually.
Of course I have reaped the usual results of doing so. I have had bad experiences with some back-up systems that I either didn't quite understand of simply were not up to the task.
I am looking at using "rsync grysnc" as a solution to this problem (For Desktop Use).
I will not use a cloud scheme.
What do you use? What do you recommend? Pitfalls? Pro's/Con's?

(2) I have raid hardware in my back-up server. It is older hardware so i am cautious using
hardware raid because a failure in that card might not be able to be replaced easily if at
all, (and i have no spare) therefore complicating data recovery (T-320) 3 drives approx 1T capacity. I have
looked into software raid and at this time do not want to take the chance of setting it up
with my current skill level. Therefore its the main reason for Question (1).

I am thinking of simply using the drives on the T-320 to contain data from my Desktop or
laptop using the rsync scheme --> to one or two drives on that server.

Thoughts, suggestions, Pitfalls, Tips, check on my thinking?

As Usual, Tysm

Last edited by frcc; 4th February 2023 at 12:07 PM. Reason: clarity and corrections
Reply With Quote
  #2   (View Single Post)  
Old 4th February 2023
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

I use a cloud service for backup, out-of-scope for you, so I will just state the following general backup and recovery advice back from a time when I sold business continuity services.
  • Make (or take) your backups off-site. You do not want a physical disaster (flood, fire, earthquake, ...) to cause loss of both your site and your data.
  • Test your recovery process. Regularly. We backup due to fear, but we restore in a panic. You want a well-understood and well-practiced recovery process that begins with "bare metal" hardware platforms.
Your idea of using on-site hardware for backup is fine for many types of data loss, but does not account for any of the many possible types of physical disasters that could occur.
Reply With Quote
  #3   (View Single Post)  
Old 4th February 2023
bsd-keith bsd-keith is offline
Real Name: Keith
Open Source Software user
 
Join Date: Jun 2014
Location: Surrey/Hants Border, England
Posts: 344
Default

As just an ordinary desktop user, I back up to an external disk, several copies, taken at different times/months, usually keep 3 copies of my data & rotate, (that is, I overwrite the earliest copy with the latest backup).
__________________
Linux since 1999, & also a BSD user.
Reply With Quote
  #4   (View Single Post)  
Old 5th February 2023
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

See Using rsync to synchronize or mirror your home directory files to an USB stick

In case you change your mind about the cloud service, I can highly recommend https://www.tarsnap.com/index.html
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #5   (View Single Post)  
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Quote:
Originally Posted by jggimi View Post
I use a cloud service for backup, out-of-scope for you, so I will just state the following general backup and recovery advice back from a time when I sold business continuity services.
  • Make (or take) your backups off-site. You do not want a physical disaster (flood, fire, earthquake, ...) to cause loss of both your site and your data.
  • Test your recovery process. Regularly. We backup due to fear, but we restore in a panic. You want a well-understood and well-practiced recovery process that begins with "bare metal" hardware platforms.
Your idea of using on-site hardware for backup is fine for many types of data loss, but does not account for any of the many possible types of physical disasters that could occur.

Thanks for that jggimi, I have dual data issues I deal with. On my small business end I have moved my Websites about a dozen or so to a hosting company. I manage those sites with C-panel. This has worked well as of this writing. Since these are "static" sites I simply back-up the "Index.html" files there or download and place on Optical disc locally. In the past I ran OpenBsd httpd which I prefer, however, noting my location, the servers were
often rebooting themselves and re-establishing connectivity to the net following storms. Although the power is cheap here it is somewhat erratic because of the weather.This can take its toll on older hardware. I loved the simplicity and reliability of httpd when run in conjunction with a well thought out pf.conf. That said I feel "relatively" comfortable with the current business data set-up. That set-up although now a bit dated was a nice OpenBsd
learning experience.

My concern is my personal data which is now at my age mostly medical, photo, notes, documents, videos, pictures, contact info etc which are either somewhat essential or sentimental. I have no tape drive and probably will not invest in that direction as my needs are shrinking with my age which is swelling. "" I do/have used Optical, HD, and or USB devices in the past and have just a slight order of preference with the three. That said my biggest issue is maintaining order with the process of syncing the data with previous methods. Now, i note the comment by J65nko referring to "Jggimi's immortal words, a script that uses plain 'rsync' actually is a "backup data loss" script. This is one of them". Therefore, before I use rsync it would be a good idea to understand the sync process of that software so that I don't delete files on the destination device during back-up by improper use. I will also look at
https://www.tarsnap.com/index.html as stated by J65nko. Daily I use a laptop which I transfer data to a usb stick and then to my office desktop or server, therefore the laptop at the end of the day has no user data on it, so to speak with the exception of .conf files etc. So the problem is actually limited to obtaining and understanding and using back-up software correctly so that my files are up to date somewhere. Right now I have a mixture of data files on optical and usb usually duplicated which translates into a ga-zillion keystrokes. For personal data storage I probably don't need my T-320 as it is dated and a huge energy consumer. It appears at this point utilizing rsync etc
correctly to an optical or usb device is the least complicated path.

Last edited by frcc; 5th February 2023 at 10:41 AM.
Reply With Quote
  #6   (View Single Post)  
Old 5th February 2023
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Just like J65nko, I'm a Tarsnap customer. For all of my systems, local and remote.

If you ever reconsider external services, I do recommend the book "Tarsnap Mastery: Online Backups for the Truly Paranoid," by Michael W. Lucas.
Reply With Quote
  #7   (View Single Post)  
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Quote:
Originally Posted by bsd-keith View Post
As just an ordinary desktop user, I back up to an external disk, several copies, taken at different times/months, usually keep 3 copies of my data & rotate, (that is, I overwrite the earliest copy with the latest backup).
bsd-keith, what back-up scheme do you use for this?
Reply With Quote
  #8   (View Single Post)  
Old 5th February 2023
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
Now, i note the comment by J65nko referring to "Jggimi's immortal words, a script that uses plain 'rsync' actually is a "backup data loss" script. This is one of them". Therefore, before I use rsync it would be a good idea to understand the sync process of that software so that I don't delete files on the destination device during back-up by improper use.
Read further down in that thread. That was misquote from another user.

My statement, misquoted, was that RAID is not backup.
Reply With Quote
  #9   (View Single Post)  
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Because of your recommendation and that of J65nko's I will look into the Tarsnap service to see which is easiest for me.
Reply With Quote
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Got-it I agree at least with older machines using hardware Raid. Also, I will not put the effort into establishing software Raid as I can see one ill thought out Keystroke can make a bad day, at least on a machine that has a installed OS.

Last edited by frcc; 5th February 2023 at 11:11 AM.
Reply With Quote
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Yes, did read the entire post. ty
Reply With Quote
Old 5th February 2023
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

I like Tarsnap as a solution and service so much that:
  • I'm the OpenBSD port maintainer for tarsnap-gui, Tarsnap's graphical user interface for use on workstations. I use tarsnap-gui on my laptop.
  • I have ported ACTS (another calendar-based tarsnap script) to OpenBSD, which I discovered via the Tarsnap Mastery book. My port is included in the contrib/ directory of the published source for ACTS, and also available in the OpenBSD work in progress ports collection. I use ACTS on all of my servers.
Reply With Quote
Old 5th February 2023
bsd-keith bsd-keith is offline
Real Name: Keith
Open Source Software user
 
Join Date: Jun 2014
Location: Surrey/Hants Border, England
Posts: 344
Default

Quote:
Originally Posted by frcc View Post
bsd-keith, what back-up scheme do you use for this?
Just simply copying my data from machine to external drive, as per the 3 copies rotation.

I picked up the idea from the way servers were backed up in the old days.
__________________
Linux since 1999, & also a BSD user.
Reply With Quote
Old 5th February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Thanks, for the comeback, was wondering if anything other than what i thought you were doing, which i think is adequate for most desktop users most of the time.
Reply With Quote
Old 16th February 2023
dayid dayid is offline
Real Name: Dayid
System Archaeologist
 
Join Date: May 2013
Location: GA
Posts: 6
Default

I use a mixture of: rsync (with personal script); git; dump; & tarsnap.

I mention each because they each have different use-cases for me:
  • git: Yes, it's not "for backups"; but for things like /etc/ on machines I keep each in their own branch (hostname) and push them to my central server. This provides me with an easy way of comparing configurations on multiple machines (simple diff) while also giving me a super-fast way to back-out changes (checkout previous commit). It is also useful when duplicating a configuration as I can just checkout/clone files and fork. This is a backup for recovering simple files from local as well as a remote and protects from human error most readily as well as inconsistencies during upgrades and the like.
  • rsync: My main method for quickly-changing filesystems (I'm looking at you /home). Particularly with a script and writing to different subdirs for "snapshots" in conjunction with --link-dest this is very powerful. Lightweight, quick, and easy to configure for both local-backup and remote this is useful for human errors and system/update errors.
  • dump: Classic for a reason and since it "just works" I use dump as a nightly method to write across the network to another machine w/i the same LAN. Dump/Restore are powerful and with a small amount of learning are very useful for full-system recoveries (disk-failure, catastrophies, the like) but can also be good for human errors like "Where's that .png I .... oh.... deleted..... a long time ago.... longer than I keep in my rsync-mirror".
  • tarsnap: My main offsite/third-party resource. I'd heard of it before MWL's book, but the simplicity of it (and familiarity of it after having used rsync/dump and tar for so long) made it a shoe-in. The deduplication and encrypted storage are also fantastic benefits. I use this in addition to all my local methods, but unless there's a large incident I don't restore from it as usually I can restore from a method on my own LANs and avoid fees (and network speeds) in restoring.
Reply With Quote
Old 22nd February 2023
frcc frcc is offline
Don't Worry Be Happy!
 
Join Date: Jul 2011
Location: hot,dry,dusty,rainy,windy,straight winds, tornado,puts the fear of God in you-Texas
Posts: 335
Default

Comprehensive approach with a great reason/answer/use of the methods you employ. It provokes thought on the subject even through my needs are v. modest. Thanks for the reply..
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Back to VGA attilio OpenBSD General 6 23rd April 2020 11:47 PM
Partitioning schemes geek OpenBSD General 7 30th March 2016 09:58 PM
Back again hitest Off-Topic 8 27th January 2014 11:37 PM
CAPTCHA schemes still easy to bypass J65nko News 8 7th November 2011 08:09 PM
We're back! rpindy Feedback and Suggestions 4 5th July 2010 09:09 AM


All times are GMT. The time now is 11:43 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick