DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 27th October 2009
jb_daefo jb_daefo is offline
Package Pilot
 
Join Date: May 2008
Posts: 196
Thanked 5 Times in 5 Posts
Default A little confused. Do "snapshots" (vs dump=image) have any correlation, non-unix?

I made a snapshot, it was "2.9G" but only actually
a few MB; then the dump was 503M. I've long
known about dumps, but the snapshot is
actually== ? in windows or "boot utility" terminology,
if an equivalent exists ??
..............
(Not an urgent question) afaik.
__________________
using /LOOKAT/ with /var/db/pkg files and portmaster/aliases/pipes/find/grep to meteorically speedup port upgrades/installs...
Reply With Quote
  #2   (View Single Post)  
Old 28th October 2009
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,116
Thanked 182 Times in 149 Posts
Default

http://en.wikipedia.org/wiki/Snapsho...puter_storage)) explains it rather well.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #3   (View Single Post)  
Old 29th October 2009
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 699
Thanked 90 Times in 81 Posts
Default

In Windows terminology, a snapshot is a "Volume Shadow Copy".

Snapshots only take up space if files change, and they only track the changed files.
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
  #4   (View Single Post)  
Old 3rd November 2009
windependence's Avatar
windependence windependence is offline
Real Name: Tim
Shell Scout
 
Join Date: May 2008
Location: Phoenix, Arizona
Posts: 116
Thanked 4 Times in 3 Posts
Default

So, you can't back up a machine with JUST snapshots, you would need the system backup PLUS the snapshots, correct?

Sorry, Phoenix, you did such a great job of explaining this to me a while back, but I'm still a but confused on whether a snapshot is usable by itself, i.e., can I use just a snapshot for backup?

Thanks,

-Tim
__________________
www.windependence.org
Get your Windependence today!
Reply With Quote
  #5   (View Single Post)  
Old 3rd November 2009
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 699
Thanked 90 Times in 81 Posts
Default

Yes, and no.

You can use ZFS snapshots as a poor-man's backup solution, in that you will have old versions of files accessible. However, using the same set of disks for "live" data and "backup" data is just asking for trouble in the long-run. This is what I do at home (3-disks in a raidz1 configuration, using daily ZFS snapshots to keep 30-60 days of "backups" around).

However, you can also use snapshots as a staging ground for creating off-system backups (copy the contents of the snapshot off to another server), and you can also create a backup server that uses snapshots to provide access to historical data (this is what we do at work; rsync data off remote server to ZFS, create snapshot, repeat daily).

It all depends on what you need, and what hardware you have available.

UFS snapshots are a little more tricky, but are still usable in roughly the same ways.
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
  #6   (View Single Post)  
Old 5th November 2009
windependence's Avatar
windependence windependence is offline
Real Name: Tim
Shell Scout
 
Join Date: May 2008
Location: Phoenix, Arizona
Posts: 116
Thanked 4 Times in 3 Posts
Default

Thanks once again Freddie. So just to be clear, if the only thing I have is a snapshot, I'm probably out of luck if I have to rebuild the entire system, but if I just need some files from a prior date, I'm OK.

I haven't messed with zfs, but it may be worth it for use in my customer accounts. I will be building a NAS box for a client next week, and I am considering zfs as the file system. Is there anything I should or should not do in this case? We are going to have two RAID5 arrays, with 4 1.5TB drives in each. The second array will be used for backup of the first array. The production array will serve up NFS shares for our VM disk space.

Thanks so much again for your prompt responses.

-Tim
__________________
www.windependence.org
Get your Windependence today!
Reply With Quote
  #7   (View Single Post)  
Old 5th November 2009
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 699
Thanked 90 Times in 81 Posts
Default

Quote:
Originally Posted by windependence View Post
Thanks once again Freddie. So just to be clear, if the only thing I have is a snapshot, I'm probably out of luck if I have to rebuild the entire system, but if I just need some files from a prior date, I'm OK.
Again, yes and no. Depends on how the snapshot is made, and what is stored in the snapshot.

For example, our backup server does a full system rsync of a remote server. Then takes a snapshot. The next night, it does another full system rysnc, and takes a snapshot. Any of those snapshots can be used to restore a system (boot off LiveCD, partition/format harddrives, rysnc system from backup server directory, install boot loader, reboot).

Thus, if your snapshots are of full, complete systems then they can be used to restore full, complete systems.

Quote:
I haven't messed with zfs, but it may be worth it for use in my customer accounts. I will be building a NAS box for a client next week, and I am considering zfs as the file system. Is there anything I should or should not do in this case? We are going to have two RAID5 arrays, with 4 1.5TB drives in each. The second array will be used for backup of the first array. The production array will serve up NFS shares for our VM disk space.
To get all the benefits of ZFS, you will need to disable the hardware RAID and let ZFS manage the disks directly. If the RAID controller has an option for "Single Disk" arrays, use that, as you then get all the functionality (cache, command re-ordering, offloading, etc) of the RAID controller, but with individual disks showing in the OS. If not, you'll have to use JBOD, which turned the RAID controller into a dumb SATA controller.

For your purposes, you have a couple of options:
  • create two separate storage pools, each with a single raidz1 vdev made up of the four disks
  • create a single storage, comprised of two raidz1 vdevs, each made up of four disks, and then create two separate ZFS filesystems (one as a backup of the other)

The former provides more separation between the two, but the second provides better I/O throughput and more storage space. You also have the option to add the raidz1 vdevs to the same pool to create either a striped (RAID0) pool, or a mirrored (RAID1) pool ("zpool add" vs "zpool attach"), depending on just how much redundancy you want.

Personally, I'd look into using two separate servers. One to host the live data, one to act as a backup. Then you can either use the "zfs send" command to keep them in sync by sending snapshots between them, or use rsync to keep them in sync.

On FreeBSD, native support for NFS sharing is available via the "sharenfs" property for individual filesystems (normal NFS configuration via /etc/rc.conf). You have to install Samba to share via SMB/CIFS. And you have to install an iSCSI target port if you want to export zvols via iSCSI.

On Solaris, NFS, SMB/CIFS, and iSCSI are done natively via the sharenfs, sharecifs, and shareiscsi properties.

For NAS and SAN purposes, ZFS is very nice. For storage on a single server (direct-attached) it's useful, depending on what you are doing. On a desktop, it's a bit overkill, but still has some perks.
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
  #8   (View Single Post)  
Old 18th November 2009
windependence's Avatar
windependence windependence is offline
Real Name: Tim
Shell Scout
 
Join Date: May 2008
Location: Phoenix, Arizona
Posts: 116
Thanked 4 Times in 3 Posts
Default

OK, fantastic. I have it set up the second way with one storage pool since we'll have good backups and need the throughput. I have created two different file systems so we can use one for backup and one for production. Of course the client is being cheap and won't pay for two servers even though I agree with you that it is the way to go. I only have one question though. Since the file systems reside in the storage pool, there really is no way to know what vdev the data is in, it may even be striped across both of them as I understand it? Not that it makes any difference as if I am comprehending this correctly, the double RAID devices provide a measure of safety, more so than a single array would.

BTW, than you so much for the direction on this. I want to start using this configuration in all my client setups if it proves to be stable and reliable. Your help is always right on point as is most of the help on these forums.

-Tim
__________________
www.windependence.org
Get your Windependence today!
Reply With Quote
  #9   (View Single Post)  
Old 18th November 2009
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 699
Thanked 90 Times in 81 Posts
Default

AFAIK, you cannot determine what data is stored on what vdev, just that it's stored in that filesystem, in that storage pool. The data will be striped across all the vdevs in the pool. A single vdev or even a single disk is useless outside of the pool. You can basically treat the entire storage pool as "a single disk".

In essense, the storage pool is a RAID0 of all the vdevs.

You can, if you are really paranoid about data safety, created mirrored pools instead of striped pools. You create the pool and the first vdev as per normal. But, when you create the second (and subsequent) vdevs, you use:

zpool <poolname> attach <vdev type> <disk1> <disk2> <...>

That will create a RAID1 across all the vdevs. You lose disk space, but gain a lot more redundancy in the pool. (Probably not useful for most people/situations.)
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
Old 21st November 2009
windependence's Avatar
windependence windependence is offline
Real Name: Tim
Shell Scout
 
Join Date: May 2008
Location: Phoenix, Arizona
Posts: 116
Thanked 4 Times in 3 Posts
Default

Thanks. So far I'm blown away. It's working great for VM and ISO storage.

-Tim
__________________
www.windependence.org
Get your Windependence today!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Opera Port - conflicting pkgs in "make install" IronForge OpenBSD Packages and Ports 5 29th October 2009 05:10 AM
Fixed "xinit" after _7 _8, "how" here in case anyones' "X" breaks... using "nvidia" jb_daefo Guides 0 5th October 2009 09:31 PM
"Thanks" and "Edit Tags". diw Feedback and Suggestions 2 29th March 2009 12:06 AM
Scripted sysinstall fails with "Command 'system' failed" PeterSteele FreeBSD Installation and Upgrading 0 13th November 2008 11:31 PM


All times are GMT. The time now is 11:44 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick