DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 27th November 2008
mtx's Avatar
mtx mtx is offline
Real Name: Valentin Bud
Fdisk Soldier
 
Join Date: May 2008
Location: RO/TM
Posts: 79
Default ZFS thoughts and questions

Hello community,

I have successfully configured ZFS using FreeBSD amd64 on a quad core
box with 2Gb of RAM. With the proper tuning found on wiki, mailing lists on which i found a great post in which our phoenix clarified some things, the machine is stable.

The new things that are coming with ZFS in terms of backup (snapshots & clones) are great, quick and easy to understand and manage. As far as i understood you can move the snapshots *only* between systems that are running ZFS (zfs send | receive) and archiving a snapshot. Now the first question, what are the implications of tar-ing or gzip-ing a snapshot and moving it to an outside storage which doesn't run ZFS, this in terms of the consistency of the snapshot.

The archiving of the data is great and reduces the space usage. Here comes the second question, which one of them to use lzjb or gzip. I did google for lzjb vs gzip but couldn't find a relevant answer.

I have a separate dataset for mysql (MyISAM). I have found an article on the mysql.com website about zfs and mysql - A look at MySQL and ZFS but it only talks about InnoDB+ZFS optimization. Here i have a couple of questions too:
  • What recordsize to use for that specific data set?
  • What are the implications in term of performance if i compress the mysql zfs data set?
  • Other things i should take into account when using MyISAM + ZFS?

I would also like to hear anything about what techniques do you use for backing up a zfs system and anything you can recommend to read for a better understanding of the zfs concept as a whole.

thank you and a great day,
v
__________________
Stop! think! ... the problem is somewhere between the monitor and chair...
"First they ignore you, then they laugh at you, then they fight you, then you win." Gandhi
links: spreadbsd syk
Reply With Quote
  #2   (View Single Post)  
Old 28th November 2008
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 696
Default

Quote:
Originally Posted by mtx View Post
I have successfully configured ZFS using FreeBSD amd64 on a quad core box with 2Gb of RAM. With the proper tuning found on wiki, mailing lists on which i found a great post in which our phoenix clarified some things, the machine is stable.
You're a glutton for punishment. Put some RAM in that baby. You're starving your CPUs.

Quote:
The new things that are coming with ZFS in terms of backup (snapshots & clones) are great, quick and easy to understand and manage. As far as i understood you can move the snapshots *only* between systems that are running ZFS (zfs send | receive) and archiving a snapshot. Now the first question, what are the implications of tar-ing or gzip-ing a snapshot and moving it to an outside storage which doesn't run ZFS, this in terms of the consistency of the snapshot.
Haven't played with the snapshot send/receive bits as yet (waiting for some multi-lane SATA to arrive so we can finish our second backup box).

Quote:
The archiving of the data is great and reduces the space usage. Here comes the second question, which one of them to use lzjb or gzip. I did google for lzjb vs gzip but couldn't find a relevant answer.
lzjb requires the least amount of CPU, but only gives decent compression (don't expect more than 2:1). Use this for locations that you want to compress, but that are accessed frequently.

gzip requires more CPU, but gives much better compression rations. Use this for filesystems that won't get used as often.

Quote:
I have a separate dataset for mysql (MyISAM). I have found an article on the mysql.com website about zfs and mysql - A look at MySQL and ZFS but it only talks about InnoDB+ZFS optimization. Here i have a couple of questions too:
  • What recordsize to use for that specific data set?
Whatever you configure the record size in MySQL is what you use for the record size in ZFS.

Quote:
  • What are the implications in term of performance if i compress the mysql zfs data set?
Considering how often MySQL data is accessed, you shouldn't compress it. You don't want anything to get in the way of your data I/O.

Quote:
  • Other things i should take into account when using MyISAM + ZFS?
Only this: DON'T USE MYISAM! Really, just don't.

Quote:
I would also like to hear anything about what techniques do you use for backing up a zfs system and anything you can recommend to read for a better understanding of the zfs concept as a whole.
Snapshots are your backups. Sending the snapshots to another system running ZFS is your off-server backups. And if that system is in another building, there's your off-site backups.

The OpenSolaris website is fount of ZFS info, as are the Sun blogs. Everything ZFS is located there, including the ZFS Admin manual.
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
  #3   (View Single Post)  
Old 28th November 2008
phoenix's Avatar
phoenix phoenix is offline
Risen from the ashes
 
Join Date: May 2008
Posts: 696
Default

Quote:
Originally Posted by phoenix View Post
lzjb requires the least amount of CPU, but only gives decent compression (don't expect more than 2:1). Use this for locations that you want to compress, but that are accessed frequently.
Further reading shows lzjb may be the better for MySQL using InnoDB.

Quote:
Whatever you configure the record size in MySQL is what you use for the record size in ZFS.
The comments on the linked page, and the MySQL dev centre, show this to be 16 KB.

Quote:
Considering how often MySQL data is accessed, you shouldn't compress it. You don't want anything to get in the way of your data I/O.
I tentatively sit corrected. At least for that one benchmark setup, lzjb compression saves you space without sacrificing performance.
__________________
Freddie

Help for FreeBSD: Handbook, FAQ, man pages, mailing lists.
Reply With Quote
  #4   (View Single Post)  
Old 28th November 2008
mtx's Avatar
mtx mtx is offline
Real Name: Valentin Bud
Fdisk Soldier
 
Join Date: May 2008
Location: RO/TM
Posts: 79
Default

Quote:
Originally Posted by phoenix View Post
You're a glutton for punishment. Put some RAM in that baby. You're starving your CPUs.
2x4Gb of RAM coming soon. I had to install it to show the administration the benefits of it and after we have decided to buy some RAM.

Quote:
Whatever you configure the record size in MySQL is what you use for the record size in ZFS.
I will have to look for the default values because i haven't modified them in MySQL.

Quote:
Considering how often MySQL data is accessed, you shouldn't compress it. You don't want anything to get in the way of your data I/O.
Thought so too but I had to ask to be sure.

Quote:
Only this: DON'T USE MYISAM! Really, just don't.
This is not up to me. The whole application that the company develops is based on MyISAM. Maybe in the future they will switch to InnoDB or other storage engine.

Quote:
Snapshots are your backups. Sending the snapshots to another system running ZFS is your off-server backups.
For now i don't have another system that runs ZFS but this is a plan for the future.

Quote:
The OpenSolaris website is fount of ZFS info, as are the Sun blogs. Everything ZFS is located there, including the ZFS Admin manual.
The ZFS Admin manual is a great resource. I'm heading to opensolaris website to do the reading.

Thank you very much for your thoughts phoenix.

all the best,
v
__________________
Stop! think! ... the problem is somewhere between the monitor and chair...
"First they ignore you, then they laugh at you, then they fight you, then you win." Gandhi
links: spreadbsd syk
Reply With Quote
Reply

Tags
backup, mysql, recordsize, zfs

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Thoughts on NetBSD 5.0 Meta_Ridley NetBSD General 16 25th May 2009 06:34 PM
NFS your thoughts rex FreeBSD General 4 24th September 2008 03:32 AM
FreeBSD as a desktop - Thoughts.. harisman FreeBSD General 62 6th September 2008 01:27 AM
Your Thoughts on WINE (Codeweaver products incl.) ninjatux General software and network 5 21st July 2008 11:56 PM
MTA thoughts cajunman4life General software and network 37 8th June 2008 07:37 PM


All times are GMT. The time now is 10:15 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick