DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 21st July 2010
sharris sharris is offline
Package Pilot
 
Join Date: Jun 2010
Posts: 146
Default Cannot copy large files to Flash Drive

Hello again everyone. Sorry to be back so soon with something so weak.

...
...

Than I mount my 32GB Corsair Flash Drive:
Code:
mount_msdosfs /dev/da0s1 /mnt
cp Partiton_1.image /mnt/Partiton_1.image
ERROR:
cp: /mnt/Partiton_1.image: File too large


Now I find out why!


http://www.yellowdog-board.com/viewt...t=5103&p=29711

Quote:
Solutions:

Stick to splitting and rejoining large files across diverse external storage systems available to and within the same OS. For YDL, that includes any Linux or Unix variant, which should include OS X from Apple as it's OS is essentially a specially redesigned variant of BSD Unix.
...
...
Sorry for the wasted space.

Last edited by sharris; 21st July 2010 at 04:07 PM.
Reply With Quote
  #2   (View Single Post)  
Old 21st July 2010
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

I'm not sure if you're still asking, I honestly don't know, but the problem may be related to FAT(msdos) filesystems not supporting files larger than 2GB.

The solution is to use a different filesystem, insane compression, or file splitting.. as may have been hinted at above, I did not read the forum post you externally linked to.

Hope that helps.
Reply With Quote
  #3   (View Single Post)  
Old 21st July 2010
Beastie Beastie is offline
Daemonology student
 
Join Date: Jan 2009
Location: /dev/earth0
Posts: 335
Default

Quote:
Originally Posted by BSDfan666 View Post
I'm not sure if you're still asking, I honestly don't know, but the problem may be related to FAT(msdos) filesystems not supporting files larger than 2GB.
4GB (-1 byte).
__________________
May the source be with you!
Reply With Quote
  #4   (View Single Post)  
Old 21st July 2010
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

My mistake, I was going by my outdated assumptions of FAT16, which indeed has a 2GB file size maximum.

In general though, using FAT-based file-systems for UNIX backups is problematic.. granted most operating systems can read/write from it, so if you used split archives it could work.. best use something that preserves file permissions.

One rather untraditional option I'm fond of is writing a pax or tar (..or perhaps cpio) archive directly to a raw partition/disk, eliminating a need for a file-system, granted there may be interoperability issues with Windows, but the format is easy enough to whip something up quick.

Either way, nobody has really decided on what method is the "best" for this purpose.. use whatever is easier for you.
Reply With Quote
  #5   (View Single Post)  
Old 29th July 2010
sharris sharris is offline
Package Pilot
 
Join Date: Jun 2010
Posts: 146
Default

BSDfan666 I think you're going to really like this one. UNIX size limitation is now OVER ... now I can backup any size partition or file and save it as a file as gzip compress the stuffing out of it, FAST making it very small. Don't overlook the *, - and the period in commandline below. Works like a charm.

COPY A PARTITION:
dd if=/dev/ad4s3 | gzip -c | split -b 2000m - /b/FreeBSD-8.1-ad4s3.gz.

RESTORE A PARTITION:
cat /b/FreeBSD-8.1-ad4s3.gz.* | gzip -dc | dd of=/dev/ad4s3

...
...
output:
/b/FreeBSD-8.1-ad4s3.gz.aa
/b/FreeBSD-8.1-ad4s3.gz.ab
/b/FreeBSD-8.1-ad4s3.gz.ac
...
...
Just like the FreeBSD base system format
Reply With Quote
  #6   (View Single Post)  
Old 29th July 2010
Beastie Beastie is offline
Daemonology student
 
Join Date: Jan 2009
Location: /dev/earth0
Posts: 335
Default

It's not a "UNIX size limitation", but a FAT32 (pretty old and outdated filesystem) limitation.
But why backup your FreeBSD system on a FAT32 filesystem anyway? If you formatted that pendrive with UFS2 you'd have a maximum file size of 8ZB!!!

Also dd copies an entire slice so you end up with a file as big as the slice itself even if it's half empty. You could use dump(8) instead (without having to compress the output).
__________________
May the source be with you!
Reply With Quote
  #7   (View Single Post)  
Old 30th July 2010
sharris sharris is offline
Package Pilot
 
Join Date: Jun 2010
Posts: 146
Default

Quote:
But why backup your FreeBSD system on a FAT32 filesystem anyway?
This is great for when you have only one large over-priced flash drive that has Windows FAT32 files on it and you want to save a compressed FreeBSD partition. I like dump too but I like to fully understand the one I'm working with first, mostly, than go back to the others I played with. Anyway, this is about the only way to make gzip takes care of all of those zeros properly, which is the main point of this operation. 40GB down to 323mb is not bad at all and it decompress perfectly.

COPY A PARTITION - FINAL:
Code:
dd if=/dev/ad4s1 ibs=4096  |  gzip > /2/FreeBSD-8.1-ad4s1.gz
42,952,379,904 -   40.0GB  =  323mb -  860s =  14.0m  -  47.0MB/s
Quote:
It's not a "UNIX size limitation", but a FAT32 (pretty old and outdated filesystem) limitation.
Now I learn the hard way but look at what it forced me to learn about. Gzip is fun. It's a trip not knowing how things work but it's a gas when you come up with an un-common solution that works. dd gives you 100% backup of every single bit. Tar will defrag by 0.1% - or 1 full percent, I forgot. Don't know about dump, other than in the real world, production, cron dump would be the way to go.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating USB flash drive drive image from FreeBSD disc1.iso FBSD Guides 1 10th February 2010 04:42 PM
FYI: v4.6 install to USB Flash Drive success s2scott OpenBSD Installation and Upgrading 0 16th November 2009 05:59 PM
Backup to USB flash drive giga FreeBSD General 5 8th December 2008 11:06 PM
USB flash drive criglerj NetBSD General 1 26th July 2008 01:25 AM
mirror device detached on large file copy lil_elvis2000 FreeBSD General 24 27th June 2008 02:56 PM


All times are GMT. The time now is 02:45 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick