DaemonForums  

Go Back   DaemonForums > Miscellaneous > General Hardware

General Hardware General hardware related questions.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 16th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default hard disk: avail 0 capacity 100% is it fine to use it like this?

in my new drive (for wich http://www.daemonforums.org/showthread.php?t=3422) I've made a special partition for a huge file I want to keep by itself and mount ro to avoid stupid mistakes and loose a 100G file with a "rm *" ).
I probably did some miscalculation, and the size is very tight. Windows copied the big file fine, and then complained there was no space left and did not put in there a 3+ M file. I compressed the file, which went down to < 1M and it could be copied.
Now I have this with df:
Code:
/dev/sd0i      100G   95.1G      0B   100%
Is it fine to leave it like this? The raw file sizes (the ones in parenthesis) of the original and the copied files under Windows are the same, so the file seems to be there.
But is it fine to run a partition full like this or does it need some free space to work properly? (I remember Windows claiming something like 15% free space to do defragmenting, if I'm right you don't need that on unix, but maybe there something I don't know) (oh well, there's a lot I don't know).
tks
Reply With Quote
  #2   (View Single Post)  
Old 16th June 2009
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

As we've told you, some space is reserved for the superuser on Unix filesystems.. like EXT2 and FFS.

The Windows ext2 driver you're using obviously assumes the role of the superuser (..root) and has full access to the capacity of the partition.

Just because df(1) on OpenBSD claims 0 bytes of available space, doesn't mean the disk is full.. and even if it was full.. nothing is harmed.

If you're worried that the file was damaged during the copy, you could painfully attempt to hash the file with MD5 or SHA and compare them.. but it's more likely you're just on mental information overload.. don't worry.. you'll become familiar with it all eventually.
Reply With Quote
  #3   (View Single Post)  
Old 16th June 2009
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Tcpdump Spy
 
Join Date: Apr 2008
Location: Netherlands
Posts: 2,243
Default

As a sidenote, keeping free space on a filesystem to prevent fragmentation is rather pointless if there is only one (Or just a few) file(s) on the filesystem.
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #4   (View Single Post)  
Old 16th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default

and by the way, the filesystem is really full I don't know how windows put that file in there, but then I removed it because it was broken and in OpenBSD cannot even save an empty vi file. Never mind, I'll keep that small file in another place, don't really want to re-format for a few M difference.

Also, I found out that Windows can mount only the first partition of my usb drive. I've seen on google, it seems to be normal behaviour for windows
I had installed fedora on a spare partition in the box some time ago, I tried the usb drive there, and it mounts everything just fine

Now, this is a bit off-topic. I'm using filezilla to transfer my files from windows to the new drive (I would have done it attaching the drive to windows, but as I said, it only mounts the first partition). Filezilla is very convenient, but is there a command-line tool to transfer a whole directory? I'v tried ncftp but would not work, maybe the directory is too big? I've seen this script here:
http://www.unix.com/shell-programmin...here-last.html
Reply With Quote
  #5   (View Single Post)  
Old 16th June 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Quote:
Originally Posted by gosha View Post
...OpenBSD cannot even save an empty vi file....
Were you root at the time of your attempt? Note:
Quote:
Originally Posted by BSDfan666 View Post
As we've told you, some space is reserved for the superuser on Unix filesystems...
You asked:
Quote:
...is there a command-line tool to transfer a whole directory?
On a *nix system, cp(1) with -R can be used for local or NFS mounted filesystems, and scp(1) with -r can be used with a remote system over an SSH connection. (NFS and SSH servers are available for Windows under Cygwin.)

Last edited by jggimi; 16th June 2009 at 01:32 PM.
Reply With Quote
  #6   (View Single Post)  
Old 16th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default

Quote:
Were you root at the time of your attempt?
No, I was not, did not think of doing that

Quote:
NFS and SSH servers are available for Windows under Cygwin.
I did try Cygwin quite a long time ago, but it went not all so smooth. I could give it a try again.

Just a few minutes ago I was transferring some more files with filezilla, and at a point the system hanged, and no way to get in even through ssh. Could only switch off the power.
I was thinking to let this drive mount automatically at boot time, so I had already all the fstab entries, but then OpenBSD could not run fsck on them, so I was locked out until I commented out the corrisponding fstab entries (changing them to "noauto" would make no difference). I'll mount them manually from now on.
This problem I had could not be filesystem-related, could it? Is filezilla so unstable? Well well, this was not the best start for my big drive...
Reply With Quote
  #7   (View Single Post)  
Old 16th June 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Quote:
Is filezilla so unstable?
I have no idea. On OpenBSD, net/filezilla is just a 3rd party FTP client. 3rd party ports do not get audited for quality. net/filezilla was added to the tree in October, 2007, and no specific problems to date have been reported to the ports@ mailing list.
Quote:
...and at a point the system hanged, and no way to get in even through ssh...
A hung system can be diagnosed via ddb(4). See crash(8) for guidance.
Reply With Quote
  #8   (View Single Post)  
Old 16th June 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Regarding ddb(4) -- if an X server is running (I'm guessing you're using X) -- X will own the keyboard and video. In order to use ddb, you'll need a serial console. FAQ 7.7 discusses them.
Reply With Quote
  #9   (View Single Post)  
Old 17th June 2009
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

You could post the fstab entires here if you need us to review them, as for the crash during a file transfer (ftp or sftp?).. can you reproduce it? terminate X while testing this out.

Serial consoles on macppc are difficult.. but not impossible.. I remember reading about hardware upgrade called gPort/g4Port.

They may be harder to acquire these days though.
Reply With Quote
Old 17th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default

Quote:
You could post the fstab entires here if you need us to review them, as for the crash during a file transfer (ftp or sftp?).. can you reproduce it?
To tell the truth, I don't know if I really want to reproduce it...
This is my fstab, just in case, as you see, now my big drive is noauto, to avoid automatic fsck on reboot in case of crash.
Code:
/dev/wd0a / ffs rw 1 1
/dev/wd0f /home ffs rw,nodev,nosuid,softdep 1 2
/dev/wd0e /tmp ffs rw,nodev,nosuid,softdep 1 2
/dev/wd0g /usr ffs rw,nodev,softdep 1 2
/dev/wd0d /var ffs rw,nodev,nosuid,softdep 1 2
/dev/sd0i /home/myhome/skqs ext2fs ro,nodev,nosuid,noauto 1 2
/dev/sd0j /home/myhome/main ext2fs rw,nodev,nosuid,noauto 1 2
/dev/sd0k /home/myhome/media ext2fs rw,nodev,nosuid,noauto 1 2
/dev/sd1i /home/myhome/mnt msdos rw,nodev,nosuid,noauto 0 0
/dev/cd0a /home/myhome/cdrom cd9660 ro,noauto 0 0
This morning I went to work leaving the computer to finish checking the big drive with fsck.ext2. I came back and found out the system crashed and rebooted by it self. What could have happened? How do I find out? (I tried checking in /var/crash, and there's a file that if opened with cat shows a number: 4096). Is there any chance that this problems are caused by ext2? (I'm probably saying something silly, but before buying my drive everything was just fine).
Reply With Quote
Old 17th June 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Quote:
Originally Posted by gosha View Post
..What could have happened? How do I find out? (I tried checking in /var/crash, and there's a file...
Please read the following man pages:
  • crash(8) Please make special note the first paragraph under Analyzing a dump which describes two files which might be found in /var/crash, how they are named and what they contain.
  • savecore(8) This is a program run by rc(8) during boot. The man page describes what this program does, and what files are placed in /var/crash by it.
Quote:
...there's a file that if opened with cat shows a number: 4096).
This would be /var/crash/minfree, I assume. I have to assume because you didn't name the file. Please note the date and time of the file's creation or last modification. It is described in the savecore(8) man page.
Quote:
...Is there any chance that this problems are caused by ext2?
With the information you've provided, that's as good a guess as any other. I certainly don't know, and couldn't tell you. It could be memory demands by the fsck program consumed all available RAM and swap space.

From your fstab, there is no swap space defined.
By default, if the "b" partition of the boot drive is defined as swap in the drive's disklabel, the OS will use that space as swap on boot, automatically, without an fstab entry. But only that boot drive's "b" partition. Other swap space must be defined in the fstab or added manually.
In order to get a valid dump for analysis, swap space must be larger than physical memory.

If your system runs out of swap space or physical RAM during normal operation, it will prevent new processes from starting, and may cause a hang for any process (userland or kernel) which requests additional memory. This could result in a hung system.

The swapctl(8) and swapon(8) commands provide manual control. The top(8), vmstat(8), and systat(8) utilities can give an admin additional insight.

Last edited by jggimi; 17th June 2009 at 12:03 PM.
Reply With Quote
Old 17th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default

Here is the minfree file (sorry I forgot to mention the name before):
Code:
-rw-r--r--   1 root  wheel     5B Mar 13  2008 minfree
I have plenty of swap:
Code:
$ swapctl    
Device      512-blocks     Used    Avail Capacity  Priority
swap_device    5243616        0  5243616     0%    0
and I guess enough /var:
Code:
/dev/wd0d      2.9G    134M    2.7G     5%    /var
for 1G or ram.
Quote:
crash(8) Please make special note the first paragraph under Analyzing a dump which describes two files which might be found in /var/crash, how they are named and what they contain.
before posting I did check the crash man page and looked for the two files, but nothing there.



ps: the crash happened late in the afternoon, so it is not likely related to the fsck.ext2 which started before eight in the morning:
Code:
Jun 17 16:58:59 mini /bsd: WARNING: / was not properly unmounted
Jun 17 16:58:58 mini savecore: no core dump
(from /var/log/messages)
It could have been that electricity has been cut for a while (there's a lot of refurbishing work going around in the building), but then why does the system reboot? Is it a default behaviour in OpenBSD? (It would make sense for a server for sure).

Last edited by gosha; 17th June 2009 at 01:39 PM.
Reply With Quote
Old 17th June 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

No dump was taken, or, there was a failure of a dump to disk when attempted by the kernel.

Your dmesg may have been retained in memory during reboot. If so, examine the output from dmesg(8) it to see if the dmesg contains any indicative kernel messages from the time of the reboot.
Quote:
the crash happened late in the afternoon, so it is not likely related to the fsck.ext2 which started before eight in the morning
I don't think you can discount it. For big file systems, running an fsck takes both tons of memory and tons of time.
Quote:
Originally Posted by FAQ 14.7, highlights in red are mine
fsck(8) time and memory requirements

Another consideration with large file systems is the time and memory required to fsck(8) the file system after a crash or power interruption. One should not put a 120G file system on a system with 32M of RAM and expect it to successfully fsck(8) after a crash. A rough guideline is the system should have at least 1M of available memory for every 1G of disk space to successfully fsck the disk. Swap can be used here, but at a very significant performance penalty, so severe that it is usually unacceptable, except in special cases.

The time required to fsck the drive may become a problem as the file system size expands, but you only have to fsck the disk space that is actually allocated to mounted filesystems. This is another reason NOT to allocate all your disk space Just Because It Is There. Keeping file systems mounted RO or not mounted helps keep them from needing to be fsck(8)ed after tripping over the power cord.

Don't forget that if you have multiple disks on the system, they could all end up being fsck(8)ed after a crash at the same time, so they could require more RAM than a single disk.

By the time one gets to somewhat larger than 1TB file system with default fragment and block sizes, fsck will require 1GB RAM to run, which is the application limit under OpenBSD. Larger fragments and/or blocks will reduce the number of inodes, and allow for larger file systems.
Reply With Quote
Old 17th June 2009
gosha gosha is offline
Spam Deminer
 
Join Date: Jun 2008
Location: China
Posts: 256
Default

thanks jggimi
I remember that page, that's why I'm not allocating all the space until I'll need it. This is what I have now:
Code:
/dev/sd0j      155G   79.2M    147G     0%    /home/myhome/main
/dev/sd0k      316G   28.1G    272G     9%    /home/myhome/media
/dev/sd0i      100G   95.1G   -3.0M   100%    /home/myhome/skqs
(As you see, I managed to fit thata 3M file in it as root)
sd0i was already finished before I went out, so less than 500G were left to be checked.

Anyway, now everything seems to be normal. I've found out on Google, that if you put an ext2fs as rw in fstab, OpenBSD will attempt to check it at boot time. So, I will not put those lines in fstab, and maybe I'll put a mount command in rc.local (is it where it sould go to mount them just after booting?)
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hard disk reliability ephemera General Hardware 32 20th April 2010 10:17 AM
Formatting Hard Disk Drive to UFS in OS X 10.5 Turquoise88 Other BSD and UNIX/UNIX-like 1 7th March 2009 09:57 PM
Hard disk bottle neck deathsoul FreeBSD General 1 28th September 2008 10:44 AM
Copy VCD film to hard disk mfaridi FreeBSD General 4 5th June 2008 01:57 PM
how to mount external hard disk ? bsdnewbie999 FreeBSD General 6 27th May 2008 02:37 AM


All times are GMT. The time now is 05:05 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick