OpenBSD not reading or not using all of large hard-drive?

[sacerdos_daemonis]

I am curious about the way the OpenBSD installer handles large discs. I have a 1TB hard-drive. I installed the system a few months ago, using the entire drive with the default partitioning scheme. According to the file managers (MC, Ranger and PCmanFM) my ~/home is only 300GB. Because of information I have read that says the installer cannot read sizes larger than, if I remember correctly, 400GB, I have assumed OpenBSD cannot read the full size of the drive. Is it possible that the installer only used the drive up to the limit it can read and the rest of the drive is unused? Of course, I could test it by copying a few hundred gigabytes of files to the drive to see if it fill ups too quickly, but I do not like that idea. I ask out of curiosity. It is not an urgent matter. I took me this long to bother asking.

[J65nko]

Please provide hard data like the output of fdisk(8), disklabel(8) and # df -h

[sacerdos_daemonis]

fdisk

Disk: sd0    geometry: 121601/255/63 [1953525168 Sectors]
Offset: 0    Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
*3: A6      0   1   2 - 121600 254  63 [          64:  1953520001 ] OpenBSD

disklabel

# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: ST1000LM024 HN-M
duid: 341910ffd49c2c85
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 121601
total sectors: 1953525168
boundstart: 64
boundend: 1953520065
drivedata: 0 

16 partitions:
#                size           offset  fstype [fsize bsize  cpg]
  a:          2097152               64  4.2BSD   2048 16384    1 # /
  b:          7981104          2097216    swap                   # none
  c:       1953525168                0  unused                   
  d:          8388576         10078336  4.2BSD   2048 16384    1 # /tmp
  e:         23302240         18466912  4.2BSD   2048 16384    1 # /var
  f:          4194304         41769152  4.2BSD   2048 16384    1 # /usr
  g:          2097152         45963456  4.2BSD   2048 16384    1 # /usr/X11R6
  h:         20971520         48060608  4.2BSD   2048 16384    1 # /usr/local
  i:          4194304         69032128  4.2BSD   2048 16384    1 # /usr/src
  j:          4194304         73226432  4.2BSD   2048 16384    1 # /usr/obj
  k:        629145600         77420736  4.2BSD   4096 32768    1 # /home

df -h

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd0a     1005M   67.2M    887M     7%    /
/dev/sd0k      298G    181G    102G    64%    /home
/dev/sd0d      3.9G    1.1M    3.7G     0%    /tmp
/dev/sd0f      2.0G    483M    1.4G    25%    /usr
/dev/sd0g     1005M    194M    761M    20%    /usr/X11R6
/dev/sd0h      9.8G    3.3G    6.1G    35%    /usr/local
/dev/sd0j      2.0G    2.0K    1.9G     0%    /usr/obj
/dev/sd0i      2.0G    2.0K    1.9G     0%    /usr/src
/dev/sd0e     10.9G   25.8M   10.4G     0%    /var

If my interpretation is correct, my guess is also correct. That ~/home is full-sized, but the system displays the size as smaller. However, given my knowledge level of such readouts, there is about a 9/10ths chance my interpretation of the data is wrong.

[jggimi]

Let's start with the drive itself. This is a drive with 1,953,525,168 512-byte sectors addressable by the OS. That's 1,000,204,886,016 bytes. 1TB in "marketing" terabytes (where 1,000 = kilo, 1,000,000 = mega, etc.) It's 931.5 GB in "engineering" gigabytes (where 1,024 = kilo, 1,048,576 = mega, etc.)

The entire drive could actually fit in a single standard FFS filesystem, because those can be up to 1TB in size.

Your /home partition is 629,145,600 512-byte sectors, which is 322,122,547,200 bytes. This is exactly 300 "engineering" GB. The filesystem within is 298GB - the remaining 2GB are consumed primarily by superblocks, which are the metadata constructs that manage your filesystem.

But when you add your used and free, you end up with 283 GB. Now, where's the missing 5%?

The Fast File System reserves it, in order to automatically manage fragmentation for you. Yay!! You should never have to defrag the filesystem. In fact, there is no defrag(8) command.

You can also instruct the newfs(8) command not to reserve the space, or to reserve a different percentage. If needed, you can consume that "missing" 5%, but only if you are root. And when you do that, your Fast File System will become your Slow File System. A little more info can be found in FAQ 14.14, and in newfs(8) and tunefs(8) and in the SEE ALSO sections of both man pages.

---edited to add---

If you wish to "fill up" your drive by expanding the size of /home, you can do so with growfs(8). But you don't need to do this until/unless you need to use the space.

[sacerdos_daemonis]

1TB in "marketing" terabytes (where 1,000 = kilo, 1,000,000 = mega, etc.) It's 931.5 GB in "engineering" gigabytes (where 1,024 = kilo, 1,048,576 = mega, etc.)

Your /home partition is 629,145,600 512-byte sectors, which is 322,122,547,200 bytes. This is exactly 300 "engineering" GB.

That explains it. Thanks. I am used to Linux systems displaying drive sizes with marketing units. So I was both correct and incorrect. The entire drive is being used, but I was not familiar with the measurement system. Yippie! Time to celebrate with another cup of coffee.

Yay!! You should never have to defrag the filesystem.

I have not even thought of the degrag thingie since way back when I used Windows.

[blackhole]

No it seems like the entire drive is not being used - probably during partitioning you only used roughly a third of the drive - perhaps due to confusion of the units - this is why jggimi made reference to growfs.

[jggimi]

Exactly. There's about 594GB unallocated on the drive. And the /home partition is a good candidate for growfs(8), as there is no partition allocated after it.

Always back up your data before using growfs. I've never had a problem with it, but others have.

[Oko]

Originally Posted by sacerdos_daemonis

I am curious about the way the OpenBSD installer handles large discs. I have a 1TB hard-drive.

It handles very well That second drive is for marketing purposes 3TB ( 1024!=1000 ). Are you tryint go avoid reading documentation again? Hint: Read FAQ section about large drives.

# df -h                                                                 
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd0a     1005M   58.5M    896M     6%    /
/dev/sd0k      829G   15.0G    773G     2%    /home
/dev/sd0d      3.9G   10.0K    3.7G     0%    /tmp
/dev/sd0f      2.0G    883M    1.0G    46%    /usr
/dev/sd0g     1005M    193M    761M    20%    /usr/X11R6
/dev/sd0h      9.8G    830M    8.5G     9%    /usr/local
/dev/sd0j      2.0G    2.0K    1.9G     0%    /usr/obj
/dev/sd0i      2.0G    2.0K    1.9G     0%    /usr/src
/dev/sd0e     51.1G   17.3G   31.3G    36%    /var
/dev/sd1i      2.7T    2.1T    509G    81%    /data

[sacerdos_daemonis]

Originally Posted by cynwulf

No it seems like the entire drive is not being used - probably during partitioning you only used roughly a third of the drive - perhaps due to confusion of the units

It is not a confusion of units. I did not closely scrutinise the sizes, because I chose the option of using the entire disc and I assumed the entire disc would be used, not most of it. After spaces were allocated to the root file tree, I assumed the installer would allocate the rest to ~/home. It is time for me to imitate someone on another forum and start a crusade complaining about OpenBSD's faulty installer.

Originally Posted by jggimi
And the /home partition is a good candidate for growfs(8), as there is no partition allocated after it.

I do not need to think about enlarging /home, until I need the space, if I ever do. I was driven by curiosity.

Originally Posted by Oko
Are you tryint go avoid reading documentation again?

RTFM is one thing. Understanding it is another. Remembering everything is yet another. I can do all three, but usually not all together.

[jggimi]

Originally Posted by sacerdos_daemonis

...I chose the option of using the entire disc and I assumed the entire disc would be used...

Your partitions were assigned as described in the AUTOMATIC DISK ALLOCATION section of disklabel(8).

Disks > 7 Gigabytes

/            5% of disk.  80M – 1G 
swap         5% of disk.  80M – 2x max physical memory 
/tmp         8% of disk. 120M – 4G 
/var        13% of disk.  80M – 2x size of crash dump 
/usr         5% of disk. 900M – 2G 
/usr/X11R6   3% of disk. 512M – 1G 
/usr/local  10% of disk.   2G – 10G 
/usr/src     2% of disk.   1G – 2G 
/usr/obj     4% of disk. 1.3G – 2G 
/home       45% of disk.   1G – 300G

[IdOp]

Using the entire disk means that OpenBSD will not place itself into an existing partition defined in the MBR. Instead, the whole disk -- including sector 0, which may or may not hold an MBR -- will be allocated to the system. An existing MBR, if any, will be lost. The unallocated space (well over half of the 1TB) can be used to expand /home, or to define new disklabel partition(s) for whatever purpose you wish.