external drive partition question + fdisk question

[gosha]

Ok, I've finally bought myself a new drive which I'll use for backup and simply as extra memory.

It is 1T and has a big fat32 partition. I would like to make smaller partitions, maybe four (if I understand right this is the maximum for fdisk).
I've used fdisk before to install OpenBSD and I know that the partition must start at sector 63.
Is it the same for a non bootable drive? Is it fine to start from head 0 sector 1? Like here:

$ fdisk sd0 
Disk: sd0	geometry: 121601/255/63 [1953525168 Sectors]
Offset: 0	Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 83      0   0   1 -  32635 106  47 [           0:   524288000 ] Linux files*
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 3: 00      0   0   0 -      0   0   0 [           0:           0 ] unused

About size, I can't remember how I got this size, I think I wrote 250 g, is this the way to do it? Or should I convert size to sectors?
Once I manage to partition in fdisk, does disklabel give the opportunity to make extra partitions (or how are they called?)?
I've tried to divide sectors by 4 and have now this mbr:

Disk: sd0	geometry: 121601/255/63 [1953525168 Sectors]
Offset: 0	Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 83      0   0   1 -  30400  83  63 [           0:   488381292 ] Linux files*
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused      
 3: 00      0   0   0 -      0   0   0 [           0:           0 ] unused

Is this it? Can I run newfs? Will I need disklabel after this or is it usable like this? 240G partition isn't too big in case of fsck?

Sorry I'm adding new things all the time.
I've just seen this in a post http://www.daemonforums.org/showthre...ighlight=fdisk :

I'm using UFS2+SU with huge storage areas and sometimes ext3 in Linux with a full journal because of reliability. Ext2 is comparable to FAT in Windows95, some power shortage and you can say good bye to your lovely data

Is this right? Does it mean it is not that safe to use ext2 for backup in my external drive?

[ocicat]

Originally Posted by gosha

It is 1T and has a big fat32 partition. I would like to make smaller partitions, maybe four (if I understand right this is the maximum for fdisk).

fdisk(8) allows a maximum of four primary (MBR) partitions. You also have the choice of taking one primary partition & designating it as an extended partition. You can further subdivide this single extended partition into any number of logical partitions. You should familiarize yourself with these concepts through any number of sources; Wikipedia being one which covers the basics:

http://en.wikipedia.org/wiki/Extended_partition

You should also study Sections 4 & 14 of the FAQ:

http://www.openbsd.org/faq/faq4.html

http://www.openbsd.org/faq/faq14.html

I've used fdisk before to install OpenBSD and I know that the partition must start at sector 63.
Is it the same for a non bootable drive?

No. If the drive is not being used as a boot drive, then adhering to MBR restrictions is not required. fdisk(8) is not required for non-boot disks.

Is it fine to start from head 0 sector 1?

Yes, for non-boot drives.

Once I manage to partition in fdisk, does disklabel give the opportunity to make extra partitions (or how are they called?)?

It is unfortunate that the name "partition" is used in so many contexts, but that is the way things are in the Intel world. Other architectures (such as PPC...) do not have master boot records (MBR's).

Nevertheless, disklabel(8) can further subdivide FFS filesystems.

Is this it? Can I run newfs? Will I need disklabel after this or is it usable like this? 240G partition isn't too big in case of fsck?

Study Section 14.7 of the FAQ.

The order of preparing additional storage for use with OpenBSD is:

1. fdisk(8) is necessary for Intel boot drives, but optional for non-boot drives.
2. disklabel(8) for FFS filesystems. disklabel will fake labels for non-FFS filesystems.
3. newfs(8) to initialize each filesystem for mounting.

Does it mean it is not that safe to use ext2 for backup in my external drive?

See Section 14.16.

I would highly suggest that you change the order where you search for information.

1. Manpages are the most authoritative & vetted.
2. The official FAQ is the single best source of general (palpable) information -- especially for newbies.
3. Information on the official mailing lists is to be considered more timely -- especially if spoken by project developers.
4. Forums such as this.

[gosha]

Thanks a lot.
I know, it seems I've not read the faq and manpages, but I did many times in the past to install OpenBSD, but could never fully understand the difference between the partition in fdisk and in disklabel and also this head/sector thing.
Now, if I get it right, I could make my four partitions in fdisk and then for every of them add extended partitions with disklabel, right?
So, Ocicat, you say I could make one big partition in fdisk and then extended partitions using disklabel. That could be fine for me. I was thinking about doing it with fdisk to start using only part of the disk, and then if needed make the new partition, maybe, if needed using another filesystem (if the primary partition is, say, ffs, disklabel cannot make a ext2 partition, right?)

[BSDfan666]

No, There can only be 1 primary MBR partition for OpenBSD, (A6 identifier).. disklabel provides an additional layer of partitioning within the OpenBSD area of the disk.

As ocicat said, not every architecture uses MBR partitioning.. MBR was an IBM/MS idea, it supports 4 primary partitions.. or 3 primary partitions and 1 extended partition.. extended partitions are basically a daisy chained MBR that allows you to have additional partitions.

In your case, you might want to create 2 MBR partitions.. one for OpenBSD, and one for Linux(EXT2).

Also, most people place their first partition at LBA 63.. not LBA 0.. because you'll just trash your MBR.

[ocicat]

Originally Posted by gosha

I know, it seems I've not read the faq and manpages, but I did many times in the past to install OpenBSD, but could never fully understand the difference between the partition in fdisk and in disklabel and also this head/sector thing.

• As far as the "head/sector thing", this is not an OpenBSD-specific topic. Again, Wikipedia may help clear up ambiguities:
  
  http://en.wikipedia.org/wiki/Cylinder-head-sector
• Maybe the following will help depict the difference between the different uses of the word "partition".
  
  One of the better (free) diagrams I have seen on the difference between fdisk(8) & disklabel(8) partitions is in the NetBSD documentation. See Section 2.2.2 of the NetBSD Guide:
  
  http://netbsd.org/docs/guide/en/chap-inst.html
  
  This document also uses the term "slice" which is the same thing as "disklabel(8) partition".
  
  However, note that there are also differences between NetBSD's & OpenBSD's nomenclature. In OpenBSD, the entire disk is accessible via disklabel slice 'c' whereas in NetBSD, the same slice is entitled 'd'. Also, OpenBSD does not have an analogous construct as what NetBSD designates as slice 'd'. You can have a disklabel slice entitled 'd' in an OpenBSD filesystem, but it will not represent the MBR partition where it resides. Otherwise, this diagram is still useful.

Now, if I get it right, I could make my four partitions in fdisk and then for every of them add extended partitions with disklabel, right?

No. Only one MBR primary partition can be made into an extended partition in MBR terminology. A disk can only have one FFS (type A6 MBR partition...) filesystem as BSDfan666 has already stated.

So, Ocicat, you say I could make one big partition in fdisk and then extended partitions using disklabel.

OpenBSD's FFS filesystem can reside on an MBR fdisk(8) partition of any size & a maximum of 16 disklabel(8) partitions (or "slices" as NetBSD's documentation uses...) can reside within it. See Section 14.16 & disklabel(8)'s manpage.

...& note that one of the 16 partitions that disklabel(8) can recognize is the entire disk, so an OpenBSD system can recognize only 15 configurable partitions at a time.

I was thinking about doing it with fdisk to start using only part of the disk, and then if needed make the new partition, maybe, if needed using another filesystem (if the primary partition is, say, ffs, disklabel cannot make a ext2 partition, right?)

disklabel(8) does not create ext2 filesystems. MBR partitions can be designated as ext2 by setting the type to 83, but you will need to verify this value. disklabel(8) will be able to recognize ext2 partitions, but disklabel(8) will not be able to subpartition an ext2 filesystem, however newfs(8) will be able to initialize it.

Two observations:

• You appear to be of the opinion that disklabel(8) can subpartition filesystems of any type. disklabel(8) can only subpartition FFS filesystems. However, disklabel(8) will be b able to recognize a limited number of other filesystems & assign them disklabel(8) letters such that they can be mounted into an OpenBSD system, but there are still only 16 letters that OpenBSD can map.
• It is not entirely clear why you want to slice & dice a hard drive to such a fine granularity. Yes, dump(8) & restore(8) can be used to backup partitions, however, they can also be used to backup individual files.
  
  If you are thinking that one disk can be universally used across multiple architectures & operating systems, this may or may not work. I would think that setting up either a Samba or NFS server would be simpler approach to creating neutral storage in a diverse environment.

Final comment: disk management is very rich with a large number of knobs to turn. Don't expect to master this topic on the first pass. I don't know anyone who did.

[fbsduser]

Gosha. You can make 3 primary partitions/1 extended partition or 4 primary partitions. Then you subdivide the primary partition you select for BSD into slices.

[gosha]

Thank you all, and especially ocicat, you're always very exhaustive.

If you are thinking that one disk can be universally used across multiple architectures & operating systems

In fact, before buying the disk, I was simply thinking of reformatting it to ffs with my macmini ppc and then use it through it, and in case of ms windows access, use samba. Then bsd666fan warned me that if I format it with ppc I will not be able to mount the disk on another architecture, and in case of hardware (macmini) failure, I would be lost, since I don't have another ppc. So the ext2 thing was not to use it with other systems, but just in case not to "lose" my data.

It is not entirely clear why you want to slice & dice a hard drive to such a fine granularity.

I remember reading I think in the faq, that in case you have to use fsck, if you have a huge 1T partition it will take ages, so I was thinking of making smaller partitions and since now I don't have 1T of data, to just use about a quarter of the drive for the moment.

Don't expect to master this topic on the first pass.

I don't, this is why I ask precious advice from all of you.

Now, what would you do in my place? I'll repeat the scenario: I have a mac mini ppc (G4) on which I have installed OpenBSD (now 4.3, but when I'll start using my new disk, I will do a backup of the whole system and do a clean install to 4.5). I have bought this 1T disk (WD My Book Essential, pretty nice) to use it for backup and to put in it all the data from three different disks I have hanging in two computers. I sometimes need to use my windows box, so I will probably need to access this disk with windows. Also, my mac mini is always on and I can access it from outside with ssh, and in my usb thumb I have a ssh directory with putty in it, so that I can access it from windows boxes also. So I will have access from the outside to the big drive this way.
What should I do, trust apple hardware, and simply format it with my mac mini to ffs and use it only through it? What would you do in my place? And also, would you make smaller partitions or simply use one big 1T? Isn't it better to separate partitions I know will be written very often from those that are written to only seldom?

[ocicat]

Originally Posted by gosha

Then bsd666fan warned me that if I format it with ppc I will not be able to mount the disk on another architecture, and in case of hardware (macmini) failure, I would be lost, since I don't have another ppc. So the ext2 thing was not to use it with other systems, but just in case not to "lose" my data.

One question you need to research is whether an ext2 MBR partition created on one architecture is accessible on another. You are posing further complications by asking whether multiple filesystems on the same disk can be recognized by multiple architectures.

I do not know the complete answer to your question, but I do know that the disklabel structure created can vary from one architecture to another -- possibly to the point of incompatibility. Whether you will run into this nit between the i386 & ppc ports is unknown to me. I suspect you need to simply set it up & test. The versions of OpenBSD used probably need to be very close as well. Your other choice is to post to misc@ & wait for an answer.

I remember reading I think in the faq, that in case you have to use fsck, if you have a huge 1T partition it will take ages...

Section 14.7.

Now, what would you do in my place?

<colloquialisms use="copious">

It sounds like you are putting all of your eggs into one basket by completely trusting this one external drive. Your goose will be cooked if this drive:

• fails.
• is lost due to theft, accident, natural disaster, etc.

</colloquialisms>



Therefore, I would suggest using multiple methods in disaster recovery:

• Honestly decide what information is worth all of this effort.
• Store old stuff that is really worth keeping on a drive or CD-R which is kept off site. Store it in a safety deposit box at a bank, grandma's house, etc. This is why I mentioned Amazon's S3 service before.
• Periodically revisit the stability of these media choices. No one knows whether CD-R's can be read 50 years later because they haven't been around 50 years. The same argument applies to hard drives as well.
• Store newer or more volatile stuff more locally.
• Figure out how often you want to rotate newer stuff to more permanent storage & try to stick to this schedule.
• Factor in the reality that you may or may not stick to this schedule & accept that some data may be lost.

A universal drive which is simply attached temporarily can be part of this solution, but I would not rely on it for the reasons you have already run up against. Again, consider dedicating a machine to serve as a front-end to soften architectural differences -- Samba or NFS -- but the same bullet points above need to be considered with these options as well. Relying on any one solution in disaster recovery makes the consequences very stark. Using multiple methods helps mitigate the risks.

Finally, you are implying that this information is vital. The more important it is, more sophisticated solutions are warranted. As an OpenBSD newbie, you are having to learn more sophisticated technologies which can aid in disaster recovery. Give yourself time to learn them & learn them well. I suspect the first things you need to do are:

• Assess the true worth of your data.
• Study the Sections 4 & 14 of the FAQ in earnest. Couple this with serious study of the related manpages.
• Research the misc@ archives at http://marc.info/. Disaster recovery threads show up periodically. You can read what other people have suggested there.
• Seriously study disaster recovery through books & Google.

Isn't it better to separate partitions I know will be written very often from those that are written to only seldom?

Maybe. Personally, I would use worth & backup frequency as better metrics.

I think you will find in the misc@ archives where Nick Holland states that mistakes happen. Accept it, & factor it into your world view. The real test of preparedness is how quickly can you recover from a catastrophic event -- in whatever form it happens.

[gosha]

Thank you for you answer and thank you to remind me I'm still a newby. I've been using OpenBSD exclusively since a year, and sometimes think I know quite a lot already, but I would not ask so many questions if I did.

It sounds like you are putting all of your eggs into one basket by completely trusting this one external drive.

Well, I already have a copy of everything on dvd, probably some stuff is not on dvd now, but most of it is.
This time I want to do a new "inventary" while transferring data to the new drive.

Why am I putting so much effort on this drive/backup/storage space thing?
There are a few reasons: in my computing experience I found out that if you don't plan things well in advance and if you don't consider future compatibility, you might find yourself in trouble afterwards.
Another reason is that I want to have a unique drive to be used to store my data (I can do further backups to cd dvd). Or even better, I have to learn to backup things in a clever way, that is with a software that backups only things that have changed since last backup (if I'm right dump option 0 to 9), because in the past I often found myself with changed files (the same file) in different places and at the end not knowing wich one I should have kept, and spending time in checking them all. This summer I should finally be able to by myself a notebook, I will need this even more.
Another reason is that I lately heard very many scary story about data loss, the last one this winter when my sister had her notebook stolen, with everything in it and... no backup.
Another reason is that I know by experience that hardware sooner or later will fail (if you remember, last year my notebook's embedded ram failed, so now it's unusable, it boots fine, but if you use it for a while it hangs forever). So this ppc i386 compatibility issue is very important to me.

Honestly decide what information is worth all of this effort.

In life you vey often find out how things are important only when you lose them...

Considering all this and you suggestions, I think I will do this:
Simply format my drive with my mac mini and use it. Double check that everything is on dvd also. Then, from that point on, make sure that new data will be backed up regularly to cd or dvd (maybe get a rewritable cdrom and leave it inside).
In the future I might as well consider on line storing to have an off site backup at least for most important things.

How does it sound?

[BSDfan666]

FFS is sensitive to Endian issues, you cannot mount a FFS partiton created on a macppc on an i386.

I'm about 80% positive that ext2 does not suffer from these Endian issues, you should be able to access the partition on both your i386 and your macppc.

As I said in another similar topic, using FFS on a portable mass storage device is not really an option if you have a mixed environment like yours..

You can have several EXT2 partitions on the disk, that would be the safest bet.. there is FAT, but it does not support Unix file permissions.. so I did not mention it.

Hope that helps.

[gosha]

thanks BSDfan666
I could implement my last solution using ext2 instead than ffs.

You can have several EXT2 partitions on the disk

Does this mean primary partitions made wit fdisk? For multisystem compatibility, do I need to start form sector 63?

[gosha]

I have managed to make the ext2 partition and it works also in windows (I downloaded the driver).
I have some questions: in the ext2 utilities there's a mkfs.ext3 which should enable journaling, I tried it but then OpenBSD would refuse to mount it. It works fine formatting with mkfs.ext2 though.
Also, over 10 G have disappeared, I suppose it is this:

3052383 blocks (5.00%) reserved for the super user

What's the use of it?
Another question: why does Windows report a bigger size?

/dev/sd0i      229G   20.0K    218G     0%    /mnt

Windows 232 217

[BSDfan666]

OpenBSD doesn't support the journaling features of ext3, it mounts it as ext2.. in your case, it didn't work at all.

Sizes are a confusing thing, some people define a kilobyte as 1024 bytes, some define it as 1000 bytes.

Both FFS and EXT2 reserve some space for the superuser, the man page for both should explain how to reduce that reserved space.. but it's generally nothing worth worrying about.

[IdOp]

A small note related to ext2 and compatibility. I once had NetBSD throw me into the debugger when trying to mount an ext2 filesystem that had been created on Slackware 12.2 with its default parameters. It turned out the inode size that had been used was not supported by NetBSD. After reformatting the ext2 with inode size 128 all was well. The morals from this seem to be:

1) Don't use aggressive parameters creating the ext2 if you want it to be widely compatible;

2) Even when you think you're using "normal" default parameters, they may not be supported by all platforms.

It would seem wise then, after creating the ext2 and before filling it with multi-Gigs of stuff, to check it out first on all possible platforms of interest and see if it works.

[BSDfan666]

Thanks for mentioning that, a couple people have experienced bugs on OpenBSD due to inode sizes different then 128.

A patch was sent to the list a couple months ago, but I don't think it had enough testers.. it wasn't committed.

It shouldn't be a problem in gosha's situation, presumably he's going to create his ext2 partition(s) using the e2fsprogs port.

[gosha]

It shouldn't be a problem in gosha's situation, presumably he's going to create his ext2 partition(s) using the e2fsprogs port.

Yes I am, and as I said, it mounts fine on MS Windows.