DD command on OpenBsd

[PapaParrot]

I have used the dd command in the past, but it was on linux system, and am having trouble using it on OpenBsd.

# dd if=/dev/sda of=/dev/sdc bs=8M; sync

the sda is the HD and sdc is a portable USB drive, the problem, if I do not mount them , it says it does not see them.
I am using another USB drive, with OpenBsd on it, it is bootable and works fine, Actually I made it some time ago, but using a portable usb drive, with Linux on it, and I copied my OpenBsd HD, to the device, using the above command with no problem.
The devices all show ok when I use:

sysctl hw.disknames

, so it seemed like it should work. Any ideas ?
Thanks

[ibara]

sda, sdb, etc. are Linuxisms. That's not how the BSDs identify their disks.
On OpenBSD, they would be sd0, sd1, or something like that.
For example, on my machine,

$ sysctl hw.disknames

returns

hw.disknames=sd0:585806def6f81d18

meaning that my hard disk is sd0.

[PapaParrot]

Ok, yes I know that, my fault though I showed what I used when I had run it from a Linux system.
When I run the command using OpenBsd, it is :

# dd if=/dev/sd0 of=/dev/sd2 bs=8M; sync

But , it says it can not see the devices,....Thanks

[jggimi]

You MUST use partition letters. Partition c maps to the physical device. Using the raw device node will also improve performance. Use rsd0c, for example.

[PapaParrot]

Thanks, now that makes sense and I will give it a try .
===edited====

# dd if=/dev/rsd0c of=/dev/rsd2c bs=8M; sync

Yes, looks like it is working, now I just have to wait several hours and see how it turns out.

[jggimi]

Great!

Some utilities, such as fdisk(8) and disklabel(8), will automatically translate a short form disk device name for you. But when they do this, they note it in the man page. Here, for example, is an excerpt from the disklabel(8) man page:

     disk    Specify the disk to operate on.  It can be specified by its full
             pathname, by an abbreviated disk form, or by its disklabel UID.
             In its abbreviated form, the path to the device, the `r' denoting
             "raw device", and the partition letter, can all be omitted.  For
             example, the first IDE disk can be specified as either
             /dev/rwd0c, /dev/wd0c, or wd0.

But dd(1) cannot do this, because the if and of operands are not disks, they are path names. Note this excerpt from the dd(1) man page:

EXAMPLES
     Write an OpenBSD floppy image to a floppy disk:

           # dd if=floppy34.fs of=/dev/rfd0c bs=32k

     Create an ISO-9660 image of a CD-ROM:

           # dd if=/dev/rcd0c of=disk.iso bs=32k

[PapaParrot]

Thanks,
It did finish, but for some reason the system on USB device does not boot properly.
It boots , but fails, saying to run fsck_ffs manually . I am looking at the manual on that just now:

SYNOPSIS
     fsck_ffs [-fnpy] [-b block#] [-c level] [-m mode] filesystem

I am not sure what to use in the syntax , which options, for example, "filesystem" ? , How do I know, what block# ? Same, on level and mode, maybe the manual will clarify.
In any event it (the USB drive), does mount ok, and for example, I mounted the sd1k, which is my home directory , everything is there as it should be,... so any way, I am working on it now. Thanks

[jggimi]

I will assume you were copying sd0 partitions while they were mounted and in use. If so, I will assume that your /var partition had files open for writing, and an automatic "preen" of the open filesystem failed, requiring manual fsck(8).

To avoid this in the future, either boot into single-user mode (-s), which starts with only the root partition mounted, and mounted read-only. You could also boot the RAMDISK kernel, which has a small in-memory filesystem which I believe includes the dd(1) utility.

[PapaParrot]

No, actually they were not mounted, I used another usb drive to boot with and run the command, so that sd0, and sd2 were not mounted. The drive I booted with was sd1, all 3 did show when I used the hw.disknames command.

[jggimi]

Here's some background.

1. At boot time, init(8) mounts the root partition read-only, and passes control to rc(8).
2. Before mounting any partitions, rc(8) issues fsck(8) with the -p operand.

   -p	Enter preen mode: fsck will check all file systems listed in 
   	fstab(5) according to their pass number ... and will make minor
   	repairs without human intervention. Any major problems will cause
   	fsck to exit with a non-zero exit code, so as to alert any
   	invoking program or script that human intervention is required.

   During this process, fsck(8) skips any filesystems that were flagged as having been correctly unmounted at last usage, as there is nothing to check.
3. Any filesystem that was not marked "clean" will be inspected, and minor inconsistencies repaired.
4. If all filesystems were skipped or successfully repaired, fsck(8) returns with a zero return code, and rc(8) will continue with bringing up the system in multi-user mode.
5. If any filesystem had inconsistencies which were NOT minor in nature -- meaning, that tthere is the possibility of data loss during filesystem correction -- then rc(8) will stop processing at the completion of the fsck(8) operation, to permit the admin to run fsck(8) manually.

Because fsck(8) returned a non-zero return code, your sd0 drive had one or more filesystems that were not closed successfully when last used, AND one or more filesystems that had more than minor inconsistencies.

I will revise my assumption. Your sd0 drive is either actively in use or was previously in use without a clean unmount of all of its partitions.

[jggimi]

And I'll stick with my assumption of likely active use. By default, OpenBSD uses DUIDs in fstab(5). You can boot your kernel from sd1 and still mount all partitions from sd0, depending on the contents of the fstab(5) and the DUIDs of the disk drives involved. df(1) can tell you which device numbers have partitions mounted.

[PapaParrot]

Ok, thanks again for all the time on this, So, let me see if I understand correctly,

df(1) can tell you which device numbers have partitions mounted.

If I use 'df", to check, and neither , for example, if sd0 or sd2, do not show , or any partitions on them, then I can safely assume neither is mounted ?
It is kind of strange to me, because I have done this several times in the past, and never encountered this type of problem, also , now I am running into a new problem, but different topic all together, I will post about that separate, though. But in a nut shell, several things are going on that I have not seen before. I plan to upgrade the system to 6.4, but am now thinking maybe better to do a fresh install.
I wanted to make a copy of my current system, that I would have on the usb drive, in case something goes wrong. In any event, the one portable USB drive, has OpenBsd 6.3 on it, and it boots fine, but it is a little old, but has everything that is essential to me (important data), ... When I made that drive, I used the dd command, same as above, and did not encounter this problem. I must say, I had used a Linux live device, for that, but either way I am getting the same results now, I have tried 3 times, and get the same, a device that does not boot properly, all though all the data appears to have been copied correctly.
thanks again,

[jggimi]

There are only two ways a filesystem can be flagged dirty when copied by dd(1). You are certain it is not due to being mounted. The only other possibility is an unclean shutdown.

[Head_on_a_Stick]

@Garry, if you want to backup your system check out the /altroot facility:

https://www.openbsd.org/faq/faq14.html#altroot

You will need to configure the bootloader for the backup disk though, apparently.

[PapaParrot]

Thanks, H-O-A-S, I am looking at that right now, but will need to read it more carefully later.

You are certain it is not due to being mounted. The only other possibility is an unclean shutdown.

I need to double check on this, it is possible also that the usb drive is not shutting down properly, to be honest , I did not pay much attention when I shutdown, after the dd command was finished, I used:

 shutdown -p now

, but did not watch what all it said when shutting down,...
Not related directly, but there is a lot going lately in my personal life, activity and it is making it extra difficult to concentrate, etc.

[PapaParrot]

Here is what it says when I run the 'fsck_ffs' command , on the device, it is not mounted at this time:

garry# fsck_ffs -fyp /dev/rsd2c 
/dev/rsd2c: BAD SUPER BLOCK: MAGIC NUMBER WRONG

/dev/rsd2c: UNEXPECTED INCONSISTENCY; RUN fsck_ffs MANUALLY.
garry# fsck_ffs  /dev/rsd2c      
** /dev/rsd2c
BAD SUPER BLOCK: MAGIC NUMBER WRONG
/dev/rsd2c: NOT LABELED AS A BSD FILE SYSTEM (unused)

[ibara]

I mean, fsck(8) and its ilk don't run on raw devices. You need to specify a file system on the sd2 (in your example above) device. Try maybe:

# fsck -fyp /dev/sd2a

or something like that.

[jggimi]

Garry, it may help to consider that storage devices are a collection of blocks - "sectors" - each containing collections of ones and zeros. For mass storage devices such as disk drives (and USB sticks) the leading sectors will contain partitioning information, and may also contain boot programs.

• On OpenBSD, the disk partitioning schema is the disklabel.
• On architectures that use MBR or GPT partition tables, the disklabel is used to subdivide the OpenBSD partition defined in the MBR/GPT table.
• If the system is not multibooted, then there is only one entry in an MBR partition table for OpenBSD, and up to two entries in a GPT table, for OpenBSD and for boot, respectively.
• A disk may have up to 16 disklabel partitions, labelled "a" through "p".
• Each partition in the disklabel has a starting sector number, and a size, in sectors.
• Disklabel partition "c" is special. It is always the entire physical disk, starting at sector #0 and with a size that is the number of sectors on the disk. The 15 other partitions are available for filesystems or swap space: "a" and "b", and "d" through "p"
• On architectures that use MBRs or GPTs the first partition will normally never start at sector #0, to leave room for the MBR, the GPT, or boot programs. You will often see the first partition start at sector #64, or #128. or even sector #2048.

This is why ibara noted your fsck attempt was in error. You chose partition "c", which starts at sector #0, and your filesystems on disk drives should not start with that sector number.

[PapaParrot]

Thanks again, every one, learned a lot here. It is working now.

[jggimi]

Glad you've got it working!

By the way there are storage devices that do not use partitioning, such as floppy disks, magnetic tapes, and optical media. But most everything else does.

Yes, you can provision a disk drive without a disklabel, or an MBR, or any other partitioning schema. But this is not an operational best practice. If the leading sectors of a device are not recognizable, it is possible to accidentally overwrite its contents. This is why the GPT standard includes a "protective MBR" in sector #0, so that a system which does not recognize the GPT might recognize the MBR, and the system (or its human) might stop themselves from scribbling on it.