Single User Shell Terminated

[stanl]

I am running OpenBSD -current on a ThinkPad X220.
Trying to do a sysupgrade this morning. All files download with no problem. However upon rebooting I get an endless stream of:

init: single user shell terminated, restarting.

Any suggestions would be appreciated.

Thank you

[jggimi]

I'll have to guess. Most likely, you had provisioned root's shell to use a third party package shell. Third party package shells are stored in /usr/local, and I'll guess that it isn't mounted. To recover to the default shell:

1. Boot into single-user mode (boot> -s), and when prompted for the shell to use, take the default "sh" offered to you, or reply with "/bin/sh" explicitly.
2. Mount all filesystems with # mount -a, which will also mount the root partition read/write so you can make changes.
3. Change root's shell back to its default ksh shell with # chsh -s /bin/ksh root
4. Reboot with # reboot

[stanl]

I never changed root's shell and have no 3rd party package shells installed.
Even so, I tried your suggestion - no good.
Upon rebooting I noticed a message:
bsd.upgrade is not u+x
I chenged to permissions on bsd.upgrade and tried again - same error message - single user shell terminated.

The only changes I've made recently is provisioning proton vpn. I disabled openvpn, undid all the proton vpn changes and tried again - still no good.

I would appreciate any other suggestions you may have.

Thank you

[jggimi]

I don't know what went wrong for you. You have a couple of options for recovery. Restoring from backup is, of course, the safest.

The other option is to bring up your damaged system and run sysupgrade again. I cannot tell you if the problem will repeat, or if the problem will be circumvented.

Steps are similar:

1. Boot into single-user mode, as above.
2. Mount all filesystems, as above.
3. Start your network, with # sh /etc/netstart
4. Rerun # sysupgrade.

[stanl]

jggimi - I did as you suggested. Now when I run sysupgrade I get:

#sysupgrade
Fetching from https://cdn.openbsd.org/pub/OpenBSD/snapshots/amd64/
su: Approval failure

I guess I will have to restore from backup unless you have another suggestion.
Thanks you for your help.

[jggimi]

Unfortunately, I don't have any other suggestions -- your system will not boot multi-user and you cannot correct the error through re-running sysupgrade in single-user mode.

I've just run sysugrade and had no errors. This snapshot's amd64 kernel was built on January 8 at 23:26 MST.

kern.version=OpenBSD 7.2-current (GENERIC.MP) #927: Sun Jan  8 23:26:27 MST 2023
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

[stanl]

I believe I am completely screwed and I'd like to know why.

I have a flash drive with 7.1 on it. I installed 7.1, ran syspatch, upgraded to 7.2 and ran syspatch again all with no problem.

I then ran syspatch -s and again an endless stream of:

init: single user shell terminated, restarting.

I'm just curious if you or anyone has any idea what could have happened.

Thanks

[Head_on_a_Stick]

Originally Posted by stanl

any idea what could have happened

Perhaps the disk drive is dying.

I've used the latest snapshot of bsd.rd to recover a broken -current system in the past. That was before sysupgrade was available and it worked well. Is that an option for you?

I did find a reference to that error message on the mailing lists:

https://marc.info/?l=openbsd-misc&m=126915189823995&w=2

But unfortunately it has no replies.

These are the risks of running a development branch though. Next time stick to -release & -stable if you want a consistently reliable system.

[jggimi]

Don't confuse sysupgrade with syspatch. Only sysupgrade has the -s option.

I don't see anything in the Following -current FAQ that would account for this. https://www.openbsd.org/faq/current.html

Since you started with a working 7.1, you might just use it to download a snapshot.

[stanl]

The syspatch -s in my earlier post was a typo. Should be sysupgrade -s. I tried the bsd.rd suggestion and got the same stream of error messages. I removed the hd and replaced it with an old one from an old laptop which has 7.2 install on it. After installing and trying sysupgrade -s got the same stream of errors.
So it isn’t the hd and must be the laptop itself (; cpu?).
I guess I can either stick with what I have and give up on -current or buy a new laptop.

Thanks to Head_on_a_Stick and to jggimi for their assistance.

[jggimi]

Have you ever attempted to install a snapshot? That is, not starting with a prior release? If so, did you get the same failure?

[stanl]

I've been running -currrent for a while now and doing sysupgrade -s maybe evry 2 weeks with no problem.

I do recall several days ago power went out in the neighborhood for about a minute. My laptop was powered on but i wasn't using it at the time. Do you think the power outage could have messed something up internally?

[jggimi]

If the laptop had a battery installed, it would have continued running after a power failure. But, a brownout or a surge could have caused damage.

Until you test a directly installed snapshot, you won't know if its something you did to cause the breakage, some sort of rare problem in the sysupgrade process, or some sort of hardware problem.

A failed direct install would seem to indicate a hardware problem. A working install would inform you that the problem *isn't* hardware.

[stanl]

jggimi - It took me a while to understand what I "think" you're trying to tell me: Go to one of the mirror sites, download a snapshot directly and try to install it.
When I did I got the same stream of errors:
init: single user shell terminated, restarting.
I guess this means a hardware problem and it is time for a new laptop. Any suggestions?
And, as always, thank you for your help.
Stan

[jggimi]

I think it is a hardware issue...but more likely to be a bug in the latest snapshots rather than bad hardware.

If this was happening in one of my systems, I would be bisecting kernel builds to find the commit which introduced the problem, so that I could report it to the openbsd project.

[TronDD]

When you sysupgrade -s does it complete the install then when rebooting, spit out the errors or does booting into the installer cause them?

During install, /bin/sh is running from the ramdisk, not what's been previously installed.

[stanl]

jggimi - If it was a bug in the latest snapshot wouldn't it affect everyone, not just me? As for bisecting kernel builds that is way out my abilities. Do you think I should wait a week or so and try another snapshot?

TronDD - sysupgrade -s downloads the files with no problem. Laptop reboots, bsd.upgrade detected,runs thru about a screenful of details and then goes into an endless stream of:
init: single user shell terminated, restarting.

I had no problem upgrading from 7.1 to 7.2. I just can't upgrade to a snapshot.

My thanks to both of you.

Stan

[TronDD]

So sounds like it's after the upgrade runs and it tries to reboot back into the installed system?

WHen it's broken, at the boot> prompt can you 'boot bsd.rd' and get to a prompt? Or 'boot bsd.sp'? Try the different kernels.

If you can get to a ramdisk, you can mount and chroot into your installed system and see what might be going on.

[jggimi]

My working assumption is that you have something unique about your hardware platform, and therefore you are one of only a few people suffering from the problem.

You certainly could run 7.2 for now, and wait and try what will become 7.3 another day. I recommend careful backup before your next test.

[jggimi]

Originally Posted by stanl

As for bisecting kernel builds that is way out my abilities.

• By itself, kernel building is easy. You "cd" into the appropriate directory and issue three short `make` commands. This results in a single "bsd" kernel file stored in a sub directory.
• Testing that freshly built kernel is also easy. You copy the "bsd" into your system's root directory, giving it any sort of file name you wish, and then simply boot that kernel to test it, selecting it by name at the "boot>" prompt. If it works, great. If it fails, also great, because you haven't overlaid anything and can reboot with your default and known-to-be-good kernel.
• Bisecting through the source code to find the commit where the problem begins? Mostly tedious. You start with two known states: a point-in-time where source code is known to be good, and a point-in-time where the source code is known to be bad. Then you revise the state of the source code's working directory tree back and forth until you find the failure point. Once you've narrowed down the failure to a single commit, you have determined the commit that introduced the problem.

OpenBSD's source code is maintained via CVS, and users can use the Project's AnonCVS servers with the cvs(1) utility to manage local working source trees. While it is possible to manually bisect by date/time, or even by individual commit, I find it cumbersome to do so. I prefer using the Project's GitHub Mirror to establish and maintain my source tree, as the devel/git package includes the git-bisect(1) tool which automates the hunt for the problem commit.

I can usually identify the commit with less than a dozen build/test cycles.