OpenBSD starts painfully slow on one of two laptops - Why?

[nipos]

Hello guys,I'm new here and also new to OpenBSD,which I first tried about a month ago.
I installed it on two totally different laptops.
In both cases,it's the only BSD that has all important drivers and I wanted to finally get rid of Linux for various reasons.
The one laptop has a Intel Atom x7-Z8700 CPU with 4x1.6 GHz and 64GB eMMC storage,the other has a Intel Pentium Silver N5000 CPU with 4x1.1 GHz and 250GB NVME SSD storage.
With Linux and FreeBSD,both start within only a few seconds and perform well.
Now,with OpenBSD,the second laptop takes about 2-3 Minutes to boot to the graphical Login Prompt,while the other laptop takes only 10-20 seconds,which is totally okay.
Both systems were installed in the same week with nearly identical configuration.
I get that the CPU of the second laptop is a bit slower,but that doesn't explain this huge difference.
Also,it took me a few hours to install OpenBSD on that laptop because the installer took more than a minute each time to respond to most of the questions,while it responded nearly instant on the other laptop.
After the boot is finished,the "problematic" laptop also works rather good,sometimes WebKit2GTK-based browsers hang,but it can even play videos smoothly in Firefox.
Does anyone have a idea what causes that huge boot delay on one of the laptops?
I suspect it doesn't like that specific hardware,maybe some driver issue?
Is there a change to work-around it using some configuration options?
Or can I try to fix it in the source code with my very very very limited C knowledge? (But I need to know what to look for,in the first place...)

[jggimi]

Hello, and welcome!

Would you post the dmesg(8) of the slow platform, please? It may help.

[nipos]

Here's my dmesg: https://paste.bka.li/view/5bedb26a

[Head_on_a_Stick]

I can't see anything noteworthy in that but I may have missed something.

This is a UEFI machine though and OpenBSD uses the "removable" EFI loader location to boot so perhaps the UEFI firmware is just timing out waiting for other devices.

Can you set the hard drive itself to boot first in the firmware ("BIOS") options? That might help.

You could also try a custom NVRAM boot entry for OpenBSD by running this command from any Linux live ISO image booted in UEFI mode:

# efibootmgr --create --label 'OpenBSD' --disk /dev/sdX --part Y --loader '/EFI/Boot/bootx64.efi'

Replace X & Y with the drive letter and partition number assigned to the EFI system partition. The command assumes /dev/sda1 so if that is correct those options can be omitted.

EDIT: another possibility is old, stale NVRAM boot entries that no longer point to an extant EFI loader.

Use the plain efibootmgr command (with no options) to see the boot entries then use

# efibootmgr -b xxxx -B

to delete boot entry xxxx. Replace the x's with the actual bootnumber(s) though, obviously.

[nipos]

The internal SSD is set as first boot option already.
And I don't think it has anything to do with the UEFI settings,as it's slow in OpenBSDs own boot process.
For example,it hangs a quite long time at reordering libraries and then again at starting the X server.

[jggimi]

Just like Head_on_a_Stick, I don't see a probable cause of the delay from your dmesg(8) alone. I can see its a 4GB RAM amd64 platform running the most recent kernel syspatch(8), with Intel DRM graphics.

• Reordering libraries depends primarily on disk I/O for its performance -- see the function reorder_libs() in /etc/rc.
• Starting the X server could have multiple delay causes, the file /var/log/Xorg.0.log will show X provisioning decisions and any warnings or errors produced.

[nipos]

Here's my Xorg log: https://paste.bka.li/view/64c8df4c
Additionally I wrote a empty file with 1GB size to the disk to see if I/O performance is the bottleneck,but the dd command finished in 4.593 seconds,so that's clearly not the issue.

[jggimi]

The Xorg log looks OK to me.

Two other considerations:

1. During multi-user boot, along with library relinking discussed above, the bootup also starts a new kernel relink in a background task, for use on the next boot. This is called Kernel Address Randomized Link ("KARL"), and was introduced as a security feature in 2017. This takes longer than relinking libraries, and consumes a lot of RAM. This particular platform has only 4GB of RAM. On your first login, check the output of top(1) and look to see if any swap space has been consumed. I operate several OpenBSD/i386 machines that have only 256 MB of RAM, and relinking during boot consumes lots of swap space on those systems. Once the new kernel has been linked, performance returns to normal.
2. Some delays can be caused by network-provisioning issues, such as an inoperative reverse DNS function. Compare the network provisioning of the faster machine to the slower, noting if there are any differences in interface setup, routing, and DNS resolution.

[nipos]

I don't have any swap partition on any of the laptops as 4GB ram is more than enough for me.
Both have 4GB memory,so having too few RAM available can't be the reason for the slow boot on one of them as well.
On both devices,top shows a bit more than 300MB RAM used,about 800MB in cache and lots of free space.
Also,both laptops are connected to the same Wifi network and if I disable the Wifi connection of the slower laptop (moving hostname.iwm0 to somewhere else) that removes the 2 seconds delay at "starting network",but doesn't speed up the boot progress significantly otherwise.

[jggimi]

You may want to broaden your audience for this issue -- consider posting to the misc@ mailing list with your issue description and dmesg.

The only other suggestion I would have would be to disable KARL to see if that is the root cause of the delay.

[nipos]

Thank you very much for your help.
I searched a bit for how to disable KARL but couldn't find anything.
Can you please tell me how KARL can be disabled?

[Head_on_a_Stick]

echo no > /usr/share/compile/GENER*/SHA256

Reference: https://marc.info/?l=openbsd-tech&m=149732026405941

[jggimi]

I don't think /usr/share/compile/* exists these days.

When needed to, I have disabled it through the checksum hash file, with # echo disabled > /var/db/kernel.SHA256. I can then re-enable with # sha256 -h /var/db/kernel.SHA256 /bsd.

[nipos]

That worked,it now shows a message that kernel relinking failed at boot.
It didn't really help against the slow boot,however.
It's still stuck at reordering libraries for a quite long time.
And I know that it may be nonsense,but I have the feeling that Xorg/Xenodm starts *a lot* faster if I move the mouse cursor on the black screen.

[jggimi]

At least you have eliminated KARL as the bottleneck.

[nipos]

I did some more research and I'm finally getting an idea what goes wrong here.
I found this interesting topic about the same problem: https://teddit.net/r/openbsd/comment...ncredibly_slow
They recommend disabling library relinking with rcctl disable library_aslr,which I did and now my laptop really boots a lot faster.
I'm still not quite happy with the overall system performance,so I tried some of the things mentioned in that thread.
While not doing complicated things,apm -v shows:

Battery state: high, 95% remaining, 461 minutes life estimate
AC adapter state: not connected
Performance adjustment mode: auto (800 MHz)

When running openssl speed sha256,apm -v shows a little increase in CPU speed:

Battery state: high, 95% remaining, 364 minutes life estimate
AC adapter state: not connected
Performance adjustment mode: auto (1101 MHz)

That's only the base frequency,that CPU supports a turbo boost up to 2.70 GHz according to Intels website: https://www.intel.de/content/www/de/...fications.html
It looks like OpenBSD isn't using the turbo boost feature on my machine at all,which results in bad performance at compute-heavy tasks.
The output of openssl speed sha256 also isn't that great compared to that in the thread I linked above:

Doing sha256 for 3s on 16 size blocks: 4044718 sha256's in 2.88s
Doing sha256 for 3s on 64 size blocks: 1363932 sha256's in 2.78s
Doing sha256 for 3s on 256 size blocks: 376156 sha256's in 2.93s
Doing sha256 for 3s on 1024 size blocks: 120761 sha256's in 2.98s
Doing sha256 for 3s on 8192 size blocks: 16626 sha256's in 3.01s
LibreSSL 3.6.0
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
sha256 22470.66k 31399.87k 32865.51k 41496.40k 45249.23k

Any idea how I can force it to make use of the turbo boost?

[jggimi]

You can force performance to 100% with $ apm -H, see the apm(8) and cpu(4) man pages.

[nipos]

I did some more experiments about the performance thing.
apm -H forces the cpu to stay at 1101MHz permanently,the value it automatically set when doing something like the openssl speed benchmark.
I misinterpreted that value,however.
As https://forums.freebsd.org/threads/f...boboost.65842/ shows,the +1MHz is a sign that turbo boost *is* supported and used,so everything is okay here.
Maybe I should just accept things as they stand currently,at least the huge boot delay could be reduced and it's fast enough for surfing.

[jggimi]

You might experiment with sysutils/obsdfreqd.

$ pkg_info obsdfreqd
Information for obsdfreqd-1.1.1

Comment:
userland daemon to manage CPU frequency

Description:
obsdfreqd is a CPU frequency scheduler daemon working in userland.

It has many parameters to tweak the frequency like min/max frequency,
polling frequency, inertia, step up/down size. obsdfreqd also support
limiting the frequency upon reaching a given temperature to avoid
a system spending time above the threshold. While it has many
parameters, the defaults are good enough for everyone.

Maintainer: Florian Viehweger <redacted>

WWW: https://tildegit.org/solene/obsdfreqd/

[nipos]

Thank you again for your help.
I installed and enabled it and it feels like the speed has already improved with obsdfreqds default settings.
I'll have to keep an eye on that for the next days,but I'm quite happy for now.