DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Packages and Ports

OpenBSD Packages and Ports Installation and upgrading of packages and ports on OpenBSD.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 27th May 2021
discostew discostew is offline
Port Guard
 
Join Date: May 2021
Posts: 14
Default OpenBSD VM running OpenVPN keeps running out of entropy (viornd driver)

Hi DaemonForums!

I have a 6.8 virtual machine running OpenVPN on a Linux host, and despite all of my efforts OpenVPN keeps shutting down every 4-5 days because it appears to be running out of entropy for PRNG & TLS negotiation.

I realize that running OpenVPN in a VM on a headless host is just about the most entropy-starved setup you could come up with... but it is what it is.

Prior to setting up the viornd driver OpenVPN would only run for a few hours before shutting down. Installing viornd with default host entropy request interval (I think default is 8 minutes?) helped - OpenVPN would run for about a full day before shutting down.

I've progressively decreased the viornd request interval from the default (8 minutes) to the minimum (30 seconds), which as expected, did progressively increase OpenVPN's runtime to a maximum of 4-5 days.. However OpenVPN continues to shut down eventually due to what appears to be entropy starvation.

Code:
$ dmesg

virtio4 at pci0 dev 9 function 0 "Qumranet Virtio RNG" rev 0x00
viornd0 at virtio4: request interval: 30s
Code:
/var/log/openvpn/openvpn.log:

x.x.x.x:58663 TLS_ERROR: read tls_read_plaintext error: CTR_DRBG - The entropy source failed
x.x.x.x:58663 TLS Error: TLS object -> incoming plaintext read error
x.x.x.x:58663 TLS Error: TLS handshake failed
x.x.x.x:58663 SIGUSR1[soft,tls-error] received, client-instance restarting
x.x.x.x:60870 Re-using SSL/TLS context
x.x.x.x:60870 TLS_ERROR: read tls_read_plaintext error: CTR_DRBG - The entropy source failed
x.x.x.x:60870 TLS Error: TLS object -> incoming plaintext read error
x.x.x.x:60870 TLS Error: TLS handshake failed
x.x.x.x:60870 SIGUSR1[soft,tls-error] received, client-instance restarting
x.x.x.x:60493 Re-using SSL/TLS context
x.x.x.x:60493 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
x.x.x.x:60493 TLS Error: TLS handshake failed
x.x.x.x:60493 ERROR: Random number generator cannot obtain entropy for PRNG
x.x.x.x:60493 Exiting due to fatal error
x.x.x.x:60493 Closing TUN/TAP interface
To help troubleshoot, I've also tried monitoring the state of the entropy pool on the Linux host using the 'entropy-watcher' tool, which shows that /proc/sys/kernel/random/entropy_avail is reporting a steady average of around 3000 to 3100 bits.

Every 30 seconds there is a slight dip of between 26 to 30 bits (assuming this is viornd?) however the total steadily climbs back up to the 3000 to 31000 range shortly thereafter.

Code:
$ ./entropy-watcher 
3093 (+3093)
3099 (+6)
3073 (-26)
3074 (+1)
3077 (+3)
3086 (+9)
3087 (+1)
Am I missing something here? Is the OpenBSD entropy source really running out this frequently with the viornd request set to 30 seconds?

I recently built the viornd driver from source for the sole purpose of turning on the kernel debug (see dmesg output) - I am willing to try changing the code to request more frequently than 30 seconds (i.e. every 15 seconds) if you think this will help?? I thought I'd reach out to the forums first to see if I was missing something or there was some other way

Cheers,
discostew

P.S. uname and cpuinfo below for vm / host (Note: VM's CPU configuration is the same / copied from host)
Code:
openbsd-vm$ uname -a
OpenBSD openvpn.mydomain 6.8 GENERIC#97 amd64

linux-host$ uname -a
Linux vmhost.mydomain 4.4.0-210-generic #242-Ubuntu SMP Fri Apr 16 09:57:56 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

linux-host$ cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 158
model name	: Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
stepping	: 10
microcode	: 0xde
cpu MHz		: 984.593
cache size	: 9216 KB
physical id	: 0
siblings	: 6
core id		: 0
cpu cores	: 6
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt ssbd ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds
bogomips	: 5615.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:
Reply With Quote
  #2   (View Single Post)  
Old 27th May 2021
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

random(4) states that "The urandom device produces high quality pseudo-random output data without ever blocking".

I know from a presentation by one of the OpenBSD developers, IIRC it was Theo de Raadt himself, that the Linux urandom sometimes blocks, because it has not collected enough entropy from system events.

Does the OpenBSD package README for OpenVPN say something about viornd(4)?

I would recomment to post your issue on the OpenBSD misc mailing list. Please include a complete dmesg of your OpenBSD VPS (inline, not attached).

EDIT: If you search for "OpenVPN entropy error" you will find that you are not the first to experience this problem.....
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 27th May 2021 at 09:43 PM. Reason: Added search engine suggestion
Reply With Quote
  #3   (View Single Post)  
Old 28th May 2021
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

I'd start with posting to the ports@ mailing list, rather than misc@, as this is a problem specific to a port/package. /usr/ports/net/openvpn/pkg/README does not discuss this issue, nor do I recall seeing it discussed on ports@ previously.
Reply With Quote
  #4   (View Single Post)  
Old 3rd June 2021
discostew discostew is offline
Port Guard
 
Join Date: May 2021
Posts: 14
Default

Thank you for the quick replies J65nko / jggimi.

Yes I did think about the possibility of /dev/(u)random on the host running out of entropy and blocking if viornd exhausted the pool because it is requesting too frequently.. The output of /proc/sys/kernel/random/entropy_avail around / near the time OpenVPN shuts down seems to suggest otherwise, is it's always been running between 3000 and 3100 bits every time I would randomly check.

To look into this possibility further, I did a little test where I modified 'entropy-watcher' to give me a timestamp, and logged the results for a few days. When OpenVPN finally shut down, I then compared the output of /proc/sys/kernel/random/entropy_avail at the exact time when the entropy source started to fail until OpenVPN shut down.

I can confirm that entropy_avail does report that the pool remains in the 3000-3100 bits window on the host from the time OpenVPN reports the entropy source is failing until it shuts down.

Quick question: does anyone know how to query similar / any statistics on the current state of the OpenBSD entropy pool? I'd like to do the same test on the VM guest.

.. Maybe this is an OpenVPN / ports problem, and the host's entropy pool is fine and viornd is working correctly as well.. Perhaps there's just something different about how OpenVPN is using the entropy source on OpenBSD that is causing this?
Reply With Quote
  #5   (View Single Post)  
Old 3rd June 2021
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

I don't know how to check the OpenBSD entropy pool status. It could be complicated because the viornd(4) driver could be involved too.

But I wonder whether it indeed is an OpenVPN issue. A similar report on Linux:
https://sourceforge.net/p/openvpn/ma...e/#msg30396641
An reply in that discussion mentions it is an OpenSSL issue ....

Usually I like to figure out the cause of a problem, but sometimes you need to be practical. Did you try Wireguard yet?

BTW From EuroBSDcon Sofia, Bulgaria 25-28 September 2014:
Randomness: how arc4random has grown since 1998 by Theo de Raadt
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
  #6   (View Single Post)  
Old 4th June 2021
discostew discostew is offline
Port Guard
 
Join Date: May 2021
Posts: 14
Default

Thank you for the suggestion on Wireguard J65nko. Quick question: I already have the OpenVPN client setup on my devices and my wife's devices - does anyone know if you can use the OpenVPN clients to connect to a Wiregard gateway? In particular OpenVPN has fairly nice Mac and iPhone client apps.. I'll have to check if Wiregard has the same.

In the mean time, I punted - I setup a cron job that just restarts the OpenVPN daemon every few hours.. This is sad, but it's an acceptable workaround until I find a better solution..
Reply With Quote
  #7   (View Single Post)  
Old 4th June 2021
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,128
Default

I installed the Android Wireguard app on my Lenovo Android tablet. It is very simple, even for me, who usually, to quote a signature of one of our members is "too dumb for a smartphone" Later I will post a screenshot of that app (in a separate topic/thread)

The list of Wireguard software/apps at https://www.wireguard.com/install/ mention Mac and iPhone. So that should not be an issue.


EDIT: For the WireGuard Andoid app screenshots and the server configuration see http://daemonforums.org/showthread.php?t=11781
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 5th June 2021 at 12:16 AM. Reason: Added link to screenshots
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What do webdevs running OpenBSD, use for API testing ? akts OpenBSD Packages and Ports 2 24th October 2018 05:04 AM
Running gnome-3.24.2 on OpenBSD pawkolor OpenBSD Packages and Ports 7 27th October 2017 05:57 PM
running OpenBSD from live CD's bsd-keith OpenBSD Installation and Upgrading 22 29th March 2015 03:28 PM
Getting OpenBSD up and running Larry OpenBSD Installation and Upgrading 10 14th May 2010 11:14 PM
Anyone running an OpenBSD email server ? roundkat OpenBSD General 9 10th May 2008 03:08 AM


All times are GMT. The time now is 11:25 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick