DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Packages and Ports

OpenBSD Packages and Ports Installation and upgrading of packages and ports on OpenBSD.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 11th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default 5.7 system freeze - reproducible

I came across something curious. On this machine, httpd hosts two Tor (/usr/ports/net/tor) "hidden service" web sites - both are simple, single page HTML text; one has an animated gif of a printer printing a test page...

Here's the interesting part, when running Torshammer (A Slow POST Denial Of Service Testing Tool) against the server, pointed at the [hidden] site hosting the simple page with the gif image, the computer freezes about a minute into it. This happens every time. None of the individual pieces seem to have a problem - they all work fine separately.

Assuming the problem has to do with the DoS attack on httpd, it's *very* surprising that it causes the system to freeze (or, as they say on the TV series Silicon Valley - the computer "shit the bed" (a company "cakes its pants" (pop culture is fascinating))).
Reply With Quote
  #2   (View Single Post)  
Old 11th July 2015
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,908
Default

There is a recent thread on misc@ regarding httpd buffer exhaustion, which your DoS testing may be incurring. It affects both -stable and -current. From the thread, it has appeared on amd64, but i386 may also be susceptible.

http://marc.info/?t=143614704800002&r=1&w=2

Even if yours is not the same problem, -dvvv may tell you what is occurring.

Last edited by jggimi; 11th July 2015 at 05:50 PM. Reason: typo
Reply With Quote
  #3   (View Single Post)  
Old 11th July 2015
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,908
Default

Additionally, and perhaps more importantly, net/tor received a -stable update on 9 April 2015, which you might not be using. See revision 1.75.2.2 in the CVS web portal log for ports/net/tor/Makefile.

According to the Tor Project link found in the -stable update log entry, this update affects reliability and availability for hidden services.

If you have not deployed this -stable port, either build the port from a -stable ports branch, or use the -stable package update services offered by mtier.org.
Reply With Quote
  #4   (View Single Post)  
Old 11th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default feeling creative (in a haze of bong smoke)

Thanks, jggimi! You're like a well organized army of on-line technical support. (And you have the chomps to speculate like a engineer (or a consultant to developers)).

On the enthusiastic explorer side of things, I would have guessed/hoped that there would have been some process [*resource* & access] isolation on such a server {httpd}.

Would systrace possibly have the capabilities to enforce resource access and restrict a process (like httpd) in such a way that it can't render the system into a state of guru meditation? ( systrace takes a security perspective ).
Reply With Quote
  #5   (View Single Post)  
Old 11th July 2015
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,908
Default

Systrace will kill a process that violates its ruleset, unless it is run interactively. But those rules are by syscall, not by resource consumption limits. That's what login.conf and various ulimits are for. I think it would be the wrong tool. As you like analogies, lets say I believe it may be the wrong hammer for this particular window glazing task.

If you believe httpd is at fault, try a different webserver. We've got lots in the ports tree.

If you believe the fault is net/tor, try the -stable package, if you haven't already.

It could just be your stress test is too stressful for your i386 machine.
Reply With Quote
  #6   (View Single Post)  
Old 11th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default

Quote:
Originally Posted by jggimi View Post
Systrace will kill a process that violates its ruleset
Linux cgroups and FreeBSD sysjails are interesting to think about and that leads to [architectural] thoughts of ChromeOS, coreOS, Docker, and Kubernetes (fun stuff to speculate about), but I think some kind of containerization at the process level might be nifty on OpenBSD - process containment that could communicate to the running app/proc the current resource usage and other events so the process could throttle itself rather than trample the system or crash the system or compromise the system. Of course, some recalcitrant processes will need to be shot in the head.

Quote:
Originally Posted by jggimi View Post
It could just be your stress test is too stressful for your i386 machine.
I don't think is was a stress issue in the sense of resource exhaustion. CPU and Memory were both reasonably low - far less load than compiling software, e.g., from /usr/ports.
Reply With Quote
  #7   (View Single Post)  
Old 11th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default

Quote:
Originally Posted by jggimi View Post
Additionally, and perhaps more importantly, net/tor received a -stable update on 9 April 2015, which you might not be using. See revision 1.75.2.2 in the CVS web portal log for ports/net/tor/Makefile.
It's a fresh 5.7 installation and update[-stable] (last couple of days).

Code:
hanzer[/usr/ports/net/tor] $ cat distinfo                                                                                                       
SHA256 (tor-0.2.5.12.tar.gz) = VQ/a//60weMDW7jMQubknVrxeteVY70RivIsEQf3K0k=
SIZE (tor-0.2.5.12.tar.gz) = 3311423
hanzer[/usr/ports/net/tor] $ tor --version
Tor version 0.2.5.12 (git-99d0579ff5e0349f).
Quote:
Originally Posted by jggimi View Post
According to the Tor Project link found in the -stable update log entry, this update affects reliability and availability for hidden services.

If you have not deployed this -stable port, either build the port from a -stable ports branch, or use the -stable package update services offered by mtier.org.
I'm having a look at Tor on OpenBSD (and Erlang on OpenBSD).
Reply With Quote
  #8   (View Single Post)  
Old 11th July 2015
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,908
Default

OK, then you have the -stable package deployed. If the problem continues, you can try httpd(8) with -dvvv as mentioned in my first reply above, to see if you receive debug messages indicating that you to are having the resource issues mentioned on misc@, or, you can try an alternate webserver if you believe httpd(8) is the root cause of the problem. Or both.
Reply With Quote
  #9   (View Single Post)  
Old 11th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default

Quote:
Originally Posted by jggimi View Post
OK, then you have the -stable package deployed. If the problem continues, you can try httpd(8) with -dvvv as mentioned in my first reply above, to see if you receive debug messages indicating that you to are having the resource issues mentioned on misc@, or, you can try an alternate webserver if you believe httpd(8) is the root cause of the problem. Or both.
It would be interesting to locate the point of failure and I'll probably pursue that to some extent (somewhat casually). As part of the Erlang research & exploration, I might set up Yaws and stress test it - to see an Erlang system under load (on OpenBSD). Might as well hook up Yaws to Tor and throw the Slow POST DoS attack at it.
Reply With Quote
Old 12th July 2015
hanzer's Avatar
hanzer hanzer is offline
Real Name: Adam Jensen
just passing through
 
Join Date: Oct 2013
Location: EST USA
Posts: 314
Default

Forked the crash problem to the thread: "Tor-0.2.5.12 can crash OpenBSD-5.7-stable"
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Tool causes Apache web server to freeze J65nko News 1 25th August 2011 12:00 AM
Create MS/XP file system so it will be recognized on a XP system. FBSD Guides 0 1st May 2010 06:49 AM
System freeze when connecting USB flash drive (4.4 & 4.5 but worked in 4.3) GardenGnome OpenBSD Installation and Upgrading 11 7th May 2009 02:32 PM
Xorg freeze? xfel OpenBSD Installation and Upgrading 2 6th November 2008 09:37 PM
freebsd 6.0 system freeze: kernel(smp) problem? nkiuuu FreeBSD General 3 17th June 2008 09:57 AM


All times are GMT. The time now is 02:25 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick