DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Security

OpenBSD Security Functionally paranoid!

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 22nd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default Somebody want to log into my OpenBSD website/server

When I go into Wordpress plugin, iThemes Security log.
I find many attempts to log in (I am the only user of this test site, with close to 0 content)

I see 5 php-fpm-7.0 process at 25 % cpu load each (Celeron J2900 quad core cpu). The site feels lees responsive.

So my question:

NR1 : What logs or command to use, to monitor this from OpenBSD command line.

NR2: Is modification of pf rulset, to implement something like fail2ban the way forward to free up system resources?



Code:
Invalid Login Attempt	5	2016-12-20 19:01:31	66.199.161.103	admin			
Invalid Login Attempt	5	2016-12-20 17:06:24	173.252.206.2	admin			
Invalid Login Attempt	5	2016-12-20 16:40:49	104.40.85.104	admin			
Invalid Login Attempt	5	2016-12-20 16:06:49	37.187.71.95	admin			
Invalid Login Attempt	5	2016-12-20 14:22:59	168.77.213.88	admin			
Invalid Login Attempt	5	2016-12-20 14:21:30	173.188.123.130	admin			
Invalid Login Attempt	5	2016-12-20 13:09:18	198.1.95.13	admin			
Invalid Login Attempt	5	2016-12-20 13:04:00	104.236.61.28	admin			
Invalid Login Attempt	5	2016-12-20 10:31:06	213.246.42.176	admin			
Invalid Login Attempt	5	2016-12-20 10:25:49	129.128.185.90	admin			
Invalid Login Attempt	5	2016-12-20 09:31:35	69.28.199.240	admin			
Invalid Login Attempt	5	2016-12-20 09:18:35	191.252.63.24	admin			
Invalid Login Attempt	5	2016-12-20 08:50:00	208.75.149.84	admin			
Invalid Login Attempt	5	2016-12-20 07:42:00	212.175.19.78	admin			
Invalid Login Attempt	5	2016-12-20 07:22:38	91.121.154.52	admin			
Invalid Login Attempt	5	2016-12-20 07:07:14	159.253.208.45	admin			
Invalid Login Attempt	5	2016-12-20 06:55:58	38.123.253.149	admin			
Invalid Login Attempt	5	2016-12-20 06:31:21	203.162.76.144	admin			
Invalid Login Attempt	5	2016-12-20 06:17:18	151.236.47.224	admin			
Invalid Login Attempt	5	2016-12-20 05:07:35	167.114.157.235	admin
var/log/php-fpm-log, for weeks reported no problem, now it seem to push against its limits.

Code:
22-Dec-2016 09:23:29] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:26:18] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:31:45] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:33:22] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:33:35] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:34:21] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:35:11] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:35:18] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:35:54] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:36:16] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:36:22] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:36:29] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:36:48] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:37:53] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:56:10] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[22-Dec-2016 09:56:27] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

Last edited by psypro; 22nd December 2016 at 09:53 AM.
Reply With Quote
  #2   (View Single Post)  
Old 22nd December 2016
e1-531g e1-531g is offline
Spam Deminer
 
Join Date: Mar 2014
Posts: 298
Default

Quote:
NR2: Is modification of pf rulset, to implement something like fail2ban the way forward to free up system resources?
"Fail early". Yes, it should. PF blocking IP addresses inside PF table structure is quite efficient. Unfortunately I don't know which script can retrieve IPs from PHP logs and update table.
I have seen some people in Gnu/Linux community to do something similar using pure iptables/ipset solution (ipset is something similar to PF's tables) without fail2ban. At the firewall ruleset they are adding IPs connecting too many times per minute to blocklist.

PS.
Remember to whitelist yours IP address.
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #3   (View Single Post)  
Old 22nd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

You have posted a log showing one login attempt every 20 minutes. If that were all that were occurring, you should not have a resource consumption problem. Something else seems to be going on, perhaps caused by the attacker's script.

# systat states will show you IP traffic. sysutils/pftop will show the same information, but allow you to filter and sort.

If the attacker is flooding the webserver with connection attempts, you could easily block the attacker with PF's Stateful Tracking Options.
Reply With Quote
  #4   (View Single Post)  
Old 22nd December 2016
shep shep is offline
Rc.conf Instructor
 
Join Date: May 2008
Location: Dry and Dusty
Posts: 973
Default

Quote:
var/log/php-fpm-log, for weeks reported no problem, now it seem to push against its limits.
I have the sense that there is an escalating cyberwar going on right now. Two nights ago my internet access slowed to the point of timing out. I ran a traceroute to www.google.com and the 4th hop to Seattle -> LA had > 400ms on each relay and eventually timed out. One measure of success would be to weather the assault - didn't phase us, is that the best you got?
Reply With Quote
  #5   (View Single Post)  
Old 22nd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

With the information we have here, it is not certain what is occurring.

If there is network abuse, or an "attack" occurring, then:
  • If there is abuse occurring from individual addresses, then PF alone can block the abuse. Example: one IP address creating high numbers of parallel connections. Stateful tracking can kill states and block the IP addresses.
  • If the abuse is widely distributed, such that only the traffic in aggregate causes the problem, then PF can be used to mitigate the abuse, but may not be able to eliminate it. Example: small number of connections from individual IPs, but a large number of IP addresses making connections in parallel. Stateful tracking can limit the number of connections permitted to pass to the webserver.
Reply With Quote
  #6   (View Single Post)  
Old 22nd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default

Obama did promise Putin a return gift of cyberware actions, so who knows.The few IP i bother to manually look up all came from North east US.

What I do know, it that following the input from this friendly forum, i added this to my pf.conf

Code:
pass in on egress proto tcp to $web_server port www keep state   \
                  (max 200, source-track rule, max-src-nodes 100, \
                   max-src-states 3)
And php-fpm-7.0 is back to 0 load, as 0 content site should approximately have.

I understand better the author of the Absolute OpenBSD book, it a nasty world/net out there arm your self. And keenly remember all the fun jokes/stories from admin life in the front-line battlefield. I do hope he will update it.

Last edited by psypro; 22nd December 2016 at 07:49 PM.
Reply With Quote
  #7   (View Single Post)  
Old 22nd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

Your rule is the first example from the PF User's Guide, without change. It limits the webserver to 200 states (TCP sessions) in total, with no more than 100 unique IP addresses, and with a maximum of 3 states permitted from any single IP address.

As these particular tracking options are applied on the first incoming SYN packet before a state (TCP session) has been established, if a SYN packet exceeds any of these limits it is dropped.

The second example tracks the rate of incoming sessions, and adds violating IPs to a table of abusers ("overload") and also kills existing states with that IP address ("flush").
Reply With Quote
  #8   (View Single Post)  
Old 22nd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default

I added the second part as well.

Code:
pass in on egress proto tcp to 192.168.0.2  port 80 keep state   \
                  (max 200, source-track rule, max-src-nodes 100, \
                   max-src-states 3)

table <abusive_hosts> persist
block in quick from <abusive_hosts>

pass in on egress proto tcp to 192.168.0.2  port 80 flags S/SA keep state \
(max-src-conn 100, max-src-conn-rate 15/5, \
overload <abusive_hosts> flush)
Reply With Quote
  #9   (View Single Post)  
Old 22nd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

In PF (except for quick rules), the last matching rule wins. Your first pass rule will never be applied, as it will never be the last matching rule.
Reply With Quote
Old 22nd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default

Hmm, you are right. I see 3 php-fpm-7.0 process at 9 % load each.
So the first rule is better if I must chose.

So the problem is the values in the second rule are to lose?
Reply With Quote
Old 22nd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

Both of those rules were examples. Merely examples. They were not intended as actual values that would be perfect for psypro and this single-user Wordpress application.

In order to determine what rules would be the best for your application and server, you must examine your incoming traffic and its patterns of proper use and of misuse.

Stateful tracking is not the only solution. You could also pass or block based on IP addresses, rather than leaving the webserver open to the entire Internet. You could even block or pass based on OS "fingerprints" associated with specific operating systems.
Reply With Quote
Old 23rd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default

Code:
pass in on egress proto tcp to 192.168.0.2  port 80 keep state   \
                  (max 200, source-track rule, max-src-nodes 100, \
                   max-src-states 3)
Works perfect.

Then I need to change web server back to nat/firewall pc, for stability it is always one, and to free up hardware and unbound dns and splitt dns was a pain for now to figure out.

Code:
pass in on egress inet proto tcp from any to (egress) port 80

pass in on egress proto tcp to 192.168.0.1  port 80 keep state   \
                  (max 200, source-track rule, max-src-nodes 100, \
                   max-src-states 3)
Does not give as efficient protection. Reduce the php-fpm spam with 60 % but not 99 % as the top example and network, and hardware setup)

How to force port 80 traffic reaching the firewall, to go trough the pf filter, like before?

Last edited by psypro; 23rd December 2016 at 02:48 PM.
Reply With Quote
Old 23rd December 2016
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 5,588
Default

Quote:
Originally Posted by psypro View Post
Code:
pass in on egress proto tcp to 192.168.0.2  port 80 keep state   \
                  (max 200, source-track rule, max-src-nodes 100, \
                   max-src-states 3)
Works perfect.
I assume that 192.168.0.2 is a webserver inside your local network, such as:
Code:
{Internet} - a.b.c.d - [Router/PF] - 192.168.0.1 - {LAN} - 192.168.0.2 - [Wordpress server]
Quote:
Then I need to change web server back to nat/firewall pc, for stability it is always one, and to free up hardware and unbound dns and splitt dns was a pain for now to figure out....How to force port 80 traffic reaching the firewall, to go trough the pf filter, like before?
I assume now that you want to move the webserver into your router, such as:
Code:
{Internet} - a.b.c.d - [Router/PF/Wordpress]
If so, your second pass rule will never match any Internet traffic. The local network address 192.168.0.1 is not used by any traffic from or to the Internet. Put your stateful tracking options on the first rule, and delete the second rule (unless you need it to permit local traffic to your webserver).

Last edited by jggimi; 23rd December 2016 at 03:06 PM. Reason: typos, clarity
Reply With Quote
Old 23rd December 2016
psypro psypro is offline
Shell Scout
 
Join Date: Mar 2016
Location: Continent:Europe
Posts: 123
Default

Thank you, now I can have a peaceful Christmas.

With the concept of merging the protection from the second into the first in mind, I managed to do it. : )
All service running from nat/firewall, and php-fpm back to close to 0 load.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
A startup website in less than 12h Oko Off-Topic 2 11th March 2016 05:13 PM
blocking a website with pf pawaan General software and network 7 29th October 2013 02:28 AM
Problem with just one website !? Redrobes OpenBSD General 18 7th February 2010 07:11 PM
Book/website recommendations for IPv6 programming mdh Programming 3 7th November 2008 07:53 PM
the website is down ai-danno Off-Topic 2 1st July 2008 11:35 PM


All times are GMT. The time now is 02:41 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick