Openbsd NAT router + pf + suricata

[psypro]

I will update this post a I progress toward working setup. So I myself can use it next time i re install, or might somebody else googling.
I am no expert. Hope somebody will find a thing or two to improve, and comment on it.

I have a OpenBSD firewall with PF. It get fiber broadband in on em3.
I have a network internal usage em2 (dont want tor / p2p, )
I have a network for guests re0 (dont want tor / p2p )



Thanks to :

https://www.solrac.nl/openbsd-firewall-pf/


Part 1: Enable NAT

/etc/sysctl.conf

net.inet.ip.forwarding=1

Use this code to check it is enabled:
(I used hours trying different pf.conf, when this was the problem)

sysctl | grep forward

Should read :

net.inet.ip.forwarding=1

Part 2 : PF

/etc/pf.conf

# Create blocks that are variable
ext_if="em3"
int_if="em2"
int_if2="re0"
icmp_types="echoreq"


# Bruters

table <bruters> persist file "/etc/bruters"

block quick from <bruters>

#Badhosts -DE
table <badhosts> persist file "/etc/badhosts"

block in quick on egress from <badhosts> 


#Badstuff
table <badstuff> persist file "/etc/badstuff"

block drop out log (all) quick on $ext_if from any to <badstuff>

#Max new ssh connections 4 pr 7200s, pr ip
pass quick proto tcp from any to any port 22 \
        flags S/SA keep state \
        (max-src-conn 5, max-src-conn-rate 4/7200, \
        overload <bruters> flush global)
 
# Skip all loopback traffic
set skip on lo
 
# Perform NAT on external interface
match out on $ext_if from $int_if:network to any nat-to $ext_if
match out on $ext_if from $int_if2:network to any nat-to $ext_if

# Define default behavior
block in
pass out keep state
 
# Allow inbound traffic on internal interface
pass quick on $int_if
pass quick on $int_if2

 
# Protect against spoofing
antispoof quick for { lo $int_if $int_if2 }
 
# Allow other traffic
pass in on $ext_if proto tcp to ($ext_if) port ssh flags S/SA keep state

# Let suricata listen
pass out quick on egress inet proto tcp to port 80 divert-packet port 700

Part 3

Script to download IP of bad actors from blocklist.de and send these to badhosts.

#! /bin/ksh

ftp https://lists.blocklist.de/lists/all.txt

cat all.txt  > /etc/badhosts

pfctl -f /etc/pf.conf

exit

Part 4 Suricata

pkg_add suricata

suricata-update

rcctl set suricata flags -i em3

[/HTML]

[psypro]

With this command suricata will start in command line.

suricata -c suricata.yaml -s signatures.rules -i re0

 
3/11/2020 -- 19:28:15 - <Notice> - This is Suricata version 5.0.3 RELEASE running in SYSTEM mode
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_FOPEN(44)] - could not open: "/etc/suricata/classification.config": No such file or directory
3/11/2020 -- 19:28:15 - <Error> - [ERRCODE: SC_ERR_OPENING_FILE(40)] - please check the "classification-file" option in your suricata.yaml file
3/11/2020 -- 19:28:15 - <Error> - [ERRCODE: SC_ERR_FOPEN(44)] - Error opening file: "/etc/suricata/reference.config": No such file or directory
3/11/2020 -- 19:28:15 - <Error> - [ERRCODE: SC_ERR_OPENING_FILE(40)] - please check the "reference-config-file" option in your suricata.yaml file
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/app-layer-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/decoder-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/dns-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/files.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/http-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/smtp-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/stream-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern /var/lib/suricata/rules/tls-events.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] - No rule files match the pattern signatures.rules
3/11/2020 -- 19:28:15 - <Warning> - [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 9 rule files specified, but no rule was loaded at all!
3/11/2020 -- 19:28:15 - <Notice> - all 5 packet processing threads, 4 management threads initialized, engine started.

But rcctl suricata fails.

rcctl start suricata              
suricata(failed)

 rcctl -d start suricata           
doing _rc_parse_conf
doing _rc_quirks
suricata_flags >-i em3rcctl set suricata flags -i em3<
doing rc_check
suricata
doing rc_start
doing _rc_wait start
doing rc_check
3/11/2020 -- 19:40:16 - <Notice> - This is Suricata version 5.0.3 RELEASE running in SYSTEM mode
3/11/2020 -- 19:40:16 - <Warning> - [ERRCODE: SC_ERR_SYSCALL(50)] - Failure when trying to get MTU via ioctl for 'em3rcctl': Device not configured (6)
3/11/2020 -- 19:40:16 - <Warning> - [ERRCODE: SC_ERR_SYSCALL(50)] - Failure when trying to get MTU via ioctl for 'em3rcctl': Device not configured (6)
3/11/2020 -- 19:40:16 - <Error> - [ERRCODE: SC_ERR_INITIALIZATION(45)] - pid file '/var/run/suricata.pid' exists but appears stale. Make sure Suricata is not running and then remove /var/run/suricata.pid. Aborting!
doing _rc_rm_runfile
(failed)

[psypro]

rm suricata pid file. Then it starts witt rcctl.

suricata is starting to fil in information in /var/log/suricata/fastlog

[jb_daefo]

Can you detail the types of NIC on that machine that are involved in this router setup?

[psypro]

ifconfig

(removed public ip)

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
	index 7 priority 0 llprio 3
	groups: lo
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
	inet 127.0.0.1 netmask 0xff000000
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
	lladdr 00:1b:21:18:10:18
	index 1 priority 0 llprio 3
	media: Ethernet autoselect (none)
	status: no carrier
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
	lladdr 00:1b:21:18:10:19
	index 2 priority 0 llprio 3
	media: Ethernet autoselect (none)
	status: no carrier
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	lladdr 00:1b:21:18:10:20
	index 3 priority 0 llprio 3
	media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
	status: active
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
em3: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> mtu 1500
	lladdr XXXXXXXXXXX
	index 4 priority 0 llprio 3
	groups: egress
	media: Ethernet autoselect (100baseTX full-duplex)
	status: active
	inet XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	lladdr c0:3f:d5:ee:9d:0b
	index 5 priority 0 llprio 3
	media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
	status: active
	inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
enc0: flags=0<>
	index 6 priority 0 llprio 3
	groups: enc
	status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136
	index 8 priority 0 llprio 3
	groups: pflog

[psypro]

I added :
stream:
checksum-validation: auto


Auto will turn off checksum-validation if it keep giving errors from what I understand.
Why is it recommended to turn in ON? and then drop the packages.
If one are to use checksum-validation it cant be offloaded to hardware, and in OpenBSD this is stander for hardware with intel em driver and re driver to do.

Outbound packets don't have a correct checksum yet due to checksum offloading,
therefore suricata will log "SURICATA TCPv4 invalid checksum" and ignore these
packets. To workaround this set the following in suricata.yaml:

	stream:
	  checksum-validation: yes

Also adjust the configuration to drop packets:

	- drop:
	  enabled: yes

[psypro]

Part from suricata.yaml

Add own IP.

## Step 1: inform Suricata about your network
##

vars:
  # more specific is better for alert accuracy and performance
  address-groups:
    HOME_NET: "[192.168.1.0/24,192.168.0.0/24]"
    #HOME_NET: "[192.168.0.0/16]"
    #HOME_NET: "[10.0.0.0/8]"
    #HOME_NET: "[172.16.0.0/12]"
    #HOME_NET: "any"

    EXTERNAL_NET: "!$HOME_NET"
    #EXTERNAL_NET: "any"

    HTTP_SERVERS: "$HOME_NET"
    SMTP_SERVERS: "$HOME_NET"
    SQL_SERVERS: "$HOME_NET"
    DNS_SERVERS: "$HOME_NET"
    TELNET_SERVERS: "$HOME_NET"
    AIM_SERVERS: "$EXTERNAL_NET"
    DC_SERVERS: "$HOME_NET"
    DNP3_SERVER: "$HOME_NET"
    DNP3_CLIENT: "$HOME_NET"
    MODBUS_CLIENT: "$HOME_NET"
    MODBUS_SERVER: "$HOME_NET"
    ENIP_CLIENT: "$HOME_NET"
    ENIP_SERVER: "$HOME_NET"

  port-groups:
    HTTP_PORTS: "80"
    SHELLCODE_PORTS: "!80"
    ORACLE_PORTS: 1521
    SSH_PORTS: 22
    DNP3_PORTS: 20000
    MODBUS_PORTS: 502
    FILE_DATA_PORTS: "[$HTTP_PORTS,110,143]"
    FTP_PORTS: 21
    VXLAN_PORTS: 4789
    TEREDO_PORTS: 3544

[SimpL]

Hi psypro,

Did you write the suricata settings into /etc/rc.d/suricata? If you did not then rcctl daemon wont start, and it does not know where your config files are.
Hope this helps.

BR
SimpL

[psypro]

#cat /etc/rc.conf.local
dhcpd_flags=
pkg_scripts=suricata
suricata_flags=-d 700
unbound_flags=

# rcctl check suricata
suricata(ok)

[SimpL]

Wrong file there Psypro
I ment this:
/etc/rc.d/suricata

#!/bin/ksh
#
# $OpenBSD: suricata.rc,v 1.2 2018/02/15 16:19:41 ajacoutot Exp $

daemon="/usr/local/bin/suricata -D -c /etc/suricata/suricata.yaml -s /var/suricata/rules/suricata.rules -i XXXXXXX(the network card u want suricata to run on)"

. /etc/rc.d/rc.subr

rc_cmd $1

This file is responsible for suricata daemon
If you dont set this then the daemon wont work properly

[psypro]

With you rc.d guide I get suricata running. From command line.
But recomondation from pkg_readme does not work.
Running 6.9 beta.

/etc/rc.d/suricata

#!/bin/ksh
#
# $OpenBSD: suricata.rc,v 1.2 2018/02/15 16:19:41 ajacoutot Exp $

daemon="/usr/local/bin/suricata -D -c /etc/suricata/suricata.yaml -s /var/lib/suricata/rules/suricata.rules -i re0"

. /etc/rc.d/rc.subr

rc_cmd $1

rcctl enable suricata                                                                       
rcctl: service suricata does not exist

[TronDD]

Did you make the rc file executable?

Actually, why are you modifying the rc.d/suricata file at all? Write all of your flags into the suricata_flags in rc.conf.local

[psypro]

Thank you TronDD. I was trying following advice : "Wrong file there Psypro
I ment this:
/etc/rc.d/suricata"


Was about to ask question about that. Reading webpage about it https://www.oreilly.com/library/view...d_scripts.html

I will revert back to orginal rc.d/suricata, and put changes into rc.conf.local

/etc/rc.conf.llocal

dhcpd_flags=
unbound_flags=
pkg_scripts="suricata"
suricata_flags="-c /etc/suricata/suricata.yaml -s /var/lib/suricata/rules/suricata.rules -i re0"

/etc/rc.d/suricata

#!/bin/ksh
#
# $OpenBSD: suricata.rc,v 1.3 2021/01/31 18:03:49 sthen Exp $

daemon="/usr/local/bin/suricata -D"

. /etc/rc.d/rc.subr

rc_pre() {
        /usr/bin/install -d -o _suricata -g _suricata -m 0750 /var/run/suricata
}

rc_cmd $1

[psypro]

I can now start suricata with rcctl. But it "dies".I guess it happens when it goes from "root" user process to "suricata" user process. I see in "top" command.

Firestorm# rcctl start suricata               
suricata(ok)
Firestorm# top
Firestorm# nano /etc/rc.conf.local            
Firestorm# top                      
Firestorm# rcctl check suricata               
suricata(failed)

Is this user for _suricata correct?

_suricata:*:800:800:Suricata Account:/nonexistent:/sbin/nologin

[psypro]

I get the upload limit by adding my desktop ip to the slow and slow_up file at the NAT/firewall.
If I set default to miniway (slow download) I get slow download at my desktop.
Why does it not take affect, when added as ip

To avoid cluttering this forum with my work (slow) in progress, I have started a github page, where I upload files.

https://github.com/Puffy-path/Openbsd-NAT-firewall

#Queue download
queue std on em1 bandwidth 100M
queue miniway parent std bandwidth 1M max 1M 
queue maxway parent std bandwidth 99M max 99M default

pass inet proto tcp from <slow_down> to port $ports queue miniway 

#Queue upload
queue std_up on em0 bandwidth 100M
queue miniup parent std_up bandwidth 1M max 1M 
queue maxup parent std_up bandwidth 99M max 99M default

pass inet proto tcp from <slow> to port $ports queue miniup