DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 13th February 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default URL logging

I'm looking to set up some kind of mechanism to log all of the URLs that go over my home web connection (and probably do some blocking as well). I had initially considered using a squid proxy, but I recently discovered that relayd can do this! Is this the best route to take, or is there something better to be using?
Reply With Quote
  #2   (View Single Post)  
Old 13th February 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

It comes as part of OpenBSD, so there is nothing to install. But note that relayd(8) uses syslog(3) for logging. If you want to record those logs, you will likely want to provision syslog.conf(5) to isolate the messages. See http://openbsd-archive.7691.n7.nabbl...e-td76656.html for a provisioning discussion.
Reply With Quote
  #3   (View Single Post)  
Old 13th February 2019
e1-531g e1-531g is offline
ISO Quartermaster
 
Join Date: Mar 2014
Posts: 628
Default

Do you need all URL or domain is enough?
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #4   (View Single Post)  
Old 13th February 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Quote:
Originally Posted by jggimi View Post
It comes as part of OpenBSD, so there is nothing to install. But note that relayd(8) uses syslog(3) for logging. If you want to record those logs, you will likely want to provision syslog.conf(5) to isolate the messages. See http://openbsd-archive.7691.n7.nabbl...e-td76656.html for a provisioning discussion.
Nice, I'll take a look at changing the logging destination for relayd. (This will also be nice to help set a separate place to stash the noise from dhcpd. )

Quote:
Originally Posted by e1-531g View Post
Do you need all URL or domain is enough?
I'll probably end up settling for just the domains, but I want to be sure to capture those for all http and https traffic.
Reply With Quote
  #5   (View Single Post)  
Old 13th February 2019
e1-531g e1-531g is offline
ISO Quartermaster
 
Join Date: Mar 2014
Posts: 628
Default

One problem with DNS is that browsers have DNS prefetch. But some browsers also have HTTP(S) prefetch as well.
When you log DNS queries you also end up logging all DNS queries. Not only HTTP and HTTPS.
You can also log IP addresses of TCP connection destination.
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #6   (View Single Post)  
Old 14th February 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Now that I think about it, monitoring full URLs might be more what I'm after. I did manage to get relayd working on both http and https, although the browsers are now complaining about cert mismatch, and I'm not sure how to work around that.

My ultimate goal is to filter out all the baddies (ads, tracking, etc) at the gateway level. To that end I've set up a big blacklist on unbound(8), and force all traffic on 53 to my own server via pf. So now I'm interested in setting up some monitoring to see what else might be getting by. I figured a transparent proxy would be a good way to go about that. Or, is there a better option I've not thought of?
Reply With Quote
  #7   (View Single Post)  
Old 14th February 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
...cert mismatch...
TronDD posted in a misc@ mailing list discussion about relayd certificates (source):
Quote:
'ca key' and 'ca cert' is for MITM roll your own certs on the fly.

For server certs, like a web server would have, you don't specify them.
relayd looks for address:port.key and address:port.crt as per the 'listen
on' description in relayd.conf(5)
Reply With Quote
  #8   (View Single Post)  
Old 14th February 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Yep, that's how I've set it up.

certs:
Code:
/etc/ssl/ca.crt
/etc/ssl/127.0.0.1:8443.crt
/etc/ssl/private/ca.key
/etc/ssl/private/127.0.0.1:8443.key
/etc/relayd.conf:
Code:
http protocol httpfilter {
    return error
    match request label "URL filtered!"
    block request quick url "example.com/" value "*"
}

http protocol tlsfilter {
    return error
    match request label "URL filtered!"
    block request quick url "example.com/" value "*"
    tls ca key "/etc/ssl/private/ca.key" password "password"
    tls ca cert "/etc/ssl/ca.crt"
}

relay httpproxy {
    listen on 127.0.0.1 port 8080
    protocol httpfilter
    forward to destination
}

relay tlsproxy {
    listen on 127.0.0.1 port 8443 tls
    protocol tlsfilter
    forward with tls to destination
}
relevant portion of /etc/pf.conf:
Code:
pass in quick log on $int_ifs inet proto { tcp udp } from $wired_if:network to port 53 rdr-to $wired_if:0
pass in log on $int_ifs inet proto tcp from $wired_if:network to port www   divert-to localhost port 8080
pass in log on $int_ifs inet proto tcp from $wired_if:network to port https divert-to localhost port 8443
With this setup, Chromium complains that the cert for https://duckduckgo.com/ doesn't match (since the name on the cert is 127.0.0.1).

Last edited by beavers; 14th February 2019 at 03:35 PM.
Reply With Quote
  #9   (View Single Post)  
Old 20th February 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Is there some trick I'm missing to get things rewritten so that the cert appears to a browser to be valid?
Reply With Quote
Old 20th February 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

My guess? (Yes, its only a guess.) The cert you provide to the browser need to be authenticated with a CA the browser accepts.
Reply With Quote
Old 4th March 2019
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Yeah, that's definitely what's happening. How would I get a cert that's CA-approved for a host on a private IP? Hoping to avoid having to import the cert manually on each and every browser that crosses my network.
Reply With Quote
Old 4th March 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

One way might be via letsencrypt.org, a free certificate service operated by the ISRG. It requires you to have an on-the-Internet web server to obtain the certificate and key, and is designed to renew automatically. See the acme-client(1) man page.

Once you have the cert and key acquired at your public web server, you may be able to copy the pair to your private server, replacing it every with every 60-90 day renewal. However I expect you would also require a split-horizon DNS, so that the private server resolves correctly for those browsers using it.
Reply With Quote
Old 6th March 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

While I'm not (currently) a relayd user, I happened to test yesterday's scenario -- a private server using a public server's key pair -- because I am considering putting relayd in front of a wordpress server to block access to /wp-admin and /wp-login paths. The browser's connection to the private server works fine, as long as the FQDN resolves to the private IP address.

Testing with relayd still pends. My plan is to use use the web server's key pair for TLS inspection.

At the moment, those two paths are protected with httpd.conf(5) authenticate, and then with some wordpress security plugins. Not good enough for me.

Last edited by jggimi; 6th March 2019 at 02:04 PM. Reason: clarity and a typo
Reply With Quote
Old 7th March 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
Originally Posted by beavers View Post
Hoping to avoid having to import the cert manually on each and every browser that crosses my network.
Well, in reading up on relayd this evening in preparation for initial testing, I have learned that this is what most organizations that use TLS inspection tools actually do. They distribute their private CA certificates to all clients.
Reply With Quote
Old 8th March 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

I've tested TLS acceleration. Not much to accelerate, as http(8) was listening on a loopback address on the same test system. TLS inspection works fine. However password prompts from the back end web server pop up a cleartext warning, as the back end server was using cleartext. I may be able to remove this with header modification, which would mean more testing.

I'm able to move my key pair from system to system, as long as the IP address resolves at the calling client system (such as with an /etc/hosts entry or a non-authoritative DNS server). I can uses my key pair with relayd. My CA will confirm OCSP queries issued by the browser, because the certificate is valid.
Reply With Quote
Old 9th March 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
Originally Posted by jggimi View Post
I may be able to remove this with header modification...
No, it seems all I needed was to configure the back end server to use TLS also.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Gnome keeps logging me out - screen res? jwood OpenBSD General 4 20th April 2018 02:40 AM
Doas has logging? cpaulette OpenBSD General 1 13th March 2016 10:24 AM
dnsspoof logging issue joostvgh OpenBSD Security 5 19th January 2010 12:04 AM
pflog not logging. bsdnewbie999 OpenBSD General 9 13th March 2009 11:19 PM
spamd logging question roundkat OpenBSD General 10 11th June 2008 01:27 PM


All times are GMT. The time now is 12:30 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick