DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 3 Days Ago
beavers beavers is offline
Fdisk Soldier
 
Join Date: Nov 2017
Posts: 60
Default URL logging

I'm looking to set up some kind of mechanism to log all of the URLs that go over my home web connection (and probably do some blocking as well). I had initially considered using a squid proxy, but I recently discovered that relayd can do this! Is this the best route to take, or is there something better to be using?
Reply With Quote
  #2   (View Single Post)  
Old 3 Days Ago
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,446
Default

It comes as part of OpenBSD, so there is nothing to install. But note that relayd(8) uses syslog(3) for logging. If you want to record those logs, you will likely want to provision syslog.conf(5) to isolate the messages. See http://openbsd-archive.7691.n7.nabbl...e-td76656.html for a provisioning discussion.
Reply With Quote
  #3   (View Single Post)  
Old 3 Days Ago
e1-531g e1-531g is offline
VPN Cryptographer
 
Join Date: Mar 2014
Posts: 456
Default

Do you need all URL or domain is enough?
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #4   (View Single Post)  
Old 3 Days Ago
beavers beavers is offline
Fdisk Soldier
 
Join Date: Nov 2017
Posts: 60
Default

Quote:
Originally Posted by jggimi View Post
It comes as part of OpenBSD, so there is nothing to install. But note that relayd(8) uses syslog(3) for logging. If you want to record those logs, you will likely want to provision syslog.conf(5) to isolate the messages. See http://openbsd-archive.7691.n7.nabbl...e-td76656.html for a provisioning discussion.
Nice, I'll take a look at changing the logging destination for relayd. (This will also be nice to help set a separate place to stash the noise from dhcpd. )

Quote:
Originally Posted by e1-531g View Post
Do you need all URL or domain is enough?
I'll probably end up settling for just the domains, but I want to be sure to capture those for all http and https traffic.
Reply With Quote
  #5   (View Single Post)  
Old 3 Days Ago
e1-531g e1-531g is offline
VPN Cryptographer
 
Join Date: Mar 2014
Posts: 456
Default

One problem with DNS is that browsers have DNS prefetch. But some browsers also have HTTP(S) prefetch as well.
When you log DNS queries you also end up logging all DNS queries. Not only HTTP and HTTPS.
You can also log IP addresses of TCP connection destination.
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #6   (View Single Post)  
Old 2 Days Ago
beavers beavers is offline
Fdisk Soldier
 
Join Date: Nov 2017
Posts: 60
Default

Now that I think about it, monitoring full URLs might be more what I'm after. I did manage to get relayd working on both http and https, although the browsers are now complaining about cert mismatch, and I'm not sure how to work around that.

My ultimate goal is to filter out all the baddies (ads, tracking, etc) at the gateway level. To that end I've set up a big blacklist on unbound(8), and force all traffic on 53 to my own server via pf. So now I'm interested in setting up some monitoring to see what else might be getting by. I figured a transparent proxy would be a good way to go about that. Or, is there a better option I've not thought of?
Reply With Quote
  #7   (View Single Post)  
Old 2 Days Ago
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,446
Default

Quote:
...cert mismatch...
TronDD posted in a misc@ mailing list discussion about relayd certificates (source):
Quote:
'ca key' and 'ca cert' is for MITM roll your own certs on the fly.

For server certs, like a web server would have, you don't specify them.
relayd looks for address:port.key and address:port.crt as per the 'listen
on' description in relayd.conf(5)
Reply With Quote
  #8   (View Single Post)  
Old 2 Days Ago
beavers beavers is offline
Fdisk Soldier
 
Join Date: Nov 2017
Posts: 60
Default

Yep, that's how I've set it up.

certs:
Code:
/etc/ssl/ca.crt
/etc/ssl/127.0.0.1:8443.crt
/etc/ssl/private/ca.key
/etc/ssl/private/127.0.0.1:8443.key
/etc/relayd.conf:
Code:
http protocol httpfilter {
    return error
    match request label "URL filtered!"
    block request quick url "example.com/" value "*"
}

http protocol tlsfilter {
    return error
    match request label "URL filtered!"
    block request quick url "example.com/" value "*"
    tls ca key "/etc/ssl/private/ca.key" password "password"
    tls ca cert "/etc/ssl/ca.crt"
}

relay httpproxy {
    listen on 127.0.0.1 port 8080
    protocol httpfilter
    forward to destination
}

relay tlsproxy {
    listen on 127.0.0.1 port 8443 tls
    protocol tlsfilter
    forward with tls to destination
}
relevant portion of /etc/pf.conf:
Code:
pass in quick log on $int_ifs inet proto { tcp udp } from $wired_if:network to port 53 rdr-to $wired_if:0
pass in log on $int_ifs inet proto tcp from $wired_if:network to port www   divert-to localhost port 8080
pass in log on $int_ifs inet proto tcp from $wired_if:network to port https divert-to localhost port 8443
With this setup, Chromium complains that the cert for https://duckduckgo.com/ doesn't match (since the name on the cert is 127.0.0.1).

Last edited by beavers; 2 Days Ago at 03:35 PM.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Gnome keeps logging me out - screen res? jwood OpenBSD General 4 20th April 2018 02:40 AM
Doas has logging? cpaulette OpenBSD General 1 13th March 2016 10:24 AM
dnsspoof logging issue joostvgh OpenBSD Security 5 19th January 2010 12:04 AM
pflog not logging. bsdnewbie999 OpenBSD General 9 13th March 2009 11:19 PM
spamd logging question roundkat OpenBSD General 10 11th June 2008 01:27 PM


All times are GMT. The time now is 05:20 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick