DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Security

OpenBSD Security Functionally paranoid!

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 13th May 2019
e1-531g e1-531g is offline
ISO Quartermaster
 
Join Date: Mar 2014
Posts: 508
Default How to prevent crawling my contact info from my website

I would like to create my personal website with contact info (i.e. e-mail, maybe phone number). Preferably using OpenBSD. I would like to protect it from being harvested by crawling bots. What to do besides putting content="noindex" into html code and creating robots.txt? I think that these things would not prevent bots created by spamers that use these data illegally. I was thinking about some sort of captcha i.e. Google's reCAPTCHA. Can I integrate reCAPTCHA with site running built-in base OpenBSD system or should I use some packages/ports?
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #2   (View Single Post)  
Old 13th May 2019
PapaParrot's Avatar
PapaParrot PapaParrot is offline
parrot
 
Join Date: Jul 2015
Location: Durango, Mx.
Posts: 415
Default

Actually, the OS that runs the server has very little to do with it, What web server will you be using,? IE: Nginx, Apache2, or Httpd , see
Code:
man httpd
(comes with OpenBsd, by default, just needs to be configure)
-----
Quote:
Can I integrate reCAPTCHA with site running built-in base OpenBSD system or should I use some packages/ports?
As for googles "recaptcha", I don't like or use it my self, but any way , it is not integrated with the OS at all, it works by connecting to google,:
https://developers.google.com/recaptcha/intro

========
Another key factor will be what software, (if any ) is used, IE: site builders, forum software, etc....
One of the simplest methods is using some forum software, and setting up a board that only registered users/members can view it. If you have your personal information viewable by the public, then any one can view it , like wise anyone can copy it in some way. So you really need to think twice, before you go putting your business or personal e-mail , etc on a public website.
The key being, limit what the public can see, and only allow them to see the private info if they login, and make it where they need to register, and the registration be accepted before they can view the private content.
I recently disabled a "contact form" on a phpBB forum I have, because it was getting to much spam, but that is another topic.
Quote:
creating robots.txt? I think that these things would not prevent bots created by spamers
You are right on that, the "robots.txt" does not do anything to stop spam bots.
For keeping out most bots, including spam bots, scrapers, etc. I would suggest registering here:
https://zb-block.net/zbf/index.php
And learning how to use it, again it does not matter what OS you are using, the script is integrated with your website html, or php, and effectively blocks most known bad bots from even viewing your site.
In relation to spam bots, https://www.stopforumspam.com/forum/ is a good place to start.
We actually do use google Recaptcha, there as well.
We have data bases, and you can get lists of most known spam bots, or use the API, and check the credentials they use when registering, this keeps most from being able to register, thus being able to view topics, personal info, etc,...
Also SFS is a good place to ask about keeping spam bots off of your website, and again, the OS being used is not really relevant. Guess for now that is about it.
On OpenBsd there is also "pf",
Code:
man pf
More info :
http://0x9900.com/spam-filtering-wit...little-harder/
=============
You could use some of the info from the SFS data base, to decide what IP's you want to add hostnames or IP using pf and
'pfctl'
https://man.openbsd.org/pfctl.8
__________________
My best friends are parrots
Reply With Quote
  #3   (View Single Post)  
Old 13th May 2019
e1-531g e1-531g is offline
ISO Quartermaster
 
Join Date: Mar 2014
Posts: 508
Default

I would like to use httpd. I know reCAPTCHA is not integrated into base. I would like to use as much built-in OpenBSD base software components and then add as little 3rd party software as possible. I am fine with adding a few 3rd party components from ports or GitHub, though.
If setting up robust anti-crawling system is difficult on httpd, I am going to use some other http server.

I think about static website: semantically correct HTML5 (without Javascript) + CSS. The only "dynamic" thing on site I foresee is something to keep crawling bots away.

Quote:
If you have your personal information viewable by the public, then any one can view it , like wise anyone can copy it in some way. So you really need to think twice, before you go putting your business or personal e-mail , etc on a public website.
The key being, limit what the public can see, and only allow them to see the private info if they login, and make it where they need to register, and the registration be accepted before they can view the private content.
I would like website without registration. Just captcha on some pages. I am not going to post things as sensitive as social security number (or rather something similar because I don't live in USA). On the other hand I think about posting one of aliases to my personal e-mail.
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #4   (View Single Post)  
Old 13th May 2019
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 6,820
Default

I perceive there are two "built-in" options to make some or all of a website private. Both of these are included in httpd(8):

  • HTTP Authentication

    HTTP Userid/PW authentication can be set for any location{} or server{} in your httpd.conf(5) file. If you're using HTTP, this will be in plaintext, so obviously, HTTPS is recommended.

    While it can sustain brute force attacks with little resource consumption, I would not recommend using it as your only authentication method. Either use it as one factor in a 2-factor website authentication, or use PF stateful tracking to limit brute force attempts.
  • X.509 Client Certificates

    httpd(8) supports client certificates, and if enabled as required for a server{} only browsers establishing connections with the client certificate can establish a session with the server.
I'm using both methods currently, with different web applications.

Last edited by jggimi; 13th May 2019 at 11:07 PM. Reason: clarity
Reply With Quote
  #5   (View Single Post)  
Old 14th May 2019
e1-531g e1-531g is offline
ISO Quartermaster
 
Join Date: Mar 2014
Posts: 508
Default

But I don't want completely hide these details. I am okay if let's say some friend I didn't saw for years would send me an e-mail or stranger from human resources would copy my e-mail into Outlook and send me invitation to job interview. I just want to hide these contact details from bots. I understand there may be some spammer manually harvesting e-mail addresses, but I accept that risk. I am going to post not original, but alias to my e-mail account, so if something goes wrong I can just remove that alias from my e-mail account.
It doesn't even need to be reCAPTCH, just something I can use to differentiate between bots and people.
__________________
Signature: Furthermore, I consider that systemd must be destroyed.
Based on Latin oratorical phrase
Reply With Quote
  #6   (View Single Post)  
Old 14th May 2019
Beastie Beastie is offline
Daemonology student
 
Join Date: Jan 2009
Location: /dev/earth0
Posts: 324
Default

How about the age-old technique of "coding" the information in a manner that would be easy for a human to understand or stating how to "decode" it?

Code:
To contact us send an email to someone ~at~ something ~dot~ com
Code:
To contact us send an email to somezzzone@somzzzething.com (remove the zzz)
As far as I know crawlers/spambots are not that clever so as to understand natural languages.

You could also enclose parts of it with HTML tags
Code:
someone<code>@</code>something.com
But that may be problematic if the bots strip tags while parsing pages.

You can also use an image to show the information.
__________________
May the source be with you!
Reply With Quote
  #7   (View Single Post)  
Old 14th May 2019
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 882
Default

Along the line of encoding, I've heard of people using base64 to encode the email address. Don't know if bots are onto that or not.
Reply With Quote
  #8   (View Single Post)  
Old 25th May 2019
bsdcord bsdcord is offline
Port Guard
 
Join Date: Apr 2019
Posts: 23
Default

my advice is to use a photo of your email address.
For example:

I have used this site
Reply With Quote
Reply

Tags
bots, captcha, httpd

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to prevent hardrives from pinning down hanzer FreeBSD General 2 18th February 2019 08:10 PM
Prevent SSH tunneling through port 80 pttymuth General software and network 7 15th July 2013 05:35 PM
the website is down ai-danno Off-Topic 2 1st July 2008 11:35 PM


All times are GMT. The time now is 11:00 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick