DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Packages and Ports

OpenBSD Packages and Ports Installation and upgrading of packages and ports on OpenBSD.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 14th July 2011
plexter plexter is offline
Shell Scout
 
Join Date: May 2008
Posts: 124
Question Log Parsing Not Working

Hello,

I'm trying to run "webalizer" for OpenBSD.

But for some reason I cannot get it to read my log file for ProFTPd.

I just get:

Code:
Skipping Bad Record
My config file is:

Code:
#
# Sample Webalizer Xtended configuration file
# Copyright 2005-2007 by Patrick K. Frei (frei@caramail.com)
# Based on the sample Webalizer configuration file
# Copyright 1997-2000 by Bradford L. Barrett (brad@mrunix.net)
#
# Distributed under the GNU General Public License.  See the
# files "Copyright" and "COPYING" provided with the webalizer
# distribution for additional information.
#
# This is a sample configuration file for the Webalizer (ver 2.01)
# Lines starting with pound signs '#' are comment lines and are
# ignored.  Blank lines are skipped as well.  Other lines are considered
# as configuration lines, and have the form "ConfigOption  Value" where
# ConfigOption is a valid configuration keyword, and Value is the value
# to assign that configuration option.  Invalid keyword/values are
# ignored, with appropriate warnings being displayed.  There must be
# at least one space or tab between the keyword and its value.
#
# As of version 0.98, The Webalizer will look for a 'default' configuration
# file named "webalizer.conf" in the current directory, and if not found
# there, will look for "/etc/webalizer.conf".


# LogFile defines the web server log file to use.  If not specified
# here or on on the command line, input will default to STDIN.  If
# the log filename ends in '.gz' (ie: a gzip compressed file), it will
# be decompressed on the fly as it is being read.

#LogFile        /var/www/logs/access_log
LogFile         /var/log/xferlog

# LogType defines the log type being processed.  Normally, the Webalizer
# expects a CLF or Combined web server log as input.  Using this option,
# you can process ftp logs as well (xferlog as produced by wu-ftp and
# others), or Squid native logs.  Values can be 'clf', 'ftp' or 'squid',
# with 'clf' the default.

#LogType	clf
LogType         ftp

# OutputDir is where you want to put the output files.  This should
# should be a full path name, however relative ones might work as well.
# If no output directory is specified, the current directory will be used.

#OutputDir      /var/www/htdocs/usage

# HistoryName allows you to specify the name of the history file produced
# by the Webalizer.  The history file keeps the data for up to 12 months
# worth of logs, used for generating the main HTML page (index.html).
# The default is a file named "webalizer.hist", stored in the specified
# output directory.  If you specify just the filename (without a path),
# it will be kept in the specified output directory.  Otherwise, the path
# is relative to the output directory, unless absolute (leading /).

#HistoryName	webalizer.hist

# Incremental processing allows multiple partial log files to be used
# instead of one huge one.  Useful for large sites that have to rotate
# their log files more than once a month.  The Webalizer will save its
# internal state before exiting, and restore it the next time run, in
# order to continue processing where it left off.  This mode also causes
# The Webalizer to scan for and ignore duplicate records (records already
# processed by a previous run).  See the README file for additional
# information.  The value may be 'yes' or 'no', with a default of 'no'.
# The file 'webalizer.current' is used to store the current state data,
# and is located in the output directory of the program (unless changed
# with the IncrementalName option below).  Please read at least the section
# on Incremental processing in the README file before you enable this option.

#Incremental	no

# IncrementalName allows you to specify the filename for saving the
# incremental data in.  It is similar to the HistoryName option where the
# name is relative to the specified output directory, unless an absolute
# filename is specified.  The default is a file named "webalizer.current"
# kept in the normal output directory.  If you don't specify "Incremental"
# as 'yes' then this option has no meaning.

#IncrementalName	webalizer.current

# ReportTitle is the text to display as the title.  The hostname
# (unless blank) is appended to the end of this string (seperated with
# a space) to generate the final full title string.
# Default is (for english) "Usage Statistics for".

#ReportTitle    Usage Statistics for

# HostName defines the hostname for the report.  This is used in
# the title, and is prepended to the URL table items.  This allows
# clicking on URL's in the report to go to the proper location in
# the event you are running the report on a 'virtual' web server,
# or for a server different than the one the report resides on.
# If not specified here, or on the command line, webalizer will
# try to get the hostname via a uname system call.  If that fails,
# it will default to "localhost".

#HostName       localhost

# HTMLExtension allows you to specify the filename extension to use
# for generated HTML pages.  Normally, this defaults to "html", but
# can be changed for sites who need it (like for PHP embeded pages).

#HTMLExtension  html

# PageType lets you tell the Webalizer what types of URL's you
# consider a 'page'.  Most people consider html and cgi documents
# as pages, while not images and audio files.  If no types are
# specified, defaults will be used ('htm*', 'cgi' and HTMLExtension
# if different for web logs, 'txt' for ftp logs).

PageType	htm*
PageType	cgi
#PageType	phtml
#PageType	php3
#PageType	pl

# UseHTTPS should be used if the analysis is being run on a
# secure server, and links to urls should use 'https://' instead
# of the default 'http://'.  If you need this, set it to 'yes'.
# Default is 'no'.  This only changes the behaviour of the 'Top
# URL's' table.

#UseHTTPS       no
UseHTTPS        yes

# DNSCache specifies the DNS cache filename to use for reverse DNS lookups.
# This file must be specified if you wish to perform name lookups on any IP
# addresses found in the log file.  If an absolute path is not given as
# part of the filename (ie: starts with a leading '/'), then the name is
# relative to the default output directory.  See the DNS.README file for
# additional information.

#DNSCache	dns_cache.db

# DNSChildren allows you to specify how many "children" processes are
# run to perform DNS lookups to create or update the DNS cache file.
# If a number is specified, the DNS cache file will be created/updated
# each time the Webalizer is run, immediately prior to normal processing,
# by running the specified number of "children" processes to perform
# DNS lookups.  If used, the DNS cache filename MUST be specified as
# well.  The default value is zero (0), which disables DNS cache file
# creation/updates at run time.  The number of children processes to
# run may be anywhere from 1 to 100, however a large number may effect
# normal system operations.  Reasonable values should be between 5 and
# 20.  See the DNS.README file for additional information.

#DNSChildren	0

# HTMLPre defines HTML code to insert at the very beginning of the
# file.  Default is the DOCTYPE line shown below.  Max line length
# is 80 characters, so use multiple HTMLPre lines if you need more.

#HTMLPre <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

# HTMLHead defines HTML code to insert within the <HEAD></HEAD>
# block, immediately after the <TITLE> line.  Maximum line length
# is 80 characters, so use multiple lines if needed.

#HTMLHead <META NAME="author" CONTENT="The Webalizer">

# HTMLBody defined the HTML code to be inserted, starting with the
# <BODY> tag.  If not specified, the default is shown below.  If
# used, you MUST include your own <BODY> tag as the first line.
# Maximum line length is 80 char, use multiple lines if needed.

#HTMLBody <BODY BGCOLOR="#E8E8E8" TEXT="#000000" LINK="#0000FF" VLINK="#FF0000">

# HTMLPost defines the HTML code to insert immediately before the
# first <HR> on the document, which is just after the title and
# "summary period"-"Generated on:" lines.  If anything, this should
# be used to clean up in case an image was inserted with HTMLBody.
# As with HTMLHead, you can define as many of these as you want and
# they will be inserted in the output stream in order of apperance.
# Max string size is 80 characters.  Use multiple lines if you need to.

#HTMLPost 	<BR CLEAR="all">

# HTMLTail defines the HTML code to insert at the bottom of each
# HTML document, usually to include a link back to your home
# page or insert a small graphic.  It is inserted as a table
# data element (ie: <TD> your code here </TD>) and is right
# alligned with the page.  Max string size is 80 characters.

#HTMLTail <IMG SRC="msfree.png" ALT="100% Micro$oft free!">

# HTMLEnd defines the HTML code to add at the very end of the
# generated files.  It defaults to what is shown below.  If
# used, you MUST specify the </BODY> and </HTML> closing tags
# as the last lines.  Max string length is 80 characters.

#HTMLEnd </BODY></HTML>

# The Quiet option suppresses output messages... Useful when run
# as a cron job to prevent bogus e-mails.  Values can be either
# "yes" or "no".  Default is "no".  Note: this does not suppress
# warnings and errors (which are printed to stderr).

#Quiet		no
Quiet           yes

# ReallyQuiet will supress all messages including errors and
# warnings.  Values can be 'yes' or 'no' with 'no' being the
# default.  If 'yes' is used here, it cannot be overriden from
# the command line, so use with caution.  A value of 'no' has
# no effect.

#ReallyQuiet	no

# TimeMe allows you to force the display of timing information
# at the end of processing.  A value of 'yes' will force the
# timing information to be displayed.  A value of 'no' has no
# effect.

#TimeMe		no
TimeMe          yes

# GMTTime allows reports to show GMT (UTC) time instead of local
# time.  Default is to display the time the report was generated
# in the timezone of the local machine, such as EDT or PST.  This
# keyword allows you to have times displayed in UTC instead.  Use
# only if you really have a good reason, since it will probably
# screw up the reporting periods by however many hours your local
# time zone is off of GMT.

#GMTTime		no

# Debug prints additional information for error messages.  This
# will cause webalizer to dump bad records/fields instead of just
# telling you it found a bad one.   As usual, the value can be
# either "yes" or "no".  The default is "no".  It shouldn't be
# needed unless you start getting a lot of Warning or Error
# messages and want to see why.  (Note: warning and error messages
# are printed to stderr, not stdout like normal messages).

#Debug		no

# FoldSeqErr forces the Webalizer to ignore sequence errors.
# This is useful for Netscape and other web servers that cache
# the writing of log records and do not guarentee that they
# will be in chronological order.  The use of the FoldSeqErr
# option will cause out of sequence log records to be treated
# as if they had the same time stamp as the last valid record.
# Default is to ignore out of sequence log records.

#FoldSeqErr	no

# VisitTimeout allows you to set the default timeout for a visit
# (sometimes called a 'session').  The default is 30 minutes,
# which should be fine for most sites.
# Visits are determined by looking at the time of the current
# request, and the time of the last request from the site.  If
# the time difference is greater than the VisitTimeout value, it
# is considered a new visit, and visit totals are incremented.
# Value is the number of seconds to timeout (default=1800=30min)

#VisitTimeout	1800

# IgnoreHist shouldn't be used in a config file, but it is here
# just because it might be usefull in certain situations.  If the
# history file is ignored, the main "index.html" file will only
# report on the current log files contents.  Usefull only when you
# want to reproduce the reports from scratch.  USE WITH CAUTION!
# Valid values are "yes" or "no".  Default is "no".

#IgnoreHist	no

# ResponseGraph allows the response code graph to be disabled.
# Values can be 'yes' or 'no', default is 'yes'.

#ResponseGraph	yes

# Country Graph allows the usage by country graph to be disabled.
# Values can be 'yes' or 'no', default is 'yes'.

#CountryGraph	yes

# DailyGraph and DailyStats allows the daily statistics graph
# and statistics table to be disabled (not displayed).  Values
# may be "yes" or "no". Default is "yes".

#DailyGraph	yes
#DailyStats	yes

# HourlyGraph and HourlyStats allows the hourly statistics graph
# and statistics table to be disabled (not displayed).  Values
# may be "yes" or "no". Default is "yes".

#HourlyGraph	yes
#HourlyStats	yes

# GraphLegend allows the color coded legends to be turned on or off
# in the graphs.  The default is for them to be displayed.  This only
# toggles the color coded legends, the other legends are not changed.
# If you think they are hideous and ugly, say 'no' here :)

#GraphLegend	yes

# GraphLines allows you to have index lines drawn behind the graphs.
# I personally am not crazy about them, but a lot of people requested
# them and they weren't a big deal to add.  The number represents the
# number of lines you want displayed. Default is 4, you can disable
# the lines by using a value of zero ('0') [max is 20].
# Note, due to rounding errors, some values don't work quite right.
# The lower the better, with 1,2,3,4,6 and 10 producing nice results.

#GraphLines     4

# The "Top" options below define the number of entries for each table.
# Defaults are Sites=30, URL's=30, Referrers=30 and Agents=15, and
# Countries=30. TopKSites and TopKURLs (by KByte tables) both default
# to 10, as do the top entry/exit tables (TopEntry/TopExit).  The top
# search strings and usernames default to 20.  Tables may be disabled
# by using zero (0) for the value.

#TopSites        30
#TopKSites       10
#TopURLs         30
#TopKURLs        10
#TopReferrers    30
#TopAgents       15
#TopCountries    30
#TopEntry        10
#TopExit         10
#TopSearch       20
#TopUsers        20

# The All* keywords allow the display of all URL's, Sites, Referrers, User
# Agents, Search Strings, Usernames and 404-errors.  If enabled, a seperate
# HTML page will be created, and a link will be added to the bottom
# of the appropriate "Top" table.  There are a couple of conditions
# for this to occur. First, there must be more items than will fit
# in the "Top" table (otherwise it would just be duplicating what is
# already displayed). Second, the listing will only show those items
# that are normally visable, which means it will not show any hidden
# items.  Grouped entries will be listed first, followed by individual
# items.  The value for these keywords can be either 'yes' or 'no',
# with the default being 'no' (exc. 404-errors). Please be aware that these
# pages can be quite large in size, particularly the sites page, and seperate
# pages are generated for each month, which can consume quite a lot
# of disk space depending on the traffic to your site.

#AllSites	no
AllSites        yes
#AllURLs	no
AllURLs         yes
#AllReferrers	no
AllReferrers    yes
#AllAgents	no
AllAgents       yes
#AllSearchStr	no
AllSearchStr    yes
#AllUsers       no
AllUsers        yes
#AllErrors      yes

# The Webalizer normally strips the string 'index.' off the end of
# URL's in order to consolidate URL totals.  For example, the URL
# /somedir/index.html is turned into /somedir/ which is really the
# same URL.  This option allows you to specify additional strings
# to treat in the same way.  You don't need to specify 'index.' as
# it is always scanned for by The Webalizer, this option is just to
# specify _additional_ strings if needed.  If you don't need any,
# don't specify any as each string will be scanned for in EVERY
# log record... A bunch of them will degrade performance.  Also,
# the string is scanned for anywhere in the URL, so a string of
# 'home' would turn the URL /somedir/homepages/brad/home.html into
# just /somedir/ which is probably not what was intended.

#IndexAlias     home.htm
#IndexAlias	homepage.htm

# The Hide*, Group* and Ignore* and Include* keywords allow you to
# change the way Sites, URL's, Referrers, User Agents and Usernames
# are manipulated.  The Ignore* keywords will cause The Webalizer to
# completely ignore records as if they didn't exist (and thus not
# counted in the main site totals).  The Hide* keywords will prevent
# things from being displayed in the 'Top' tables, but will still be
# counted in the main totals.  The Group* keywords allow grouping
# similar objects as if they were one.  Grouped records are displayed
# in the 'Top' tables and can optionally be displayed in BOLD and/or
# shaded. Groups cannot be hidden, and are not counted in the main
# totals. The Group* options do not, by default, hide all the items
# that it matches.  If you want to hide the records that match (so just
# the grouping record is displayed), follow with an identical Hide*
# keyword with the same value.  (see example below)  In addition,
# Group* keywords may have an optional label which will be displayed
# instead of the keywords value.  The label should be seperated from
# the value by at least one 'white-space' character, such as a space
# or tab.
#
# The value can have either a leading or trailing '*' wildcard
# character.  If no wildcard is found, a match can occur anywhere
# in the string. Given a string "www.yourmama.com", the values "your",
# "*mama.com" and "www.your*" will all match.

# Your own site should be hidden
HideSite	mysite.com
#HideSite	localhost

# Your own site gives most referrals
HideReferrer	mysite.com/

# This one hides non-referrers ("-" Direct requests)
#HideReferrer	Direct Request

# Usually you want to hide these
HideURL		*.gif
HideURL		*.GIF
HideURL		*.jpg
HideURL		*.JPG
HideURL		*.png
HideURL		*.PNG
HideURL		*.ra

# Hiding agents is kind of futile
#HideAgent	RealPlayer

# You can also hide based on authenticated username
#HideUser	root
#HideUser	admin

# Grouping options
#GroupURL	/cgi-bin/*	CGI Scripts
#GroupURL	/images/*	Images

#GroupSite	*.aol.com
#GroupSite	*.compuserve.com

#GroupReferrer	yahoo.com/	Yahoo!
#GroupReferrer	excite.com/     Excite
#GroupReferrer	infoseek.com/   InfoSeek
#GroupReferrer	webcrawler.com/ WebCrawler

#GroupUser      root            Admin users
#GroupUser      admin           Admin users
#GroupUser      wheel           Admin users

# The following is a great way to get an overall total
# for browsers, and not display all the detail records.
# (You should use MangleAgent to refine further...)

GroupAgent	Googlebot	Googlebot
HideAgent	Googlebot
GroupAgent	Gigabot		Gigabot
HideAgent	Gigabot
GroupAgent	slurp@inktomi.com	Inktomi Slurp
HideAgent	slurp@inktomi.com
GroupAgent	msnbot		msnbot
HideAgent	msnbot
GroupAgent	Wget		GNU wget
HideAgent	Wget
GroupAgent	Yahoo!		Yahoo! Slurp
HideAgent	Yahoo!
GroupAgent	AppleWebKit/	Safari (OSX)
HideAgent	AppleWebKit/
GroupAgent	Camino		Camino (OSX)
HideAgent	Camino
GroupAgent	Epiphany	Epiphany (Gentoo)
HideAgent	Epiphany
GroupAgent	Firebird/	Firebird
HideAgent	Firebird/
GroupAgent	Firefox/	Firefox
HideAgent	Firefox/
GroupAgent	Galeon/		Galeon
HideAgent	Galeon/
GroupAgent	Iceweasel/	Iceweasel
HideAgent	Iceweasel/
GroupAgent	Konqueror/	Konqueror
HideAgent	Konqueror/
GroupAgent	Lynx*		Lynx
HideAgent	Lynx*
GroupAgent	Opera		Opera
HideAgent	Opera
GroupAgent	Netscape6/	Netscape 6
HideAgent	Netscape6/
GroupAgent	Netscape/7	Netscape 7
HideAgent	Netscape/7
GroupAgent	Netscape/8	Netscape 8
HideAgent	Netscape/8
GroupAgent	Mozilla/3       Netscape 3
HideAgent	Mozilla/3
GroupAgent	Mozilla/4.03	Netscape 4.03
HideAgent	Mozilla/4.03
GroupAgent	Mozilla/4.04	Netscape 4.04
HideAgent	Mozilla/4.04
GroupAgent	Mozilla/4.05	Netscape 4.05
HideAgent	Mozilla/4.05
GroupAgent	Mozilla/4.06	Netscape 4.06
HideAgent	Mozilla/4.06
GroupAgent	Mozilla/4.08	Netscape 4.08
HideAgent	Mozilla/4.08
GroupAgent	Mozilla/4.5	Netscape 4.5
HideAgent	Mozilla/4.5
GroupAgent	Mozilla/4.61	Netscape 4.6
HideAgent	Mozilla/4.61
GroupAgent	Mozilla/4.6	Netscape 4.6
HideAgent	Mozilla/4.6
GroupAgent	Mozilla/4.72	Netscape 4.72
HideAgent	Mozilla/4.72
GroupAgent	Mozilla/4.73	Netscape 4.73
HideAgent	Mozilla/4.73
GroupAgent	Mozilla/4.75	Netscape 4.75
HideAgent	Mozilla/4.75
GroupAgent	Mozilla/4.76	Netscape 4.76
HideAgent	Mozilla/4.76
GroupAgent	Mozilla/4.77	Netscape 4.77
HideAgent	Mozilla/4.77
GroupAgent	Mozilla/4.78	Netscape 4.78
HideAgent	Mozilla/4.78
GroupAgent	Mozilla/4.79	Netscape 4.79
HideAgent	Mozilla/4.79
GroupAgent	Mozilla/4.7	Netscape 4.7
HideAgent	Mozilla/4.7
GroupAgent	Mozilla/4.8	Netscape 4.8
HideAgent	Mozilla/4.8
GroupAgent	3.0		Internet Explorer 3.0
HideAgent	3.0
GroupAgent	3.01		Internet Explorer 3.01
HideAgent	3.01
GroupAgent	4.01		Internet Explorer 4.01
HideAgent	4.01
GroupAgent	4.5		Internet Explorer 4.5
HideAgent	4.5
GroupAgent	5.0		Internet Explorer 5.0
HideAgent	5.0
GroupAgent	5.01		Internet Explorer 5.01
HideAgent	5.01
GroupAgent	5.12		Internet Explorer 5.12 (Mac)
HideAgent	5.12
GroupAgent	5.13		Internet Explorer 5.13 (Mac)
HideAgent	5.13
GroupAgent	5.14		Internet Explorer 5.14 (Mac)
HideAgent	5.14
GroupAgent	5.15		Internet Explorer 5.15 (Mac)
HideAgent	5.15
GroupAgent	5.16		Internet Explorer 5.16 (Mac)
HideAgent	5.16
GroupAgent	5.17		Internet Explorer 5.17 (Mac)
HideAgent	5.17
GroupAgent	5.21		Internet Explorer 5.21 (Mac)
HideAgent	5.21
GroupAgent	5.22		Internet Explorer 5.22 (Mac)
HideAgent	5.22
GroupAgent	5.23		Internet Explorer 5.23 (Mac)
HideAgent	5.23
GroupAgent	5.5		Internet Explorer 5.5
HideAgent	5.5
GroupAgent	6.0		Internet Explorer 6.0
HideAgent	6.0
GroupAgent	7.0		Internet Explorer 7.0
HideAgent	7.0

# HideAllSites allows forcing individual sites to be hidden in the
# report.  This is particularly useful when used in conjunction
# with the "GroupDomain" feature, but could be useful in other
# situations as well, such as when you only want to display grouped
# sites (with the GroupSite keywords...).  The value for this
# keyword can be either 'yes' or 'no', with 'no' the default,
# allowing individual sites to be displayed.

#HideAllSites	no

# HideInOutVolume allows forcing the input/output volume to be hidden in
# the report.  This is particularly useful if you don't have the Apache
# module "mod_logio" enabled on your webserver.  The value for this
# keyword can be either 'yes' or 'no', with 'no' the default, allowing
# the input/output volume to be displayed.

#HideInOutVolume no

# The GroupDomains keyword allows you to group individual hostnames
# into their respective domains.  The value specifies the level of
# grouping to perform, and can be thought of as 'the number of dots'
# that will be displayed.  For example, if a visiting host is named
# cust1.tnt.mia.uu.net, a domain grouping of 1 will result in just
# "uu.net" being displayed, while a 2 will result in "mia.uu.net".
# The default value of zero disable this feature.  Domains will only
# be grouped if they do not match any existing "GroupSite" records,
# which allows overriding this feature with your own if desired.

#GroupDomains	0

# The GroupShading allows grouped rows to be shaded in the report.
# Useful if you have lots of groups and individual records that
# intermingle in the report, and you want to diferentiate the group
# records a little more.  Value can be 'yes' or 'no', with 'yes'
# being the default.

#GroupShading	yes

# GroupHighlight allows the group record to be displayed in BOLD.
# Can be either 'yes' or 'no' with the default 'yes'.

#GroupHighlight	yes

# The Ignore* keywords allow you to completely ignore log records based
# on hostname, URL, user agent, referrer or username.  I hessitated in
# adding these, since the Webalizer was designed to generate _accurate_
# statistics about a web servers performance.  By choosing to ignore
# records, the accuracy of reports become skewed, negating why I wrote
# this program in the first place.  However, due to popular demand, here
# they are.  Use the same as the Hide* keywords, where the value can have
# a leading or trailing wildcard '*'.  Use at your own risk ;)

#IgnoreSite	bad.site.net
#IgnoreURL	/test*
#IgnoreReferrer	file:/*
#IgnoreAgent	RealPlayer
#IgnoreUser     root

# The Include* keywords allow you to force the inclusion of log records
# based on hostname, URL, user agent, referrer or username.  They take
# precidence over the Ignore* keywords.  Note: Using Ignore/Include
# combinations to selectivly process parts of a web site is _extremely
# inefficent_!!! Avoid doing so if possible (ie: grep the records to a
# seperate file if you really want that kind of report).

# Example: Only show stats on Joe User's pages...
#IgnoreURL	*
#IncludeURL	~joeuser*

# Or based on an authenticated username
#IgnoreUser     *
#IncludeUser    someuser

# The MangleAgents allows you to specify how much, if any, The Webalizer
# should mangle user agent names.  This allows several levels of detail
# to be produced when reporting user agent statistics.  There are six
# levels that can be specified, which define different levels of detail
# supression.  Level 5 shows only the browser name (MSIE or Mozilla)
# and the major version number.  Level 4 adds the minor version number
# (single decimal place).  Level 3 displays the minor version to two
# decimal places.  Level 2 will add any sub-level designation (such
# as Mozilla/3.01Gold or MSIE 3.0b).  Level 1 will attempt to also add
# the system type if it is specified.  The default Level 0 displays the
# full user agent field without modification and produces the greatest
# amount of detail.  User agent names that can't be mangled will be
# left unmodified.

#MangleAgents    0

# The SearchEngine keywords allow specification of search engines and
# their query strings on the URL.  These are used to locate and report
# what search strings are used to find your site.  The first word is
# a substring to match in the referrer field that identifies the search
# engine, and the second is the URL variable used by that search engine
# to define it's search terms.

SearchEngine	yahoo.com	p=
SearchEngine	altavista.com	q=
SearchEngine	google.com	q=
SearchEngine	eureka.com	q=
SearchEngine	lycos.com	query=
SearchEngine	hotbot.com	MT=
SearchEngine	msn.com		MT=
SearchEngine	infoseek.com	qt=
SearchEngine	webcrawler	searchText=
SearchEngine	excite		search=
SearchEngine	netscape.com	search=
SearchEngine	mamma.com	query=
SearchEngine	alltheweb.com	query=
SearchEngine	northernlight.com  qr=

# The Dump* keywords allow the dumping of Sites, URL's, Referrers
# User Agents, Usernames and Search strings to seperate tab delimited
# text files, suitable for import into most database or spreadsheet
# programs.

# DumpPath specifies the path to dump the files.  If not specified,
# it will default to the current output directory.  Do not use a
# trailing slash ('/').

#DumpPath	/var/lib/httpd/logs

# The DumpHeader keyword specifies if a header record should be
# written to the file.  A header record is the first record of the
# file, and contains the labels for each field written.  Normally,
# files that are intended to be imported into a database system
# will not need a header record, while spreadsheets usually do.
# Value can be either 'yes' or 'no', with 'no' being the default.

#DumpHeader	no

# DumpExtension allow you to specify the dump filename extension
# to use.  The default is "tab", but some programs are pickey about
# the filenames they use, so you may change it here (for example,
# some people may prefer to use "csv").

#DumpExtension	tab

# These control the dumping of each individual table.  The value
# can be either 'yes' or 'no'.. the default is 'no'.

#DumpSites	no
#DumpURLs	no
#DumpReferrers	no
#DumpAgents	no
#DumpUsers	no
#DumpSearchStr  no

# If you compiled Webalizer with GeoIP library, it becomes enabled
# by default. But if you wish to disable it, just set GeoIP to 'no'.
# You may also want to specify database file path manually, if you
# don't have one installed on system (in case of static build).

#GeoIP		yes
#GeoIPDatabase	/usr/local/share/GeoIP/GeoIP.dat

# CodeNotFound defines the number of entries in the HTTP-Code 404 error table.
# The default value for CodeNotFound is 20. The code 404 error table may be
# disabled by using zero (0) for the value. Using one (1) for the value
# shows all code 404 errors in the table.

#CodeNotFound    20

# The custom page colors are defined here. Declare them in the standard
# hexadecimal way (as HTML, without the '#'). If none are given, you will get
# the standard webalizer colors.

#ColorBackground        e8e8e8
#ColorText              000000
#ColorLink              0000ff
#ColorVLink             ff0000
#ColorALink             ff0000

# The custom table and chart colors are defined here. Declare them in the
# standard hexadecimal way (as HTML, without the '#'). If none are given, you
# will get the standard webalizer colors.

#ColorGrp               d0d0e0
#ColorHeadline          c0c0c0
#ColorCounter           c0c0c0
#ColorHit               008040
#ColorFile              0080ff
#ColorSite              ff8000
#ColorKbyte             ff0000
#ColorPage              00e0ff
#ColorVisit             ffff00

#ChartBackgroundColor	c0c0c0
#ChartLegendColor	000000
#ChartShadowColor1      ffffff
#ChartShadowColor2      808080

# Colors for four additional pie slices are defined here. Declare them in the
# standard hexadecimal way (as HTML, without the '#'). If none are given, you
# will get the following standard colors: 800080/80ffc0/ff00ff/ffc480.

#PieColor1              800080
#PieColor2              80ffc0
#PieColor3              ff00ff
#PieColor4              ffc480

# TableBorder defines the border size of all tables.
# The default value for TableBorder is 1. The table border may be disabled by
# using zero (0) for the value. Only values between 0 and 5 are allowed.

#TableBorder     1

# ChartBorder defines the border size of all charts.
# The default value for ChartBorder is 1. The chart border may be disabled by
# using zero (0) for the value. Only values between 0 and 5 are allowed.

#ChartBorder     1

# End of configuration file...  Have a nice day!
Any thoughts would be great!

Thanks.
Reply With Quote
  #2   (View Single Post)  
Old 14th July 2011
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by plexter View Post
I'm trying to run "webalizer" for OpenBSD.
Which version of OpenBSD is this, & did you install Webalizer through packages, ports, or did you download this from somewhere else? This is basic information which should be provided when seeking support.
Quote:
But for some reason I cannot get it to read my log file for ProFTPd.
I don't use Webalizer, but in reading through OpenPort's description, it appears that the log file(s) need to exist before Webalizer can analyze them. You should verify for yourself that this is a correct statement.
Quote:
I just get:

Code:
Skipping Bad Record
If Webalizer does expect log files to exist a priori (beforehand), the next step you need to take is to isolate what logging statement is giving it fits. I would suggest:
  1. perusing ProFTPd's log file to determine what atomic actions were taken.
  2. move/rename the current log file.
  3. restart ProFTP & perform whatever ProFTP action is necessary to generate a log message specific to a chosen action.
  4. run Webalizer on this log file containing this single action. Determine whether Webalizer works with this limited data set or not. If Webalizer appears to work, start again with #2 above with a different ProFTP action seen in the original log file(s).
Yes, this may be laborious work, but until you can pinpoint what the offending log message/action is, there isn't much anyone can do for you. It may be that ProFTP is writing an errant character or failing to write a necessary character, but it's in your court to figure this out.
Reply With Quote
  #3   (View Single Post)  
Old 14th July 2011
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

I found this as the first Google hit searching with these two applications as keywords. Please note that they tell you to set a LogType in your config file. http://www.proftpd.org/localsite/Use...nked/x341.html
Reply With Quote
  #4   (View Single Post)  
Old 14th July 2011
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by jggimi View Post
I found this as the first Google hit searching...

0. read available documentation.
Reply With Quote
  #5   (View Single Post)  
Old 15th July 2011
plexter plexter is offline
Shell Scout
 
Join Date: May 2008
Posts: 124
Default

Hi all,

Thanks for your replies. I will do my best to respond to all...

1. I'm running OpenBSD 4.8 Generic
2. I installed Webalizer through package installation
3. Yes I've seen that page and many others.

I must appologize as I seem to have got mixed up with what software I am running. :P I wrote proftpd however I am running pure-ftpd.

I have since got this working by setting my pure-ftpd startup commands to force a partular log format. Both CLF and xferlog formats work fine.

This brings me to another question.

Without specifying a log format/path in the config for pure-ftpd it automatically outputs to /var/log/xferlog which I had figured was in xferlog format. Clearly this is not the case.

I notice that the info in /var/log/xferlog is actually quite detailed. Shows uploads/downloads and other info like when a file is deleted. Yet when I force xferlog in the config I only see when people access a file but not deletions..etc. I'm wondering if there is anyway to tell what type of log the "/var/log/xferlog" actually is?

A snippit of the log is below for reference (cleaned to remove usernames...etc)

Code:
Jul  1 00:15:49 myserver pure-ftpd[23519]: (?@1.1.1.1) [INFO] New connection from 1.1.1.1
Jul  1 00:15:50 myserver pure-ftpd[23519]: (?@1.1.1.1) [INFO] Logout.
Jul  1 00:32:29 myserver pure-ftpd[11500]: (?@1.1.1.1) [INFO] New connection from 1.1.1.1
Jul  1 00:32:30 myserver pure-ftpd[11500]: (?@1.1.1.1) [INFO] Logout.
Jul  1 00:49:09 myserver pure-ftpd[4163]: (?@1.1.1.1) [INFO] New connection from 1.1.1.1
Jul  1 00:49:10 myserver pure-ftpd[4163]: (?@1.1.1.1) [INFO] Logout.
Jul  1 01:00:01 myserver pure-ftpd[9548]: (?@1.1.1.1) [INFO] New connection from 1.1.1.1
Jul  1 01:00:06 myserver pure-ftpd[9548]: (?@1.1.1.1) [WARNING] Authentication failed for user [baduser]
Jul  1 01:00:06 myserver pure-ftpd[9548]: (?@1.1.1.1) [INFO] Logout.
Jul  1 01:05:49 myserver pure-ftpd[28242]: (?@1.1.1.1) [INFO] New connection from 1.1.1.1
Jul  1 01:05:50 myserver pure-ftpd[28242]: (?@1.1.1.1) [INFO] Logout.

Some other notices:

Jul  1 01:45:45 myserver pure-ftpd[29082]: (username@1.1.1.1) [NOTICE] File successfully renamed or moved: [New Folder]->[pictures12]
Jul  1 02:22:38 myserver pure-ftpd[9411]: (username@1.1.1.1) [NOTICE] Deleted /ftp/data1/bigfile.tar

Any thoughts would be great. Once again sorry for the mix-up!

Cheers!
Reply With Quote
  #6   (View Single Post)  
Old 15th July 2011
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Quote:
Any thoughts would be great.
I think you are asking in the wrong place. This is a forum about managing the deployment of third party software to OpenBSD, not the specific use of those programs.

In this particular case, you are asking us to help you understand the interfaces between two of those third party programs, and the specific information produced by one of the two as input to the other.

Your best bet, plexter, is to ask those who are supporting those two programs, through their official channels, or through media run by their interest groups, if any.

You're much more likely to get useful results that way. Few of us (if any) use Webalizer, and / or your particular choice of third party FTP daemon.
Reply With Quote
  #7   (View Single Post)  
Old 15th July 2011
plexter plexter is offline
Shell Scout
 
Join Date: May 2008
Posts: 124
Default

Thanks for your help.

This log file has nothing to do with Webalizer and I was (am) under the impression that it is not from pure-ftpd as when I manually forced xferlog format the logs were quite a bit different. I figured it was a feature from the base install of OpenBSD. Thought someone might know, or be willing to take a stab at it...

Have a nice weekend.
Reply With Quote
  #8   (View Single Post)  
Old 15th July 2011
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

I'll stab. Unless you are using ftpd(8) the format is whatever the third party software wants it to be.
Reply With Quote
  #9   (View Single Post)  
Old 15th July 2011
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,975
Default

Let me be clearer. The use of /var/log/xferlog is defined by your syslog.conf(5) file. The default for that is anything syslog is sent, by any daemon, with selector field entries of "ftp.info" or "ftp.debug".

That daemon can send anything it wants in a character message, there is no standard. Any text string at all, within the limits of a printf(3) string. See syslog(3) for the programming details.

That stated, ftpd(8) writes to a number of logs, because it uses various selector fields. The format of its ftp.info and ftp.debug syslog messages may be found in its source code -- if you have the source code on your system it will be in /usr/src/libexec/ftpd -- the files of interest are ftpd.c for ftp.info messages, and ftpcmd.y, ftpd.c once more, and monitor.c for ftp.debug messages.

If you don't have the source code, and don't feel like installing it, you can dig through the CVS repository for src/libexec/ftpd on the web starting here: http://www.openbsd.org/cgi-bin/cvsweb/src/libexec/ftpd/
Reply With Quote
Old 16th July 2011
plexter plexter is offline
Shell Scout
 
Join Date: May 2008
Posts: 124
Default

Thanks for the info. That helps explain where /var/log/xferlog is coming from. I had no idea that was from syslog, but it does make sense since pure-ftpd does support syslog.

Cool well at least I understand where that file is coming from now and no kidding it doesn't work with webalizer..etc. I guess if I am to pull the results from that log I'll have to write a custom parser of some kind.

Thanks for the clarification.
Reply With Quote
Reply

Tags
log, pureftpd

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Parsing emails with 'awk' and 'perl' J65nko Guides 1 24th February 2011 03:34 AM
Mouse:X (not-working) and tty-Console (working), in 8.0 ykt FreeBSD General 1 22nd December 2009 12:26 PM
Working with CVS? Zmyrgel OpenBSD General 15 6th October 2009 01:32 PM
wireless not working. bsdnewbie999 OpenBSD General 17 19th July 2009 03:06 AM


All times are GMT. The time now is 02:02 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick