DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 11th June 2008
centerstage centerstage is offline
New User
 
Join Date: Jun 2008
Posts: 4
Default Interface - total bandwidth

Hello all,

I am trying to find a simple solution to a seemingly common requirement. We need to keep track of total bandwidth by month on our external interface as our
new ISP charges $15/gb over our monthly cap

We also don't need fancy graphs just text is fine.

If anyone can recommend something it would be greatly appreciated.
Reply With Quote
  #2   (View Single Post)  
Old 11th June 2008
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 192
Default

Run into it a while ago, see if it helps you:

https://calomel.org/bandwidth_usage.html
Reply With Quote
  #3   (View Single Post)  
Old 11th June 2008
mish's Avatar
mish mish is offline
Port Guard
 
Join Date: May 2008
Posts: 25
Default

Have a look at the following and see if any one suits your needs:

/usr/ports/net/bwm-ng <= probably your best bet
/usr/ports/net/ifstat
/usr/ports/net/iftop

There's also ipband (Google it), but it's not in OpenBSD ports. See if you can compile it anyway.

Good luck.
Reply With Quote
  #4   (View Single Post)  
Old 11th June 2008
centerstage centerstage is offline
New User
 
Join Date: Jun 2008
Posts: 4
Default

Thanks for the suggestions. I have set them all up and will see what i get after they have a chance to run for a bit. The camomel script seems to have the best output. I put it in the daily cron so I am emailed the stats every day. The only issue i can see is that the statistics get reset on reboot.

I am going to keep searching as ideally i would like the output to look like:

Today's Bandwidth: x/gb
Yesterday's Bandwidth: x/gb
Current Month: x/gb
Previous Month: x/gb

Don't know if i am expecting too much......

Anyways I will give an update tomorrow
Reply With Quote
  #5   (View Single Post)  
Old 12th June 2008
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

This is a raw extraction of the byte-counting code from the netstat command.. this may be useful to someone who's attempting to write their own application.

Code:
#include <sys/param.h>
#include <sys/socket.h>
#include <sys/sysctl.h>

#include <net/if.h>
#include <net/if_dl.h>
#include <net/route.h>

int
main(void)
{
	struct if_msghdr ifm;
	int mib[6] = { CTL_NET, AF_ROUTE, 0, 0, NET_RT_IFLIST, 0 };
	struct rt_msghdr *rtm;
	struct if_data *ifd;
	struct sockaddr *sa, *rti_info[RTAX_MAX];
	struct sockaddr_dl *sdl;
	char *buf, *next, *lim;
	char name[IFNAMSIZ];
	size_t len;
	int loop_index;

	if (sysctl(mib, 6, NULL, &len, NULL, 0) == -1)
		err(1, "sysctl");

	if ((buf = (char *)malloc(len)) == NULL)
		err(1, NULL);

	if (sysctl(mib, 6, buf, &len, NULL, 0) == -1)
		err(1, "sysctl");

	lim = buf + len;
	for (next = buf; next < lim; next += rtm->rtm_msglen) {
		rtm = (struct rt_msghdr *)next;
		if (rtm->rtm_version != RTM_VERSION)
			continue;

		if (rtm->rtm_type == RTM_IFINFO) {
			bcopy(next, &ifm, sizeof(ifm));
			ifd = &ifm.ifm_data;

			sa = (struct sockaddr *)(next + rtm->rtm_hdrlen);
			for (loop_index = 0; loop_index < RTAX_MAX; loop_index++) {
				if (ifm.ifm_addrs & (1 << loop_index)) {
					rti_info[loop_index] = sa;
					sa = (struct sockaddr *)((char *)(sa) + 
						roundup(sa->sa_len, sizeof(long)));
				} else {
					rti_info[loop_index] = NULL;
				}
			}

			sdl = (struct sockaddr_dl *)rti_info[RTAX_IFP];
			if (sdl == NULL || sdl->sdl_family != AF_LINK)
				continue;

			bzero(name, sizeof(name));
			if (sdl->sdl_nlen >= IFNAMSIZ) {
				memcpy(name, sdl->sdl_data, IFNAMSIZ - 1);
			} else if (sdl->sdl_nlen > 0) {
				memcpy(name, sdl->sdl_data, sdl->sdl_nlen);
			}

			if(!strcmp(name, "rl0")) {
				printf("%10llu %10llu\n", ifd->ifi_ibytes, ifd->ifi_obytes);
			}
		}
	}
}
The second strcmp argument must be altered to match your Ethernet interface, any LAN traffic over the link is also recorded.

The equivalent command is:
$ netstat -ib
Take care.

Last edited by BSDfan666; 12th June 2008 at 03:43 AM.
Reply With Quote
  #6   (View Single Post)  
Old 14th June 2008
ephemera's Avatar
ephemera ephemera is offline
Knuth's homeboy
 
Join Date: Apr 2008
Posts: 537
Default

Quote:
Originally Posted by centerstage View Post
Hello all,

I am trying to find a simple solution to a seemingly common requirement. We need to keep track of total bandwidth by month on our external interface as our
new ISP charges $15/gb over our monthly cap

We also don't need fancy graphs just text is fine.
I have written a daemon for my own amusement. You can try it out.

It can track the b/w usage correctly across reboots or even an OS crash.

the log file /var/log/bwmon.[ifn] can be used to find the b/w usage across any time range.

startup and error messages are logged to /var/log/daemon.

Source code:
Code:
/*
 *      Persistant net bandwidth usage monitor daemon 
 *      
 *      Copyright: ephemera @ daemonforums.org
 */
  
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>
#ifdef BSD
# include <sys/socket.h>
# include <ifaddrs.h>
#endif /* BSD */
#ifdef SOLARIS
# include <kstat.h>
#endif /* SOLARIS */

#include <net/if.h>

#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#ifdef LINUX
# include <regex.h>
#endif /* LINUX */
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <syslog.h>
#include <time.h>
#include <unistd.h>

#define ARGV0   "bwmond"

typedef unsigned long long      Bwbyte_t;

static char     LOGFILE[PATH_MAX] = "/var/log/"ARGV0".";
static char     RECFILE[PATH_MAX] = "/var/spool/"ARGV0".";
static char     interface[IFNAMSIZ];

static void     usage(void);
static int      update(Bwbyte_t *, Bwbyte_t *);
static void     logpr(int, char *, ...);
static void     err_sys(int, char *, char *);
static void     sig_term(int);

int
main(int argc, char *argv[])
{
        int             i, recfd, logfd;
        time_t          t, t2;
        Bwbyte_t        u, d, last_up, last_down, hr_up, hr_down;
        Bwbyte_t        *record;
        enum            {UP, DOWN};
        int    		record_len = 2 * sizeof(*record);
        const int       interval = 60;     
        int		log_interval = 60 * 5; 
        pid_t           pid;
        sigset_t        sigset;
        struct sigaction       sigact;

        switch (--argc) {
        case 2:
		log_interval = 60 * strtoul(*++argv, (char **)NULL, 10);
                if (log_interval <= 0) 
                        usage();
                /* FALLTHROUGH */

        case 1:
		strncpy(interface, *++argv, sizeof(interface));
                interface[sizeof(interface) - 1] = '\0';
                strncat(RECFILE, interface, sizeof(interface) - 1);
                strncat(LOGFILE, interface, sizeof(interface) - 1);
                break;

        default:
		usage();
        }

        if ( (pid = fork()) < 0) 
                err_sys(1, "fork", "");

        if (pid > 0)
                exit(0);

        setsid();

        chdir("/");

        umask(0);

        for (i = getdtablesize(); i >= 0; --i) 
                close(i); 

        i = open("/dev/null", O_RDWR);
        dup2(i, STDIN_FILENO);
        dup2(i, STDOUT_FILENO);
        dup2(i, STDERR_FILENO);

        sigfillset(&sigset);    
        sigdelset(&sigset, SIGTERM);   
        sigprocmask(SIG_BLOCK, &sigset, NULL);
        sigact.sa_handler = sig_term;
        sigemptyset(&sigact.sa_mask);
        sigact.sa_flags = 0;
        sigaction(SIGTERM, &sigact, NULL);

        openlog(ARGV0, LOG_PID, LOG_DAEMON);
        syslog(LOG_INFO, "Started on %s.", interface);

        if (access(LOGFILE, F_OK) != 0) 
                syslog(LOG_NOTICE, "Creating Logfile: %s. "
			       "If this is not the first time that "ARGV0" is being run then this indicates a Problem."
			       , LOGFILE);

        logfd = open(LOGFILE, O_WRONLY | O_APPEND | O_CREAT, S_IRUSR | S_IWUSR);
        if (logfd < 0) 
                err_sys(1, "open", LOGFILE);

        if (access(RECFILE, F_OK) != 0) {
                recfd = open(RECFILE
                             ,O_RDWR | O_CREAT | O_SYNC
                             ,S_IRUSR | S_IWUSR);

                if (recfd >= 0) {
                        syslog(LOG_NOTICE, "Creating Recordfile: %s. "
			       "If this is not the first time that "ARGV0" is being run then this indicates a Problem."
			       , RECFILE);
                        if (lseek(recfd, record_len, SEEK_SET) < 0)
                                err_sys(1, "lseek", RECFILE);

                        if (write(recfd, "", 1) < 1)
                                err_sys(1, "write", RECFILE);
                }
        } else {
                recfd = open(RECFILE, O_RDWR | O_SYNC);
        }
        if (recfd < 0)
                err_sys(1, "open", RECFILE);

        record = (Bwbyte_t *) mmap(NULL
                                   ,record_len
                                   ,PROT_READ | PROT_WRITE
                                   ,MAP_SHARED
                                   ,recfd
                                   ,0);

        if ( (caddr_t)record == (caddr_t)-1)
                err_sys(1, "mmap", RECFILE);

        close(recfd);

        u = d = last_up = last_down = hr_up = hr_down = 0;
	t = time(NULL);

        while (!update(&hr_up, &hr_down) || !update(&last_up, &last_down))
		sleep(interval);

        for ( ; ; ) {
                if (update(&u, &d)) {
                         if (u < last_up || d < last_down) {
                                last_up = last_down = hr_up = hr_down = 0;
                                continue;
                         }
                         record[UP] += u - last_up;
                         record[DOWN] += d - last_down;
                         last_up = u;
                         last_down = d;
			 t2 = time(NULL);
			 if (t2 - t >= log_interval) {
                                 logpr(logfd
                                       ,"%6llu kb Up  %6llu kb Dn  %10llu / %llu\n"
                                       ,(last_up - hr_up) / 1024 
                                       ,(last_down - hr_down) / 1024
                                       ,record[UP] / 1024
                                       ,record[DOWN] / 1024);
			 	 t = t2;
                                 hr_up = last_up;
                                 hr_down = last_down;
                         }
                         msync((void *)record, record_len, MS_SYNC);
                }
                sleep(interval);
        }
        /* NOTREACHED */
}

static void
err_sys(int quit, char *fn, char *arg)
{

        if (!*arg)
                syslog(LOG_ERR, "Error: %s: %m\n", fn); 
        else
                syslog(LOG_ERR, "Error: %s: %s: %m\n", fn, arg); 

        if (quit) {
                syslog(LOG_ERR, "Terminated due to error on %s.", interface);
                closelog();
                exit(quit);
        }
}

static int
update(Bwbyte_t *up, Bwbyte_t *down)
{
        int    updated = 0;

        errno = 0;

#ifdef BSD
        struct ifaddrs *ifa, *f;

        if (getifaddrs(&ifa) == 0) {
                for (f = ifa; f; f = f->ifa_next) {
                        if (strcmp(interface, f->ifa_name) == 0 && 
                            f->ifa_addr->sa_family == AF_LINK) {
                                *down = ((struct if_data*)f->ifa_data)->ifi_ibytes;
                                *up =((struct if_data*)f->ifa_data)->ifi_obytes;
                                updated = 1;
                                break;
                        }
                }
                freeifaddrs(ifa);
        }
#endif  /* BSD */
#ifdef LINUX
        char            line[2048];
	static char	repatrn[128] = "";
        regmatch_t      rematch[3];
        static regex_t  regex;
        FILE            *pfd;

	if (!*repatrn) {
	        snprintf(repatrn, sizeof(repatrn)
       	        	 ,"^ *%s: *([0-9]+) +[0-9]+ +[0-9]+ +[0-9]+ +[0-9]+" 
       	        	  " +[0-9]+ +[0-9]+ +[0-9]+ +([0-9]+) "
       	        	 ,interface);

                if (regcomp(&regex, repatrn, REG_EXTENDED) != 0)
			repatrn[0] = 0;
	}
        if (*repatrn && (pfd = fopen("/proc/net/dev", "r")) != NULL) {
                while(fgets(line, sizeof(line), pfd) != NULL) {
                        if (regexec(&regex, line, 3, rematch, 0) == 0) {
                                *down = atoll(line + rematch[1].rm_so);
                                *up = atoll(line + rematch[2].rm_so);
                                updated = 1;
                                break;
                        }
                }
        	fclose(pfd);
        }
#endif  /* LINUX */
#ifdef SOLARIS
        kstat_ctl_t     *kc;
        kstat_t         *ksp;
        kstat_named_t   *knp;

        if ( (kc = kstat_open()) != NULL) {
                /* workaround? kstat_open sets errno even on success */
                errno = 0;
                for (ksp = kc->kc_chain; ksp != NULL; ksp = ksp->ks_next) {
                        if (strcmp(ksp->ks_class, "net") == 0 &&
                            strcmp(ksp->ks_name, interface) == 0 &&
                            kstat_read(kc, ksp, NULL) != -1) {
                                if ((knp=kstat_data_lookup(ksp,"rbytes64")) != NULL)
                                        *down = knp->value.ui64;
                                if ((knp=kstat_data_lookup(ksp,"obytes64")) != NULL) {
                                        *up = knp->value.ui64;
                                        updated = 1;
                                        break;
                                }
                        } 
                }
                kstat_close(kc);
        }
#endif  /* SOLARIS */

        if (errno)
                err_sys(0, "update", "");

        return updated;
}

static void
logpr(int logfd, char *msg, ...)
{
#define LOG_LINE_MAX	256
        char    logent[LOG_LINE_MAX];        
	size_t	r;
        time_t  clk;
        va_list ap;

        va_start(ap, msg);
        clk = time(NULL);
        r = strftime(logent, sizeof(logent)
		     ,"%b %e %H:%M:%S %Y: "
		     ,localtime(&clk));
        vsnprintf(logent + r, sizeof(logent) - r, msg, ap);
	if (write(logfd, logent, strlen(logent)) < 0)
		err_sys(0, "logpr", "");

        va_end(ap);
}

static void
usage(void)
{

        fprintf(stderr, "Usage: %s [log interval] interface\n",
                ARGV0);
        exit(1);
}

static void
sig_term(int signo)
{
        
        syslog(LOG_INFO, "Stopped on %s.", interface);   
        closelog();
        exit(0);
}
save the above to a file bwmon.c.

then compile: cc -O2 -DBSD -o bwmond bwmon.c && strip bwmond

you might want to run it from rc.local at startup. Example:

/path/to/bwmond em0 && echo -n ' bwmon'

(note: the interface argument is mandatory.)

Last edited by ephemera; 22nd October 2008 at 04:04 PM.
Reply With Quote
  #7   (View Single Post)  
Old 15th June 2008
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

Whoo, ephemera used my alterations to the netstat code.

EDIT: While you're not likely to run into this yet, consider changing the size of the interface variable to IFNAMSIZ.
EDIT2: You're using OpenBSD, also consider using strlcpy/strlcat instead of strncpy/strncat..
EDIT3: I'm nagging now, ignore me.. -pedantic, anyway, consider making this daemon privsep, there shouldn't be a need to run this as root.

Last edited by BSDfan666; 15th June 2008 at 04:36 PM.
Reply With Quote
  #8   (View Single Post)  
Old 15th June 2008
ephemera's Avatar
ephemera ephemera is offline
Knuth's homeboy
 
Join Date: Apr 2008
Posts: 537
Default

Quote:
Originally Posted by centerstage View Post
I am going to keep searching as ideally i would like the output to look like:

Today's Bandwidth: x/gb
Yesterday's Bandwidth: x/gb
Current Month: x/gb
Previous Month: x/gb

Don't know if i am expecting too much......

Anyways I will give an update tomorrow
Code:
#!/usr/bin/perl
# Copyright: ephemera @ daemonforums.org

use POSIX("mktime");
die "Usage: $0 interface\n" if $#ARGV;
$logfile="/var/log/bwmond.$ARGV[0]";
open LOG,"< $logfile" or die "$logfile: $!\n"; 
@abbr = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
($min,$date,$mon,$year)=(localtime)[1,3..5]; 
($ydate,$ymon,$yyear)=(localtime mktime 0, 0, 0, $date-1, $mon, $year)[3..5];
($ldate,$lmon,$lyear)=(localtime mktime 0, 0, 0, $date, $mon-1, $year)[3..5]; 
$year+=1900; $yyear+=1900; $lyear+=1900;
%what=(d=>'Today',yd=>'Yesterday',m=>'This Month',lm=>'Last Month');
for (reverse <LOG>) {
	$end=1;
	if (!defined $lent) {
		/(.*):.*/;
		$lent=$1;
	}	
	tr/:/ /;
	@F = split;
	if ($F[5] == $year && $F[0] eq $abbr[$mon]) { 
		push @m, [@F];
		if ($F[1] == $date) {
			push @d, [@F];
		}
		$end=0;
	} 
	if ($F[5] == $yyear && $F[0] eq $abbr[$ymon] && $F[1]==$ydate) {
		push @yd,[@F]; $end=0;
	} 
	if ($F[5] == $lyear && $F[0] eq $abbr[$lmon]) {
		push @lm,[@F]; $end=0;
	} 
	last if $end;
}
printf "\n%23s%10s\n", '| Dnload(MB)', '| Upload(MB)|';
printf "%s\n", '-----------+-----------+-----------+';
for (('d','yd','m','lm')) {
 if (defined @{"$_"}) {
  $up = ${"$_"}[0]->[12] - ${"$_"}[-1]->[12] + ${"$_"}[-1]->[6]; 
  $down = ${"$_"}[0]->[14] - ${"$_"}[-1]->[14] + ${"$_"}[-1]->[9];
  printf "%-10s |  %8.1f |  %8.1f |\n",$what{"$_"},$down/1024,$up/1024;
 } else {
  printf("%-10s |         - |         - |\n", $what{"$_"});
 }
}
print "\nLast updated on $lent\n";

Last edited by ephemera; 22nd October 2008 at 12:36 PM.
Reply With Quote
  #9   (View Single Post)  
Old 15th June 2008
ephemera's Avatar
ephemera ephemera is offline
Knuth's homeboy
 
Join Date: Apr 2008
Posts: 537
Default

Quote:
Originally Posted by BSDfan666 View Post
Whoo, ephemera used my alterations to the netstat code.
I ripped your code in true OSS spirit.
but didn't you say earlier that it was a 'raw' copy?

Last edited by ephemera; 15th June 2008 at 07:52 PM.
Reply With Quote
Old 15th June 2008
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Banned
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Default

Quote:
Originally Posted by ephemera View Post
I ripped your code in true OSS spirit.
but didn't you say earlier that it was a 'raw' copy?
Well, raw-ish, changes some things to make it pretty, and drop a local function. (get_rtaddrs).

It should be noted, Copyright (c) 1983, 1988, 1993 - The Regents of the University of California. All rights reserved.
Reply With Quote
Old 15th June 2008
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 4,125
Default

Quote:
Originally Posted by centerstage View Post
We need to keep track of total bandwidth by month on our external interface as our new ISP charges $15/gb over our monthly cap
BTW running a caching nameserver and/or a proxy like squid are great for reducing your bandwidth and thus your costs.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump
Reply With Quote
Old 18th June 2008
centerstage centerstage is offline
New User
 
Join Date: Jun 2008
Posts: 4
Default

Wow I am truly blown away by the responses. Thanks a bunch!

Ephemera: "I have written a daemon for my own amusement." that's amazing I will be trying this out asap.

J65nko: All of our "regular" traffic goes out an unlimited connection so there is no worry there. We are already running a proxy to filer out ad's and other various crap.

If anyone was wondering, I ran the script s0xxx suggested for a few days and something is messed up with it. Here is the output:

External interface bandwidth usage:
uptime 10 days
ExtIf in total -1 GBytes
ExtIf out total 1 GBytes
ExtIf in/day 0 MBytes/day
ExtIf out/day 0 MBytes/day
ExtIf in/30day 0 GBytes/month
ExtIf out/30day 0 GBytes/month
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Web interface for rTorrent Beastie FreeBSD Ports and Packages 0 24th August 2009 11:53 AM
Total Success divadgnol67 OpenBSD General 0 6th August 2009 07:15 PM
CARP interface with DHClient xinform3n OpenBSD General 5 22nd July 2009 12:41 PM
NAT with only one interface zapov General software and network 4 16th February 2009 03:45 AM
Web interface for pf? windependence OpenBSD Security 4 20th May 2008 03:58 AM


All times are GMT. The time now is 06:23 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick