NetFlow: installation and configuration of NFDUMP and NfSen on Debian

After the brief overview about the installation of flow-tools and FlowViewer, in this post I’d like to share my experience about the setup of a basic solution based on another pair of tools: NFDUMP and NfSen. As always on my posts, the starting point is a fresh Debian 5.0 setup.

Components used in the solution

As it was for the previous solution, two main components are involved, one to acquire NetFlow data, the other to analyze them in an easy way.

NFDUMP is a suite of tools composed by many programs; nfcapd is one of them and it is the daemon which listens for incoming NetFlow data. It is launched by NfSen.

NfSen is the web based front end we use to analyze NFDUMP NetFlow data. It is splitted up in two parts: a Perl program which runs in the background and launches nfcapd, and some PHP web pages.

The biggest difference between a NFDUMP/NfSen based solution and a flow-tools/FlowViewer solution is that the former does support NetFlow v9, while the latter may result in a simpler and easier tool to work with. Anyway, there is not one perfect solution for all the needs!

In this post I’ll use NFDUMP 1.6.1 and NfSen 1.3.2, the latest stable releases I can find on SourceForge while I’m writing this blog entry; you can find their web pages in the References section of this post. I’ll build them from source code.

NFDUMP

Let’s add some tools we’ll need to build the package:

 

1 apt-get install gcc flex librrd-dev make

We are ready to start! The steps are very simple: download source code and extract the tar:

1 cd /usr/local/src/
3 gzip -dc nfdump-1.6.1.tar.gz | tar -xf -
4 cd nfdump-1.6.1

In order to use NFDUMP with NfSen we need to enable nfprofile in the configure step; then we have to make && make install it:

 

1 ./configure --enable-nfprofile
2 make
3 make install

Now it is ready to be ran.

If we want to use it without NfSen we can stop here and create a simple init.d script to launch NFDUMP at startup; it will acquire NetFlow data and we can use command line tools (such as nfdump) to analyze them. You can find a basic init.d script at the end of this post (“Annex A: stand-alone NFDUMP”).

But that’s not our goal, we want NfSen!

NfSen

As said, NfSen has a PHP front-end so it needs a web server with PHP support in order to be used. It also uses RRD to store data and paint nice graphs and some Perl modules to produce alerts: let’s satisfy its prerequisites:

1 apt-get install apache2 libapache2-mod-php5 php5-common libmailtools-perl rrdtool librrds-perl

Now it’s time to download and configure the program:

1 cd /usr/local/src/
3 gzip -dc nfsen-1.3.2.tar.gz | tar -xf -
4 cd nfsen-1.3.2

Once extracted, we have to copy the config file into /etc and modify it:

1 cp etc/nfsen-dist.conf /etc/nfsen.conf
2 nano /etc/nfsen.conf

As you can see from the support web page it presents a lot of parameters, but many of them may be left unchanged; in this example I chose to leave as many parameters as possible at their default settings, so we have all the files in a single directory: /data/nfsen. Here the parameters I changed:

$USER    = "www-data";
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";

%sources = (
    'MYROUTER'    => { 'port'    => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
);

$MAIL_FROM   = 'MYEMAIL@MYDOMAIN.COM';
$SMTP_SERVER = 'MY.SMTPSERVER.COM';

For a better setup I suggest you to read the configuration guide on the project’s web site.

Let’s make the destination directory and complete the installation by running the install.pl script:

1 mkdir -p /data/nfsen
2 ./install.pl /etc/nfsen.conf

We are ready to run the nfsen program and let it executes nfcapd daemon:

1 cd /data/nfsen/bin
2 ./nfsen start

We can also schedule it to be ran at startup:

1 ln -s /data/nfsen/bin/nfsen /etc/init.d/nfsen
2 update-rc.d nfsen defaults 20

It’s ready: point your browser at http://YOUR_IP_ADDRESS/nfsen/nfsen.php and enjoy! Of course, this is just a very basic setup, both NFDUMP and NfSen have many options you can set and use to have better performances and to improve stability and scalability. If you want to go deep into their configuration I suggest you to use the links you can find at the end of this post.

P.S.: Don’t forget to configure your routers to export NetFlow data! ;-)

Annex A: stand-alone NFDUMP

As said, NFDUMP may be used in a stand-alone way, without the help of NfSen.

We just have to build a destination directory for NetFlow data…

1 mkdir -p /var/flows/MYROUTER
2 mkdir -p /var/flows/MYSECONDROUTER

and add an init.d script like the following one:

001 #! /bin/sh
002 ### BEGIN INIT INFO
003 # Provides:          nfcapd
004 # Required-Start:    $network
005 # Required-Stop:
006 # Default-Start:     2 3 4 5
007 # Default-Stop:      0 1 6
008 # Short-Description: netflow capture daemon
009 # Description:       nfcapd is the netflow capture daemon of the nfdump tools.
010 ### END INIT INFO
011
012 # Author: Erik Wenzel <erik@debian.org>
013 # Edited by Pierky for the blog post "NetFlow: installation and configuration of
014 # NFDUMP and NfSen on Debian" on http://pierky.wordpress.com
015
016 # Do NOT "set -e"
017
018 # PATH should only include /usr/* if it runs after the mountnfs.sh script
019 PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin
020 DESC="netflow capture daemon"
021 NAME=nfcapd
022 DAEMON=/usr/local/bin/$NAME
023 DATA_BASE_DIR="/var/flows"
024 PIDFILE=/var/run/$NAME.pid
025 DAEMON_ARGS="-D -w -S 1 -P $PIDFILE -n MYROUTER,192.168.0.1,$DATA_BASE_DIR/MYROUTER"
026 SCRIPTNAME=/etc/init.d/nfdump
027
028 # Exit if the package is not installed
029 [ -x "$DAEMON" ] || exit 0
030
031 # Load the VERBOSE setting and other rcS variables
032 . /lib/init/vars.sh
033
034 # Define LSB log_* functions.
035 # Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
036 . /lib/lsb/init-functions
037
038 #
039 # Function that starts the daemon/service
040 #
041 do_start()
042 {
043         # Return
044         #   0 if daemon has been started
045         #   1 if daemon was already running
046         #   2 if daemon could not be started
047         start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
048                 || return 1
049         start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
050                 $DAEMON_ARGS \
051                 || return 2
052         # Add code here, if necessary, that waits for the process to be ready
053         # to handle requests from services started subsequently which depend
054         # on this one.  As a last resort, sleep for some time.
055 }
056
057 #
058 # Function that stops the daemon/service
059 #
060 do_stop()
061 {
062         # Return
063         #   0 if daemon has been stopped
064         #   1 if daemon was already stopped
065         #   2 if daemon could not be stopped
066         #   other if a failure occurred
067         start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
068         RETVAL="$?"
069         "$RETVAL" = 2 ] && return 2
070         # Wait for children to finish too if this is a daemon that forks
071         # and if the daemon is only ever run from this initscript.
072         # If the above conditions are not satisfied then add some other code
073         # that waits for the process to drop all resources that could be
074         # needed by services started subsequently.  A last resort is to
075         # sleep for some time.
076         #
077         # Disabled second call, because is kills nfsen controlled nfcapd
078         #start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
079         #[ "$?" = 2 ] && return 2
080         # Many daemons don't delete their pidfiles when they exit.
081         rm -f $PIDFILE
082         return "$RETVAL"
083 }
084
085 #
086 # Function that sends a SIGHUP to the daemon/service
087 #
088 do_reload() {
089         #
090         # If the daemon can reload its configuration without
091         # restarting (for example, when it is sent a SIGHUP),
092         # then implement that here.
093         #
094         start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
095         return 0
096 }
097
098 case "$1" in
099   start)
100         "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
101         do_start
102         case "$?" in
103                 0|1) [ q"$VERBOSE" != qno ] && log_end_msg 0 ;;
104                 2)   [ p"$VERBOSE" != pno ] && log_end_msg 1 ;;
105         esac
106         ;;
107   stop)
108         "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
109         do_stop
110         case "$?" in
111                 0|1) [ "r$VERBOSE" != rno ] && log_end_msg 0 ;;
112                 2) [ "s$VERBOSE" != sno ] && log_end_msg 1 ;;
113         esac
114         ;;
115   #reload|force-reload)
116         #
117         # If do_reload() is not implemented then leave this commented out
118         # and leave 'force-reload' as an alias for 'restart'.
119         #
120         #log_daemon_msg "Reloading $DESC" "$NAME"
121         #do_reload
122         #log_end_msg $?
123         #;;
124   restart|force-reload)
125         #
126         # If the "reload" option is implemented then remove the
127         # 'force-reload' alias
128         #
129         log_daemon_msg "Restarting $DESC" "$NAME"
130         do_stop
131         case "$?" in
132           0|1)
133                 do_start
134                 case "$?" in
135                         0|1) log_end_msg 0 ;;
136                         *) log_end_msg 1 ;; # Failed to start
137                 esac
138                 ;;
139           *)
140                 # Failed to stop
141                 log_end_msg 1
142                 ;;
143         esac
144         ;;
145   *)
146         #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
147         echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
148         exit 3
149         ;;
150 esac
151 exit 0

I edited this script starting from the one included in the Debian’s package.

Please note the DATA_BASE_DIR and the DAEMON_ARGS variables. The first contains the base destination directory for our NetFlow data. The second represents arguments we want to use when launching nfcapd; you can add as many -n options as you want, one for each NetFlow source you want to collect data from. Remember: man nfcapd is your friend! ;)

Finally:

1 chmod a+x /etc/init.d/nfdump
2 update-rc.d nfdump defaults 20
3 /etc/init.d/nfdump start

 

References

Cisco.com: NetFlow version 9

NFDUMP: http://nfdump.sourceforge.net/

NfSen: http://nfsen.sourceforge.net/

Comments are closed.