Building CentOS 7 NetFlows Monitoring station with nfsen and nfdump

Posted On October 14, 2016

In this article we will look into setting up NetFlows monitoring station with open source tools. It is extremely important to keep track of what is happening on your network, who are the highest talkers and which users or programs accessing which resources. In our LAB example we will have Cisco Router on which we will be exporting netflows and Linux based server on which we will run a tool called softflowd. Softflowd is open source tool capable of generating netflows. Nfsen is an open source tool and more information on this tool can be found at http://nfsen.sourceforge.net/#mozTocId467189.

 

General Preparation

#yum update -y    # update system and packages.

#vi /etc/selinux/config   #change directive SELINUX=disabled

We will need to install a number of packages for CentOS 7

#yum install -y httpd php wget gcc make rrdtool-devel rrdtool-perl perl-MailTools perl-Socket6 flex byacc

yum install perl-Sys-Syslog

yum install perl-Data-Dumperu

Create user account and add it to proper group

#useradd netflow
#usermod -a -G apache netflow

Create directories which we will specify later in configuration file

#mkdir -p /data/nfsen
#mkdir -p /var/www/html/nfsen

Now we will download latest nfdump and nfsen packages at this time nfdump-1.6.13.tar.gz and nfsen-1.3.6p1.tar.gz

#cd /opt
#wget http://downloads.sourceforge.net/project/nfdump/stable/nfdump-1.6.13/nfdump-1.6.13.tar.gz
#wget http://downloads.sourceforge.net/project/nfsen/stable/nfsen-1.3.6p1/nfsen-1.3.6p1.tar.gz

Configure iptables

In this tutorial we will disable original firewall that comes with CentOS 7 and install old good iptables.

#systemctl mask firewalld
#systemctl stop firewalld
#yum -y install iptables-services
#systemctl enable iptables
#systemctl start iptables
#iptables -I INPUT -p tcp –dport 80 -j ACCEPT
#iptables -I INPUT -s "destination ip" -p udp --dport 9666 -j ACCEPT

also add any additional ports that you need open

Start httpd service

#service httpd start

Install nfdump

 

Untar downloaded nfdump package

/opt#tar -zxvf nfdump-1.6.13.tar.gz

/opt#cd nfdump-1.6.13

Compile nfdump

/opt/nfdump-1.6.13#./configure --enable-nfprofile --enable-nftrack --enable-sflow

/opt/nfdump-1.6.13#make && sudo make install

Install and configure nfsen

Untar nfsen

/opt/#tar -zxvf nfsen-1.3.6p1.tar.gzls
/opt/#cd nfsen-1.3.6p1

Copy nfsen example configuration file to etc directory

/opt/nfsen-1.3.6p1/cp etc/nfsen-dist.conf /opt/nfsen-1.3.6p1/etc/nfsen.conf

Now we need to edit this configuration file to make sure all variables are set correctly

#vi /etc/nfsen.conf

Make sure all data path variables are set correctly

$BASEDIR= "/data/nfsen";

$HTMLDIR = "/var/www/nfsen";

For CentOS based systems change

$WWWUSER  = "www";   change to apache
$WWWGROUP = "www"; change to apache
$WWWUSER  = "apache";
$WWWGROUP = "apache";

We will now run perl installation script to install nfsen

/opt/nfsen-1.3.6p1/#./install.pl etc/nfsen.conf

Press enter to accept default path. You may get Errors since we did not configure any flows at this point.

Lets now create a startup script for the service

#vi /etc/init.d/nfsen

#!/bin/bash
#
# chkconfig: - 50 50
# description: nfsen

DAEMON=/data/nfsen/bin/nfsen

case "$1" in
start)
$DAEMON start
;;
stop)
$DAEMON stop
;;
status)
$DAEMON status
;;
restart)
$DAEMON stop
sleep 1
$DAEMON start
;;
*)
echo "Usage: $0 {start|stop|status|restart}"
exit 1
;;
esac

exit 0

 

make sure the script is executable

#chmod +x /etc/init.d/nfsen

Start nfsen deamon

#/etc/init.d/./nfsen start

At this point you should be able to access nfsen at http://127.0.0.1/nfsen/nfsen.php

 

 

Configuration of Cisco router to export netflows

This is very basic example showing how to export netflows on Cisco 870 series Router.

On the Cisco router

conf#int fa04
ip route-cache flow
show ip cache flow
clear ip flow stats
ip flow-export destination 92.168.2.47 2055

On nfsen system edit /etc/nfsen.conf file and make sure sources entry exists

%sources = (
'CiscoRouter'    => { 'port' => '2055', 'col' => '#0000ff', 'type' => 'netflow' },
);

#data/nfsen/bin/nfsen reconfig
#/etc/init.d/nfsen restart

Softflowd configuration on Linux server

This is basic example of how to setup netflows on Linux based server

On Linux server:

yum install libtool automake autoconf python-devel
yum install libpcap-devel
wget http://softflowd.googlecode.com/files/softflowd-0.9.9.tar.gz
tar -zxvf softflowd-0.9.9.tar.gz
./configure
make
make install
#softflowd -i eth0 -n 192.168.2.47:9666

You may also add this line at the end of /etc/rc.d/rc.local file to make sure it will be started as system starts.

make sure that the software is actually working, check the softflow statistics after softflowd has been running for a few moments.

#softflowctl statistics

 
On nfsen workstation:

Add the followinf to /etc/nfsen.conf

%sources = (

'CiscoRouter'    => { 'port' => '2055', 'col' => '#0000ff', 'type' => 'netflow' },
'LinuxServer'    => { 'port' => '9666', 'col' => #088A08', 'type' => 'netflow' },
);
data/nfsen/bin/nfsen reconfig
/etc/init.d/nfsen restart

 

Troubleshooting

1. Install tcpdump and verify that flows are being send on specified port.

#yum install tcpdump

#tcpdump -i eth0 port 9666

Make sure that you see traffic on this port from required host.

2. With nfdump you can read flow collection files from command line

cd /data/nfsen/profiles-data/live/LinuxServer/2014/12/13

nfdump -r  "your file name"

3. Make sure is that your system data and php date set correctly. You may need to edit /etc/php.ini and adjust your date.timezone = "US/Eastern"

4. When you first visit your nfsen website you get error message Frontend – Backend version mismatch! You get this message the first time you connect because your browser doesn’t use the correct cookie/session id. After you navigate to a new page you get a cookie. This error can safely be ignored.

5. Make sure remote system, the one generating netflows has correct timezone and time set.

6. Troubleshoot nfcapd

First check running fcapd processes

#ps axo command | grep '[n]fcapd'

Check which ports nfcapd is listenoing on

#lsof -Pni | grep nfcapd

To test if nfcapd receiving data

#pidof nfcapd
"port num"
#strace -p "port num"

7. If you suspect there is a problem with nfcapd you can try runing it on Linux Server, You will need to install nfdump first.

#softflowd -i eth2 -n 127.0.0.1:9997

#mkdir test
#nfcapd -w -D -l test -p 9997