Building CentOS 7 NetFlows Monitoring station with nfsen and nfdump
In this article we will look into setting up NetFlows monitoring station with open source tools. It is extremely important to keep track of what is happening on your network, who are the highest talkers and which users or programs accessing which resources. In our LAB example we will have Cisco Router on which we will be exporting netflows and Linux based server on which we will run a tool called softflowd. Softflowd is open source tool capable of generating netflows. Nfsen is an open source tool and more information on this tool can be found at http://nfsen.sourceforge.net/#mozTocId467189.
General Preparation
#yum update -y # update system and packages. #vi /etc/selinux/config #change directive SELINUX=disabled
We will need to install a number of packages for CentOS 7
#yum install -y httpd php wget gcc make rrdtool-devel rrdtool-perl perl-MailTools perl-Socket6 flex byacc yum install perl-Sys-Syslog yum install perl-Data-Dumperu
Create user account and add it to proper group
#useradd netflow #usermod -a -G apache netflow
Create directories which we will specify later in configuration file
#mkdir -p /data/nfsen #mkdir -p /var/www/html/nfsen
Now we will download latest nfdump and nfsen packages at this time nfdump-1.6.13.tar.gz and nfsen-1.3.6p1.tar.gz
#cd /opt #wget http://downloads.sourceforge.net/project/nfdump/stable/nfdump-1.6.13/nfdump-1.6.13.tar.gz #wget http://downloads.sourceforge.net/project/nfsen/stable/nfsen-1.3.6p1/nfsen-1.3.6p1.tar.gz
Configure iptables
In this tutorial we will disable original firewall that comes with CentOS 7 and install old good iptables.
#systemctl mask firewalld #systemctl stop firewalld #yum -y install iptables-services #systemctl enable iptables #systemctl start iptables #iptables -I INPUT -p tcp –dport 80 -j ACCEPT #iptables -I INPUT -s "destination ip" -p udp --dport 9666 -j ACCEPT
also add any additional ports that you need open
Start httpd service
#service httpd start
Install nfdump
Untar downloaded nfdump package
/opt#tar -zxvf nfdump-1.6.13.tar.gz /opt#cd nfdump-1.6.13
Compile nfdump
/opt/nfdump-1.6.13#./configure --enable-nfprofile --enable-nftrack --enable-sflow /opt/nfdump-1.6.13#make && sudo make install
Install and configure nfsen
Untar nfsen
/opt/#tar -zxvf nfsen-1.3.6p1.tar.gzls /opt/#cd nfsen-1.3.6p1
Copy nfsen example configuration file to etc directory
/opt/nfsen-1.3.6p1/cp etc/nfsen-dist.conf /opt/nfsen-1.3.6p1/etc/nfsen.conf
Now we need to edit this configuration file to make sure all variables are set correctly
#vi /etc/nfsen.conf
Make sure all data path variables are set correctly
$BASEDIR= "/data/nfsen"; $HTMLDIR = "/var/www/nfsen";
For CentOS based systems change
$WWWUSER = "www"; change to apache $WWWGROUP = "www"; change to apache $WWWUSER = "apache"; $WWWGROUP = "apache";
We will now run perl installation script to install nfsen
/opt/nfsen-1.3.6p1/#./install.pl etc/nfsen.conf
Press enter to accept default path. You may get Errors since we did not configure any flows at this point.
Lets now create a startup script for the service
#vi /etc/init.d/nfsen
#!/bin/bash
#
# chkconfig: - 50 50
# description: nfsen
DAEMON=/data/nfsen/bin/nfsen
case "$1" in
start)
$DAEMON start
;;
stop)
$DAEMON stop
;;
status)
$DAEMON status
;;
restart)
$DAEMON stop
sleep 1
$DAEMON start
;;
*)
echo "Usage: $0 {start|stop|status|restart}"
exit 1
;;
esac
exit 0
make sure the script is executable
#chmod +x /etc/init.d/nfsen
Start nfsen deamon
#/etc/init.d/./nfsen start
At this point you should be able to access nfsen at http://127.0.0.1/nfsen/nfsen.php
Configuration of Cisco router to export netflows
This is very basic example showing how to export netflows on Cisco 870 series Router.
On the Cisco router
conf#int fa04 ip route-cache flow show ip cache flow clear ip flow stats ip flow-export destination 92.168.2.47 2055
On nfsen system edit /etc/nfsen.conf file and make sure sources entry exists
%sources = (
'CiscoRouter' => { 'port' => '2055', 'col' => '#0000ff', 'type' => 'netflow' },
);
#data/nfsen/bin/nfsen reconfig
#/etc/init.d/nfsen restart
Softflowd configuration on Linux server
This is basic example of how to setup netflows on Linux based server
On Linux server:
yum install libtool automake autoconf python-devel yum install libpcap-devel wget http://softflowd.googlecode.com/files/softflowd-0.9.9.tar.gz tar -zxvf softflowd-0.9.9.tar.gz ./configure make make install #softflowd -i eth0 -n 192.168.2.47:9666
You may also add this line at the end of /etc/rc.d/rc.local file to make sure it will be started as system starts.
make sure that the software is actually working, check the softflow statistics after softflowd has been running for a few moments.
#softflowctl statistics
On nfsen workstation:
Add the followinf to /etc/nfsen.conf
%sources = (
'CiscoRouter' => { 'port' => '2055', 'col' => '#0000ff', 'type' => 'netflow' },
'LinuxServer' => { 'port' => '9666', 'col' => #088A08', 'type' => 'netflow' },
);
data/nfsen/bin/nfsen reconfig
/etc/init.d/nfsen restart
Troubleshooting
1. Install tcpdump and verify that flows are being send on specified port.
#yum install tcpdump #tcpdump -i eth0 port 9666
Make sure that you see traffic on this port from required host.
2. With nfdump you can read flow collection files from command line
cd /data/nfsen/profiles-data/live/LinuxServer/2014/12/13 nfdump -r "your file name"
3. Make sure is that your system data and php date set correctly. You may need to edit /etc/php.ini and adjust your date.timezone = "US/Eastern"
4. When you first visit your nfsen website you get error message Frontend – Backend version mismatch! You get this message the first time you connect because your browser doesn’t use the correct cookie/session id. After you navigate to a new page you get a cookie. This error can safely be ignored.
5. Make sure remote system, the one generating netflows has correct timezone and time set.
6. Troubleshoot nfcapd
First check running fcapd processes
#ps axo command | grep '[n]fcapd'
Check which ports nfcapd is listenoing on
#lsof -Pni | grep nfcapd
To test if nfcapd receiving data
#pidof nfcapd "port num" #strace -p "port num"
7. If you suspect there is a problem with nfcapd you can try runing it on Linux Server, You will need to install nfdump first.
#softflowd -i eth2 -n 127.0.0.1:9997 #mkdir test #nfcapd -w -D -l test -p 9997