Webview Netflow Reporter

Geesh things change quickly in IT.

I was blown away by Webview Netflow Reporter four weeks ago but I have found that it has issues with actually displaying interfaces that are receiving flows visible in the GUI.

The good news is that there is an even better pair of open source netflow collection and reporting tools called nfDump and NfSen that work together to collect netflow data, display whatever you wish to see in a GUI and supports alerting as well.

We've just run it up at work and it has now replaced our commercial platform from Fluke.

Read more about nfSen in this post.

Need a netflow reporting platform but got no budget?
Problem solved!

This product will collect, draw and drill down on conversations to help you determine what traffic is using your expensive WAN links.

To get started, you'll need to build yourself a Linux box of VM, enable the LAN interface etc. and make sure you've got a working internet connection.

First, grab yourself a download link for a copy of Webview Netflow Reporter from http://wvnetflow.sourceforge.net/#download

Substitute the link where the brackets are on the line below and copy and paste down to but not including the validation steps section (this script is an updated copy of that provided at http://wvnetflow.sourceforge.net/INSTALL.centos)

wget (get link to 1.0.7d)

cd ~
gunzip -c wvnetflow-1.07d.tar.gz | tar -xf -
cd ~/wvnetflow-1.07d


yum --assumeyes install rsyslog gcc byacc libtool make patch zlib-devel httpd rrdtool-perl perl-GD perl-Net-DNS


wget -O - http://cpanmin.us --no-check-certificate | perl - --self-upgrade
cpanm Net::SNMP Spreadsheet::WriteExcel Net::Patricia

cd ~/wvnetflow-1.07d
wget http://iweb.dl.sourceforge.net/project/wvnetflow/flowd-sequence/cweinhold-flowd-sequence.tar.gz
gunzip -c cweinhold-flowd-sequence.tar.gz | tar -xf -
cd cweinhold-flowd-sequence
./configure
make install
mkdir -p /var/empty/dev
groupadd _flowd
useradd -g _flowd -c "flowd privsep" -d /var/empty _flowd

cd ~/wvnetflow-1.07d
wget https://flow-tools.googlecode.com/files/flow-tools-0.68.5.1.tar.bz2
bzcat flow-tools-0.68.5.1.tar.bz2 | tar -xf -
cd flow-tools-0.68.5.1/
patch -p1 <../optional-accessories/flow-tools-patches/patch.flow-tools.scan-and-hash
CC='gcc -fPIC' ./configure
make
make install

cd lib
ln -s /usr/local/flow-tools/lib/libft.a libft.a
cd ../contrib
wget http://net.doit.wisc.edu/~plonka/Cflow/Cflow-1.053.tar.gz
gunzip -c Cflow-1.053.tar.gz | tar -xf -
cd Cflow-1.053
perl Makefile.PL
make
make test
make install

sed -i.bak -e '/GLOBAL DIRECTIVES/i $AddUnixListenSocket /var/empty/dev/log\n' /etc/rsyslog.conf

cat <<EOT >/etc/rsyslog.d/40-flowd.conf
\$umask 0000
\$FileCreateMode 0644

:programname, isequal, "flowd" /var/log/flowd
:programname, isequal, "flowd" ~
EOT


chcon -t etc_t /etc/init.d/rsyslog


service rsyslog restart

cd ~/wvnetflow-1.07d
mkdir -p /opt/netflow/tmp /opt/netflow/data /opt/netflow/cache /opt/netflow/capture /usr/local/webview
cp -Rp flowage www utils /usr/local/webview
cp etc/webview.conf /etc
chmod 777 /usr/local/webview/www/flow/graphs

chcon -t httpd_sys_script_exec_t /usr/local/webview/www/flow/*.cgi

cp etc/flowd-2055.conf /usr/local/etc/
cp etc/init.d/flowd-centos /etc/init.d/flowd
chmod 755 /etc/init.d/flowd
ln -s /etc/init.d/flowd /etc/init.d/flowd-2055
chkconfig --add flowd-2055
service flowd-2055 start

iptables -I INPUT -p udp --dport 2055 -j ACCEPT
iptables -I INPUT -p tcp --dport 80 -j ACCEPT
service iptables save

crontab -l > /tmp/newcron
cat <<EOT >>/tmp/newcron

0 * * * * /usr/local/webview/utils/flow-expire-perl -E 10G -e 9000 -w /opt/netflow/capture/2055

*/5 * * * * /usr/local/webview/utils/flowd2ft 2055 >> /var/log/flowd2ft-2055.log 2>&1

#*/5 * * * * perl /usr/local/webview/flowage/flowage.pl > /tmp/flowage.stdout 2> /tmp/flowage.stderr

0 0 * * * find /opt/netflow/capture -name 'summary-*' -mtime +14 -exec rm -f {} \;

0  2 * * * find /opt/netflow/capture -name '*.rrd' -mtime +30 -exec rm -f {} \;
15 2 * * * find /opt/netflow/capture -depth -type d -empty -exec rmdir {} \;

#*/15 * * * * /usr/local/webview/flowage/monitor/monFlows.pl >> /var/log/monFlows.log 2>&1

0 0 1 * * /usr/local/bin/sudo mv -f /var/log/flow-expire.log /var/log/flow-expire.old
0 0 1 * * /usr/local/bin/sudo mv -f /var/log/flowd2ft-2055.log /var/log/flowd2ft-2055.old
0 0 1 * * /usr/local/bin/sudo mv -f /var/log/monFlows.log /var/log/monFlows.old
EOT
crontab /tmp/newcron


sed -i.bak -e'/UserDir: The name/ i\
Alias /webview "/usr/local/webview/www"\
\
<Directory /usr/local/webview/www>\
    Options Indexes Includes FollowSymLinks ExecCGI\
    order allow,deny\
    SetEnv no-gzip 1\
    allow from all\
</Directory>\
\
AddHandler cgi-script .cgi\
' /etc/httpd/conf/httpd.conf

service httpd restart


####### validation steps ########

#Edit: /usr/local/webview/flowage/flowage.cfg
#to set SNMP Read String

# If running CentOS, disable SELinux
# http://www.cyberciti.biz/faq/howto-turn-off-selinux/

# check if flowd is running
ps -fC flowd 

# check that flows are being received
ls -lR /dev/shm/

# check that flow files are being moved to the capture directory
ls -lR /opt/netflow/capture/

# view iptables rule list -- make sure port 2055 is seeing traffic
iptables -L INPUT -v

# make sure web server is running
service httpd status

# once you're sure you have flow data in the capture directory, run one of the web scripts
wget -O - 'http://127.0.0.1/webview/flow/adhoc.cgi?quiet=1&test=1&output=ascii'

# if that looks works (you see some IP addresses in the output), run flowage once from the CLI
/usr/local/webview/flowage/flowage.pl

# if that looks good (you see flow files being processed and rrd files be created), then uncomment
# the */5 and */15 flowage crontab entrys and you're all set!
# nano /tmp/newcron

ntop: Don't bother with CentOS 6 (or ntop)

Well I recently needed to look at alternative Netflow collectors of the free variety as our Fluke netflow collector has ridiculous license limitations.

I hadn't looked at ntop for a while and, well, I'll save you some time: Don't bother.

It hasn't improved in a long time.

The simple fact of the matter is nothing even touches Manage Engine Netflow Analyzer.

Now just something I noticed in my travels attempting to get ntop running on good old CentOS 64: It's virtually impossible to upgrade PERL on this distro.

Following one of the many guides out there such as:

http://cavepopo.hd.free.fr/wordpress/network/ntop-install-on-rhelcentos-6/

http://www.mikrotik-routeros.com/2012/03/installing-ntop-on-centos-6redhat-with-netflow-mikrotik-traffic-flow/

or

http://ryanwoon.wordpress.com/2011/08/20/install-ntop-in-centos-6/

resulted in the same errors message:
Please install python 2.6 or newer.


So I upgraded python to 3.3 using a variety of methods (yum install as well as wget / compile / tar / make / make install) and it simply won't upgrade.

Long story short, it just works on Ubuntu Server Edition.

And after ALL that you're greeted with one of the most lack lustre traffic monitoring platforms you've seen in a long time.

Oh well - hope I saved you some time :)