NetFlow with ntopng and nProbe

What is NetFlow?

NetFlow is a technology originally developed by Cisco that aggregates information about web traffic (called a flow) based on packet headers and routing information.

Today, flows are leveraged to gain powerful insights into the behavior and operation of a computer network.


There are two primary components to NetFlow:

1) An "exporter" that is responsible for watching traffic and aggregating the information.
2) A "collector" which is in charge of taking in all of that information.

ntopng

ntopng is a platform built on modern technologies to allow the consumption of massive amounts of network flow data from any source that can export it, Cisco, Juniper, Palo Alto, Brocade, etc...

It can act as both an Exporter and a Collector, and with the help of nProbe, we can even take in remote flow data, even behind a NAT firewall.

 

nProbe

There are three primary modes in which nProbe can behave.
(We will be configuring it in "Probe" mode for the demo.)

 

Probe Mode

nprobe_probe
The nProbe acts as an exporter and listens for packets on local interfaces and forwards flows to a remote collector.

 

 

Collector Mode
nprobe_flow_collector
The nProbe acts as an endpoint collector and stores data on local disks. It can simultaneously forward data to a remote collector.

 

 

Proxy Mode
nprobe_proxy
The nProbe acts as a NetFlow proxy with the ability to convert flows between versions.

 

Communication between ntopng and nProbe

ntopng and nProbe utilize ZeroMQ. This gives us greater flexibility with our NetFlow data. 
There are two methods that we can use to perform the "nProbe Import" above.

"Pull"

This is a "subscriber" model that allows a client to poll information only on specific flows,
cutting down on overhead.

"Push"

This mode behaves more classically where the nProbe will automatically forward data to a specified endpoint without regard for "subscribers".

 

Installing ntopng and nProbe

Getting Started

In this article we will cover the basics of setting up ntopng using nProbe as a remote NetFlow exporter.
The diagram found below will be used as a reference as we go through the installation.

(For this demo, we are running CentOS 7 on the "ntopng" and "nProbe" hosts, and no ACLs or NAT rules are configured on the LAN. The IP addresses specified for the hosts are on a separate physical link than the "Port Mirror".)

 

Untitled_Diagram.png

 

Binary Packages

We will need to add two repositories to get set up. EPEL, and ntop.

yum install epel-release

curl -o /etc/yum/repos.d/ntop.repo http://packages.ntop.org/centos-stable/ntop.repo

 

Starting ntopng

ntopng -i ens33 -i tcp://*:5556c

By default, ntopng has the ability to promiscuously monitor traffic being sent to its local interfaces.
If no interfaces are specified, it will listen to all. 

In our example we have our local interface (ens33) set up on a mirrored switchport to capture all traffic passing through the switch.

The second interface we specify (tcp://*:5556c) will be our NetFlow collector address.
In this example, the host will create a TCP socket listening on all address and on port 5556.
The "c" at the end denotes to ntopng that it is a collector.

 

 Starting nProbe

 

We start nProbe listening on interface "ens33" (our other port mirror) and specify our ZMQ endpoint (tcp://192.168.100.20:5556) which is our ntopng system we previously configured.

By default nProbe would start as a ZMQ server with "subscribers" (pull) model.
The "--zmq-probe-mode" flag denotes that we should send flow data to our endpoint regardless of subscription status.

 

 
Verifying It Works


In the top right corner of the web interface, our collector socket we started ntopng with will show up in the "Interfaces" drop-down.

If we select the interface we can start seeing some metrics and confirm that flow data is incoming.

 

Conclusion

At this point, we have a single nProbe system that is creating flow data on a remote network segment and exporting it to our ntopng collector for visualization. This example could easily be expanded to allow  the monitoring of many more remote network segments allowing the administration team unparalleled levels of access to metrics about an organization's web usage. If you'd like to know more, you can reach out to us at sales@TruePathTechnologies.com or check out our webpage: http://truepathtechnologies.com/ntop/

 

Additional resources: 

http://truepathtechnologies.com/ntop/

http://www.ntop.org/products/netflow/nprobe/

http://packages.ntop.org/

http://www.cisco.com/c/en/us/products/collateral/ios-nx-os-software/ios-netflow/prod_white_paper0900aecd80406232.html

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.