Thursday, May 21, 2009

Collecting NetFlow data

In the NetFlow world, a NetFlow exporter sends flow data to a NetFlow collector. The exporter is usually a router, the collector is usually a Unix server of some sort.

First, set up your router to export flow information:

ip flow-cache timeout active 2
mls flow ip full
mls flow ipx destination
mls nde sender
mls nde interface
mls nde flow include protocol tcp
ip flow-export source GigabitEthernet1/1
ip flow-export version 5 origin-as
ip flow-export destination X.X.X.X 2055

Where X.X.X.X is the address of your NetFlow collector, and GigabitEthernet1/1 is the router’s interface on that subnet. (This was taken from a 7600 router, you may not need the NDE stuff if you’re on a different platform)

Then, on each interface you want to capture flows for,

ip route-cache flow

You can check on the status of the export with

ROUTER#show ip flow export
Flow export is enabled
Exporting flows to X.X.X.X (2055)
Exporting using source interface GigabitEthernet1/1
Version 5 flow records, origin-as
235556663 flows exported in 7945727 udp datagrams
0 flows failed due to lack of export packet
743 export packets were sent up to process level
0 export packets were dropped due to no fib
18425 export packets were dropped due to adjacency issues
0 export packets were dropped due to fragmentation failures
0 export packets were dropped due to encapsulation fixup failures
0 export packets were dropped enqueuing for the RP
0 export packets were dropped due to IPC rate limiting

You can immediately see some statistics now that you have NetFlow enabled:

#show ip cache flow
IP packet size distribution (4086M total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.001 .627 .032 .012 .020 .019 .085 .009 .001 .002 .003 .005 .006 .006 .006
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.005 .004 .005 .066 .079 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 4456704 bytes
417 active, 65119 inactive, 235561367 added
132171494 ager polls, 0 flow alloc failures
Active flows timeout in 2 minutes
Inactive flows timeout in 15 seconds
last clearing of statistics never
Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
-------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
TCP-Telnet 12352 0.0 24 44 0.0 3.9 13.3
TCP-FTP 50507 0.0 1 55 0.0 0.7 14.4
TCP-FTPD 18867 0.0 1 499 0.0 0.5 15.0
TCP-WWW 158177053 36.8 17 186 627.8 3.3 8.9
TCP-SMTP 139330 0.0 1 135 0.0 0.0 15.4
TCP-X 23 0.0 2 222 0.0 1.8 9.4
TCP-BGP 2 0.0 1 64 0.0 0.0 15.7
TCP-NNTP 3 0.0 1 56 0.0 0.0 11.0
TCP-other 17276962 4.0 21 318 85.9 3.1 8.8
UDP-DNS 2866156 0.6 1 68 0.8 0.5 15.4
UDP-NTP 2082119 0.4 1 84 0.4 0.0 15.4
UDP-TFTP 137 0.0 5 49 0.0 20.4 15.5
UDP-Frag 3796 0.0 26195 1394 23.1 20.5 14.7
UDP-other 48352973 11.2 15 275 173.6 10.8 14.8
ICMP 3302490 0.7 6 165 5.0 6.5 14.8
GRE 1844456 0.4 38 137 16.7 116.5 1.1
IP-other 1433724 0.3 53 52 17.8 111.4 2.5
Total: 235560950 54.8 17 240 951.5 6.4 10.3

To collect the flows, install the flow-tools package, with either

yum install flow-tools

or whatever your distribution uses (apt-get install flow-tools), or install from source.

The flow-capture utility is the one that is used to write the flows to disk. It must be configured with the port (2055 in our case), and a location to write the flows to. In CentOS/RedHat/Fedora, this is done in /etc/sysconfig/flow-capture.

OPTIONS="-n 287 -N 0 -w /var/flow-tools -S 5 0/0/2055"
  • -n 287: 287 files per day, or one file every 5 minutes. I recommend doing this instead of the default 15 minutes so that you have more real time access to your data, and some tools depend on this reporting interval.
  • -N 0: Don’t nest the files. All the flow files will be in one directory instead of one per day.
  • -w /var/flow-tools: Write to this directory
  • -S 5: Syslog a message every 5 minutes with the collection statistics
  • 0/0/2055: listen on all interfaces to all exporters on port 2055

You may also want to configure something like tmpwatch in cron to clean up files (/usr/sbin/tmpwatch 720 /var/flow-tools) to only keep the last month or whatever you want. On a pipe that’s used 100-200MB/sec, you can expect at least 10G of data to be logged.

Start flow-capture (service flow-capture start), and look for files in /var/flow-tools.

The files are binary, so you can’t look at them directly. To have a look at what’s there:

# flow-cat /var/flow-tools/ft-v05.2008-12-22.080500-0600 | flow-print | head
srcIP dstIP prot srcPort dstPort octets packets
x.x.x.105 x.x.x.151 6 4511 80 744 6
x.x.x.105 x.x.x.151 6 4512 80 985 12
x.x.x.105 x.x.x.151 6 4514 80 784 7
x.x.x.105 x.x.x.185 6 4516 80 985 6
x.x.x.105 x.x.x.52 6 4517 80 1744 7
x.x.x.105 x.x.x.41 6 4518 80 850 5
x.x.x.115 x.x.x.255 17 138 138 229 1
x.x.x.252 x.x.x.62 6 2727 80 40 1
x.x.x.105 x.x.x.27 6 4521 80 2221 22

The fields should be fairly self explanatory. The -f parameter to flow-print allows you to print out new data.

Source :http://ccnprecertification.com

Network Search By Bilal

Custom Search