This fall, comedian Drew Carey took over from Bob Barker as hosting The Price is Right, a show that has aired fairly steadily since 1956 in the U.S. with local versions produced in 30 other countries. While contestants on that show have to guess at the correct price for different products, when it comes to software, the "Right Price" is often zero: Linux runs on everything from mainframes to mobile phones; Snort is the most widely used Intrusion Detection System (IDS) software; and Open Office is taking a bite out of Microsoft's dominance of the office suite market.

Of course, a free download doesn't mean the product is without cost. There is still the expense of installing, maintaining and hosting the software. Many organizations, therefore, find it cheaper to pay for a supported version: RedHat and SuSE for Linux, Sourcefire for Snort and Sun Microsystems' StarOffice.

There are also numerous free products available for network management. In this article we focus on two that can be used to capture and analyze network traffic data coming from switches and routers: ntop and Scrutinizer.

Packet Parsing

ntop and Scrutinizer are both tools for accessing and analyzing NetFlow data. NetFlow was developed by Cisco Systems, Inc. as a part of its Internetwork Operating System (IOS) that comes with its routers and some of its switches. It helps customers monitor their networks, analyze traffic patterns, track usage and plan for expansion. The most recent release is version 9.

While traditional Simple Network Management Protocol (SNMP) shows bandwidth utilization, it doesn't provide the necessary insight into exactly what is traveling on the network. Further granularity is needed to apply QoS policies, determine bandwidth hogs, and identify the affect of new applications installed on the network. To achieve this, NetFlow examines the packets passing through a network interface in reference to seven attributes: IP source address, IP destination address, source port, destination port, Layer 3 protocol type, Class of Service and router or switch interface. All packets with the same attributes are considered part of the same flow, the bits and packets for that flow are tallied, and the data on completed flows is stored in a cache for export.

NetFlow requires two elements: a data generator and a data collector. The data generator is any device that is set up to collect and export NetFlow data. It is a push technology which will bundle the data from about 24 to 30 flows and send it off to the collector. UDP format is generally used to transport the data, however other formats are available.

To activate NetFlow on a router, type in:

#ip flow-export version 5

#ip flow-export destination <ip address> <port number>

#ip flow-export source <interface#>

Then, visit each physical interface and type in:
#ip route-cache flow

The data collector is a workstation or server with a database and analysis software installed which collects the data exported by the network devices and makes it available for analysis by the network administrators. Once the collector is configured to listen to the correct UDP port (default is 2055), it starts receiving the NetFlow data. NetFlow traffic is about 2% of the current bandwidth utilization and a 20MB interface generates about 1GB of raw data daily.

NetFlow is a proprietary Cisco format, but the Internet Engineering Task Force (IETF) is developing a standard based on NetFlow v.9 called Internet Protocol Flow Information eXport (IPFIX) which will work with devices using IOS as well as products from many other vendors. For the latest news on IPFIX and drafts of the protocol, go to the working group's website at www.ietf.org/html.charters/ipfix-charter.html

NetFlow is available on nearly all Cisco network devices except for the 3660 Multiservice Platform and the 2900, 3500 and 3750 switches. In addition to Cisco, commercial vendors providing NetFlow reporting applications include AdventNet, Arbor Networks, Fluke Networks, CA, HP, IBM and NetQoS.  

Staying ntop of the Data

One option for collecting the flow data is to use ntop (www.ntop.org), an open source network traffic probe developed by Luca Deri at the University of Pisa and released under the GPL. It runs on Unix (including Linux BSD, Solaris and Mac OSX) and 32-bit Windows platforms. In addition to supporting NetFlow and IPFIX, ntop also supports VoIP and sFlow (a hardware-based flow reporting solution) data. Traffic statistics are stored in Round Robin Databases (RRD) for long term analysis, and data is presented via a web interface.

Gary Gatten, senior network engineer for financial services advisors Waddell & Reed Services Company in Shawnee Mission, Kansas, uses ntop to monitor connections to the firm's 180 locations. Headquarters has 800 users and hundreds of servers on a 100/1000 Gb switched Ethernet network using Cisco Catalyst 4500 and 6500 switches. Each remote office has five to fifty users, two to four switches, and at least one router.

"We wanted something inexpensive that would run on inexpensive *nix based systems," says Gatten. "At the time most offices had dumb hubs, so acquiring the metrics necessary to troubleshoot complaints was nearly impossible."

When there was a serious problem, the company would ship a sniffer to the site, but this was a slow and impractical solution. Using ntop gives them rapid access to the data. He uses it at aggregation points and uses NetFlow whenever possible. "Focusing on aggregation points and 'provider' type hosts such as servers, routers and firewalls, we can see all the conversations on our LAN/WAN without having to touch a workstation," he says.

Initially Waddell installed ntop to address slow network complaints, primarily by monitoring health on Layer 2. As the offices upgraded to switched LANs, the application has evolved to looking at Layer 4+ to see who is doing what. When HP's Openview Network Node Manager (NNM) alerts administrators to an overloaded circuit, they start digging deeper.

"We then use ntop to see who is using all the bandwidth, if that usage is business related, and take appropriate action," Gatten says. "We try to control bandwidth pretty carefully with various forms of QoS, but with so many applications using TCP port 80, it's very difficult."

Scrutinizer Netflow Analyzer

Another free tool for analyzing flow technologies is Scrutinizer from Plixer International, Inc. of Sanford, Maine. Scrutinizer is a collector of NetFlow data as well as the other flow technologies sFlow and IPFIX. It also collects VoIP data from Avaya, Cisco, Nortel, Asterisk and other PBXes. Scrutinizer runs on Windows boxes (2000/XP/2003) with a minimum of 2Gb of RAM and 50 Gb available disk space for trial installations. Production environments have higher recommended hardware specifications.  It integrates with products from a number of other network management software products including Ipswitch, Inc.'s WhatsUp Gold, SolarWinds' Orion and Numara Software's Track-It, as well as Packeteer appliances.

Like ntop, Scrutinizer uses a browser interface. Data is shown covering intervals from five seconds up to a week. When viewing a graph, administrators can drag the cursor over a section to drill down further into that data. A network admin, for example, when looking at a weekly graph of top talkers on a particular connection, might spot a traffic peak on Wednesday morning at 3 A.M. - a time when the office is empty and traffic should be minimal. Dragging the mouse over that peak drills down into that time period and exposes that a particular machine was acting as a zombie, sending out spam.

The Oregon State Data Center (SDC) in Salem, Ore. - a recent consolidation of about a dozen large state government data centers - uses Scrutinizer to monitor its connections to 40,000 users at more than 800 sites around the state. The connections go from 56k modems up to 2.5 Gb OC-48 lines. The network uses about 900 Cisco routers and more than 1300 Cisco switches.

"None of the tools previously used by the state agencies could scale to SDC's size," says Network Management Technical Lead, Alison Wood. "One of our first requirements was a way to know, at a glance, the status of a remote site. Scrutinizer has an easily configurable feature that uses Google Maps to display all SDC router based on longitude and latitude."

The maps appear on a wall containing six 4' by 6' LCD displays with the sites showing as green if they are up or red if down. The key second feature was its ability to collect and report NetFlow statistics. But that was just the beginning.

"The SDC uses Scrutinizer in ways that we never intended," says Wood.

She says that the SDC's favorite feature of the software was the ability to create user accounts and map them to user sites. She gives the of the Department of Forestry requesting frequent bandwidth reports on the WAN links.

"The SDC created a map in Scrutinizer that shows all of the Forestry sites, illustrates their real-time connection speed and bandwidth usage, and displays it neatly on a map," Wood says. "Forestry technicians are able to log into Scrutinizer, see only their sites, and review and troubleshoot their sites. This tool allows the SDC to empower its customers and take the mystique out of what is going over that data link."

Seeking Support

ntop and the free version of Scrutinizer, as well as other free flow collectors, are good up to a point, but they are limited in their usefulness when compared to a commercial product. Just as many companies prefer to go with a commercial release of Linux, so do they want support for their free flow collection software. And both ntop and Scrutinizer have support available for a fee.

Padraig Houlahan, IT director for the Lowell Observatory in Flagstaff, Arizona, uses ntop to find network bottlenecks.

"It is a very powerful tool and gives you an insight into how things are going on the network," he says.

Nevertheless, he doesn't use the tool on a regular basis.

"In our particular configuration, it seemed to be crashing all the time, was unstable and took too much work to administer," Houlahan says.

Fortunately his network is fairly stable so he doesn't need to be running it all the time.

"I do wake it up even now and then if I need it, so it is still a useful tool, but a frustrating tool," he says. "You can't argue with the price; at the same time I wish it was more stable."

For those who need to get more out of ntop, the ntop organization offers fee-based services from the ntop developers. The ntop website also lists companies in seven countries offering on-site ntop support (www.ntop.org/consultancy.html).

With Scrutinizer, one of the main drawbacks of the free software is that it only stores the data for 24 hours. This is still useful for immediate debugging of overloaded connections, but doesn't one the longer range view needed for capacity planning, or to spot the source of recurring, intermittent problems. Watch manufacturer Timex Corp. of Middlebury, CT, for example, started out with the free version of Scrutinizer.

"I put it in place and it became my number one tool within a matter of hours, says Dave Edgecomb the company's Manager of Global Technical Operations. "Even though the free edition only keeps information for 24 hours and then refreshes, it was giving me more information than my [Network Instruments] Observer suite. It is the equivalent of a dashboard that gives me valuable information very quickly to resolve problems."

To get a longer view of the data, however, he is switching to one of the commercial editions of Scrutinizer.

"It is almost worth scrapping your current WAN view because you really don't want that any more," he says.

Drew Robb is a freelance writer specializing in IT. 

Forward this page to a friend

Your Email: *

Your Name: *

Send To: *

Enter multiple addresses on separate lines or separate them with commas.

Message Subject: (Your Name) has forwarded a page to you from NaSPA.com

Message Body: (Your Name) thought you would like to see this page from the NaSPA web site.

Your Personal Message: *

Your vote:

1

2

3

4

5

No votes yet