Network Intelligence FAQ

Questions are listed by category. Follow the hyperlink from a question to see the complete question and answer. If you cannot find what you are looking for try performing a text search with your browser. If you still cannot find the information you want, please drop us an E-Mail with your question.

Features

1. We run a small company, and I suspect the connection to our ISP is congested. We do not have access to either end router. How do we know if the link really is overloaded? >>answer
2. Can you reset roots of an AS-path tree to achieve a different view or perspective? >>answer
3. How does the central server collect the updates from the collectors? Is this performed in bulk or in real-time? >>answer

Getting started

1. What is the quickest way to see what the software can do? >>answer
2. How many devices should I use per collector? How many collectors per server? >>answer
3. What are the requirements for the different nodes hardware wise? >>answer
4. How does Network Intelligence get a current copy of the BGP table from the routers? >>answer
5. Why is it necessary to run the software as root when re-exporting the Netflow Exports? >>answer
6. When I attempt to start the server using '/opt/ni/bin/networkintelligenced -i' I get an 'invalid or missing licence error'. >>answer
7. When I attempt to start the server using '/opt/ni/bin/networkintelligenced -i' I get error on line xxx in file nid.conf near the token '{'. >>answer
8. How do I configure Netflow Exports on my routers? >>answer
9. I can log in successfully with the client software, but I don't seem to be able to edit anything. >>answer
10. I get the message 'Could not find interface name for router xxx ifIndex xxx'. >>answer
11. Network Intelligence complains about not being able to write out data. The message is "network_db_save: Error writing out network representation data!" >>answer
12. Our visualisation team is interested in walrus graphs (caida.org). What is the difference between walrus and Network Intelligence? >>answer

Day to day running

1. What does the error "network_traffic: Could not find source router core2b-ctn for this traffic!" mean? >>answer
2. I find that the application (collector, server or client) is crashing on a regular basis. Is there a fix? >>answer

Security

1. What sort of security issues do I need to be aware of when running Network Intelligence? What about SNMP requests, and where does all this information go? >>answer

Q. We run a small company, and I suspect the connection to our ISP is congested. We do not have access to either end router. How do we know if the link really is overloaded?

A. I see your problem. The thing is, without access to the router configuration it is very difficult to know if the circuit between your company and your upstream provider is overloaded. The only way I can think of would be to introduce a box that generates traffic and can measure packet loss, and possibly round-trip times of ping packets to the outside world.

I would suggest trying to get access to the router. It really is a much better solution. The 1600, depending on the version of IOS it is running may well support Netflow Exports. That means you can then run Network Intelligence. Network Intelligence tells you a lot about traffic patterns, and really is overkill for what I think you want to do. Instead by using SNMP counters gathered from the router you can measure the traffic to and from your network. This could be graphed with MRTG or a similar package. This would provide enough information to be able to say whether the link is overloaded. I would suggest talking with your Internet provider.

Q. Can you reset roots of an AS-path tree to achieve a different view or perspective?

A. It's certainly possible to supply a BGP routing table with a different root. It's doubtful however that anything useful could be obtained from doing this. As the statistics gathered are for your network, you can only view traffic flows from the perspective of your network.

Q. How does the central server collect the updates from the collectors? Is this performed in bulk or in real-time?

A. This is a tunable variable set in the central server. It's typically "near real-time". Somewhere between 30 seconds and a couple of minutes are good starting points. It has to be fast enough to be able to view changes and trends easily, yet long enough to be able to perform some worthwhile aggregation of the stats.

Q. What is the quickest way to see what the software can do?

A. Feel free to use the Gadgets Software server for a demonstration. The only thing you might find is someone else online at the same time you are. It is however a good way to experience the software. While setting up the central server, collector software, and routers to produce Netflow Exports is actually pretty simple, the logistics of carrying it out in a large network can cause major delays.

Q. How many devices should I use per collector? How many collectors per server?

A. This is a difficult question to answer adequately. Typically a few routers per collector - say one collector per POP. Only one central server is required.

The difficulty in providing an exact answer comes from the fact that there are many variables in the equation. Each router may produce a varying number of NFEs depending on timeouts, table sizes, number of interfaces, speeds, and the traffic passing through the router. The collector software is able to vary the aggregation performed, trading off bandwidth against timeliness of the data. Likewise tuning variables at the server end have a significant effect on storage performance and size requirements.

There are other more subtle issues affecting performance. For example, the collectors are optionally able to re-transmit the Netflow Exports to other devices. This means you can run Network Intelligence while still running other software that gathers Netflow Exports. As you can imagine, this will have a performance impact if this feature is enabled.

Q. What are the requirements for the different nodes hardware wise?

Collectors:
CPU and memory are the bottlenecks for the collector. Stats are kept and aggregated in memory for speed (no disk access). Looking at one of our production machines, an AMD K6-350 w/256MB RAM, it's handling 1000 flows/second, re-exporting the Netflow Exports, running other Netflow collection software at the same time and it's still very lightly loaded!

Server:
CPU and disk speed are the bottlenecks here. A fast Pentium III or Athlon are recommended. A few GB of disk is nice. More if you plan to keep the stats for a long time (many months). 256 or 512MB RAM should suffice. Larger networks would benefit from a good caching disk controller.

Clients:
Fast graphics with hardware acceleration is recommended. A machine that plays games well will work well with the Network Intelligence client.

Q. How does Network Intelligence get a current copy of the BGP table from the routers?

A. It doesn't, at least not directly. It reads the BGP information from a local file. Typically the local file is created by an automated telnet command that runs on a machine 'near' a BGP router. The telnet command connects to the router and retrieves the table. scp is typically used between the machine and the Network Intelligence server. Network Intelligence regularly checks to see if the file has changed, and when it has, it re-reads it. We chose this method due to the flexibiliy it offers, particularly when dealing with firewalls. It's easy to arrange for ssh access through a firewall, and hence copying a file from a remote machine to the central server is straightforward.

Q. Why is it necessary to run the software as root when re-exporting the Netflow Exports?

A. Exported UDP packets have the source IP address spoofed. This requires root privileges, and as far as I'm aware it's not possible to set up the socket state then change userid because of the connectionless nature of UDP.

Q. When I attempt to start the server using '/opt/ni/bin/networkintelligenced -i' I get an 'invalid or missing licence error'.

A. This error should only occur if Network Intelligence is trying to read a nid.conf file that does not contain a correctly formed licence line. Pretty much as the error says. Run Network Intelligence again with debug (-d) to see if the nid.conf file it is looking for matches the one you think it should.

Another possibility is that the licence in the nid.conf file really is corrupt or garbled in some way. Try copying it again from the original provided to you via E-Mail. Also check to make sure it is entered on a single line.

Q. When I attempt to start the server using '/opt/ni/bin/networkintelligenced -i' I get error on line xxx in file nid.conf near the token '{'.

A. It's possible you have omitted the AS from the end of the previous line. The default example file shows the correct format:

# Format: collector <collector_name> <collector_IP_address> <AS> # { # router <router_name> <router_IP_address> # router <router_name> <router_IP_address> # (etc.) # } # collector drone1 192.168.0.51 2 { router ams1 192.168.10.127 }

Q. How do I configure Netflow Exports on my routers?

A. Network Intelligence uses the Netflow Exports produced by many different kinds of routers. To see if a particular router is supported, check out the supported routers page. To configure a Cisco router, please refer to the Cisco web site at www.cisco.com.

Q. I can log in successfully with the client software, but I don't seem to be able to edit anything.

A. When using the access_control program to maintain the list of users, make sure you use the -l option when you wish to create users with administration level access. For example

[martin@butzi martin]$ access_control -a -f access_file -u test -l ADMIN

One you have an admin level user you will be able to build your network representation. If you still find you do not have admin level, check that the access_file you are looking at and editing, is in fact the same one that Network Intelligence is using - check the /etc/opt/ni/nid.conf file. For example, there should be a line that refers to your access file.

password_file = /etc/opt/ni/access_file

Q. I get the message 'Could not find interface name for router xxx ifIndex xxx'.

A. Such an error indicates that your snmp definitions file is incorrectly configured. Remember you must have lines defining the relationship between router interface names and ifIndex numbers. If these are missing, you would certainly receive the above error. A brief sample of the file might look like the following.

ams1;ser4;1
ams2;ser4;1
ams3;ser2;1
Paris_1;ser3;1
Geneva_1;ser2;1

Q. Network Intelligence complains about not being able to write out data. The message is "network_db_save: Error writing out network representation data!"

A. This is because there is no representation. Import or build some routers and try again. The message will disappear.

Q. Our visualisation team is interested in walrus graphs (caida.org). What is the difference between walrus and Network Intelligence?

A. I have not used Walrus directly, but from what I gather it is purely a visualisation application. Network Intelligence, while the client is mainly a visualisation engine, does a lot more. Much of the work is hidden behind the scenes and includes the aggregation and filtering of statistics from the routers, through to population in a database. This in itself is a tricky process due to the large volume of statistics. Network Intelligence also lets you interact with the network being modelled. You can add, remove, modify circuits, watch graphs of traffic flow variations. Much more detail is provided in that you can go up to a router, modify the interface parameters, and see the effect immediately. The more simplistic nature of the Walrus graph makes it more suitable for displaying large numbers of nodes. Certainly more than Network Intelligence, which provides more detail on each node.

Q. What does the error "network_traffic: Could not find source router core2b-ctn for this traffic!" mean?

A. The stats being gathered say that some traffic is entering your network at a router called core2b-ctn, however there is no such router in the network representation. You should build a router with this name or import a suitable representation of that router into your model.

Q. I find that the application (collector, server or client) is crashing on a regular basis. Is there a fix?

A. Calls to pthread_create will crash with glibc2.0, so upgrade to 2.0.1 or later. You can usually find out what version of glibc you are running by looking at the /lib/libc* files, or by entering the command

rpm -q glibc

Q. What sort of security issues do I need to be aware of when running Network Intelligence? What about SNMP requests, and where does all this information go?

A. In large networks security is always an issue. Network Intelligence doesn't work with SNMP requests, but Netflow Exports. In order to generate these statistics the routers must be configured to export the statistics to a particular IP address. This address is the address of a Network Intelligence collector. This router to collector connection is unsecure. The router just sends plain vanilla UDP packets. This is not really an issue however as the collectors are also on your network, and typically very close to the router generating the statistics. Next the collectors pass an aggregated statistic to the central Network Intelligence server, also located on your network. This connection is authenticated, but not encrypted. So nobody should be able to impersonate the central server (it uses CHAP authentication), but if they have a sniffer on your network, they can eavesdrop. Likewise, for the client to central server connection, the connection is authenticated using CHAP, but it is not encrypted. So if you run the client over the Internet it's possible (though of course unlikely) that someone on the Internet can watch the packets and potentially 'see' your network setup. If you wish to be paranoid, you could limit use of the client to only on your network. Then it comes down to whether or not someone has a sniffer in your network.