Getting visibility of what is happening on your Internet connection
How to gain visibility of Internet usage
When it comes to understanding what is happening on your network, one of the most common questions I get is how you can find a data source to understand what is happening on an Internet connection. The most common data sources that people mention are
- Reports from an ISP
- Reports available directly from firewalls or routers
- SNMP data
- Log files
- Packet capture
The easiest reports to get are the ones from an ISP or directly from network devices. The image below shows an example of these.
While it does give us some idea as to what is happening, it lacks detail as to what is causing those peaks. The same problem will exist for any application which uses SNMP (simple network management protocol) as a data source. You will get an alert that there is excessive traffic on your Internet connection but you will lack the detail you need to troubleshoot why this is happening.
This then brings us on to gathering flow records like NetFlow. NetFlow and other flow standards allow you to see what systems are connecting to what and how much data is been exchanged. This is very useful information as we can now break down those peaks into what system is connecting to what.
The problem with this is that while this is a view of what system is connecting to what, it is hard to read. Users do not connect to IP addresses. They use applications and connect to services like YouTube. For that reason flow based tools are not a good option for monitoring Internet activity. The problem is even worse if you use proxy servers. Flow records will just show IP addresses connecting to the proxy server IP address and at the other side you have the IP of the proxy connecting to IP addresses outside your network.
Lets now look at two other sources of data; log files and packet capture. Server log files are inappropriate for gathering usability data. They are meant to provide server administrators with data about the behaviour of the server, not the behaviour of the user. The log file is a flat file containing technical information about requests for files or websites on the server. Log files can also be easily overwritten and need to be pulled back to a SIEM for indexing and storage.
The final data source is packet capture. The wonderful world of bits and bytes where only the geeks dare to travel. The thing is that modern deep packet inspection tools make the job of processing network packets really easy. You can download them and within minutes you can start to drill down and see what is actually moving around your network.
The following image is a good example of this. Here we can see two users downloading an OVA file from netfort.com. Really easy to read and it shows exactly what happened. No issues with trying to resolve IP addresses and no hours spent looking through packet capture files.
Finally, I spoke to someone during the week and when I mentioned that you could monitor Internet traffic with a SPAN or mirror port he reported that he had no managed switch. In most cases you need a managed switch to setup a SPAN\mirror port. However, if you do not have a managed swich you can always deploy a cheap network TAP. These devices allow you to get a copy of traffic going in and out of a network connection.
In my case the network manager had a Cisco ASA 5505 deployed. This is actually a hybrid device with an 8 port switch and firewall features. To configure a SPAN port on an ASA 5505 you need to use the following commands.
ASA(config)# int ethernet 0/0
ASA(config-if)# description Firewall Connection
ASA(config)# int ethernet 0/1
ASA(config-if)# description Deep Packet Inspection Tool
ASA(config-if)# switchport monitor ethernet 0/0 both
If you need to check if your switch supports SPAN or mirror ports there is a good guide at this link.
So that is it for this post. If you really need to find out what is happening on your Internet connection, look inside the network packets!