Taking a Deep Dive into Network Traffic
A term I often hear our customers say is that they use our LANGuardian product to “take a deep dive into network traffic“. When you hear something like ‘deep dive’ you could associate it with geeks in their Speedos taking a dive into a swimming pool. The reality is a lot more technical and maybe more boring; what they are trying to do is use network traffic as a data source to get to the root cause of network, security, application or user problems.
Client based traffic analysis
For a lot of network administrators the tool of choice may be Wireshark. It is excellent for taking a deep dive into network packets. I often use it to capture network traffic on my laptop and scroll through the packets to work out what traffic flows are present and see what packet payloads are associated with them.
The problem with this approach is that it can be very time consuming, this is especially so if you are dealing with high traffic volumes. Wireshark filters are useful but this is a foreign language to most people. Connect your laptop to a SPAN or mirror port and within minutes you could be dealing with a multi gigabyte packet capture file.
There is no doubt that tools like Wireshark or Microsoft Message Analyzer have their uses. However, if you want 24/7 traffic monitoring then you will need to look at a different solution.
Flow based traffic analysis
Many layer 3 type network devices like routers and some switches have flow export features. Standards include NetFlow, sFlow, JFlow and IPFIX. Typically a network device extracts certain information from the packet headers. This will include IP addresses and port information together with a total amount of data contained within the packet payloads. This flow information is then sent to a flow collector where its is processed and stored.
If we think of it as diving into a swimming pool, flow analysis is like getting your Speedos on and approaching the pool. You dip your toes in but that is it. You have an idea how cold the water is but it is not a deep dive.
Flow analysis is great for getting a top level view of what is happening on a network. Some flow technologies have moved towards sampled packet analysis. I am not a big fan of this due to the resource demands it puts on networking devices.
Going deep with Deep Packet Inspection
If you want to take a deep dive into network traffic, you need deep packet inspection. Technologies like this automate packet analysis so that you have 24/7 monitoring. Some solutions will store all packet data on disk (packet recorder) while others will extract certain payload data like website or file names (known as meta data).
Another feature of deep packet inspection tools is their ability to recognize applications based on packet payloads. Flow tools will make assumptions like all traffic on TCP port 80 is web but this is not always the case. Most firewalls available today include this functionality and it is vital in today’s world were so many applications are web based.
Traffic analysis tools that monitor traffic inside a network are getting more popular. The main driver for this is that IT managers want to get an insight into network activity so that they can increase security awareness. They also want historical reporting for seeing what happened at a particular point and time.
Before you make a decision on one you need to consider the following
- Do you need to record every packet or just capture important meta data. Unless you are monitoring a critical banking application or similar, meta data capture is recommended.
- Can the tool be deployed in remote data centers and provide a single console to monitor all activity.
- Don’t forget about virtual networks. Network packets can move around here and may never appear on the ‘wired’ network.
- Check if the tool supports username association. When you are dealing with LAN issues, it is very useful to be able to track activity back to actual users.
- Watch out for ease of use. Too many tools claim they can do deep packet inspection but are difficult to use. Ideally you want ‘management friendly’ graphics with drill down capabilities.
All of the solutions I mentioned above have their uses. Wireshark for client side diagnostics, flow tools for high level traffic reports, and deep packet inspection for taking a deep dive into network traffic. In some cases you may need all three, just make sure you don’t end up with the wrong solution if you only can pick one or two.