NetFort Advertising

Do you really need ‘Artificial Intelligence’ for actionable alerts

Alert image

Using Traffic Analysis as a Data Source

As we have mentioned numerous times in our blogs, Network Traffic Analysis or (DPI) Deep Packet Inspection is a very flexible technology. It can be used for many use cases including continuous monitoring of user and device activity, reporting, forensics, analytics and of course troubleshooting of everyday  problems. One of the benefits of using a DPI engine to analyse network traffic flows, is the rich application specific detail and context, metadata that can be extracted and presented in real time or stored for forensics. Data ideal for many IT security and operational use cases.

DPI can sometimes be seen as a ‘complex and expensive technology’ only suitable for large enterprise, but not with the latest engines as found in the NetFort LANGuardian. The basic principle of the LANGuardian engine is to get the engine to do all the ‘heavy lifting’, reassembly, analysis, alerting thus making it very easy to use and read, ideal for all skill levels across organisations of all sizes with minimum training.

Actionable Alerts That Our Customers Requested

Recently we have been asked by our customers to generate real time alerts on various network and user activities that are critical to them. Examples, in the customers own words include:

  • US Manufacturing company
    • ‘Alert if a user or device generates more than x GB of data over a given time?’
    • ‘Alert if certain file types are detected (e.g. mkv files)? ‘
  • Large EU University
    • ‘Alert when a machine on our network is maliciously scanning 100,000’s of IP addresses across
      the globe. ‘
  • EU Online retail company
    • ‘Any internal ip address making a connection to an external ip where the connection (TCP/UDP) was not preceded by a DNS query that returned the external ip’
  • EU Government organisation
    • ‘Alert on any web accesses not via the proxy server’
  • US City Council
    • ‘I’m trying to figure out the syntax for a rule to detect when the BitTorrent protocol is detected’
    • Oct 2016 ‘ Detect SMB1 traffic Is there a way to detect SMB1 traffic? Microsoft recommends to stop using it so I’d like to see if it’s being used in our network.’
  • US Law firm
    • ‘Alert if a lawyer uploads huge files to our shared server within a short period of time using up all our space’

Some seem very obvious, simple but on closer examination, most make sense. Also, it is interesting to note that most customers do not request that many, maybe because they are already flooded with false positives and find it almost impossible to actually spot the real actionable alerts.

Machine Learning

I had a chat with a customer last week who purchased a pretty well known ‘machine learning’  based network security products 6 months ago, when he mentioned the product name, I was very curious and asked how it was going. ‘Nothing yet, 6 months of false positives, but you know, it is still learning’. So now not alone have they invested a lot of time and money in purchasing and implementing a product but it is also costing them time wise every day, as it giving them even more false positives to investigate!

Actually, a small number of our customers who requested the alerts included in the list above have recently implemented some expensive ‘Machine learning’ based security products. We started discussing it here internally and it got us wondering about the massive hype by vendors, analysts etc, around machine learning with respect to security. What is really driving it ? The lack of skilled security analysts is definitely one factor, big data another, but another one is surely the current set of overly complex and expensive security products ? And maybe he venture capitalists who have invested huge amounts of money in companies developing this technology, many of whom are struggling with sales ?

Developing Our Own Alerting Engine

We are putting huge focus on the usability of our alert engine, make sure it is as easy as possible to define the rules that generate real actionable alerts, not false positives, the alerts important to the user, the organisation, the business.

Of course, sometimes the simple and best ones are not that easy to implement. For example, as in the case of a lack of a DNS query require context/state and some understanding of the protocol in use in order to generate an alert. As mentioned by one of our engineers, some are also somewhat vague and require more detail. It may also be that some do not require an instant alerts, a simple email sent to the administrator each morning may suffice.

It will take time to get right, some tuning, knowledge of the network etc. Ease of use, readable data, is a must otherwise it will never work. These are basics some security vendors simply do not pay enough attention to but instead spend a lot of time and money on graphics and web interfaces designed by gamers, dark constellations which look fantastic but when you start to look at the detail, looking for actionable intelligence, you start thinking what is this really telling me ?

There are many common and critical threats or ‘bad’ network and user activities that do not require sophisticated artificial intelligence or machine learning.  Most organisations do not have the resources to monitor various dashboards to actually try and detect suspicious activity in real time,  but simply want a real alert with some readable context and data to understand what the alert is actually telling them.

Where to Start

Is it not common sense, start small, work the basics. Use a network traffic analysis for example to monitor internal activity and get the visibility you need to understand what is happening on your network. Modify your ‘active’ systems for example your firewall, to get rid of everything that could widen your attack surface and then add alerts, one by one, to ensure you are immediately notified the next time.  Use forums, blogs, your own network to keep in touch and build and update your own alert set. Add them one by one, you will be amazed with the size of your list after a few months and the lack of false positives.