Rules vs. Behavioral Heuristics vs. Data Mining vs. Machine Learning

By on Feb 11, 2016

This blog was written by Barbara Kay.

Are you wandering a maze of analytics bingo? That’s understandable. Here’s a short history of the evolution of analytics, explaining why we keep inventing new forms, and including examples based on McAfee technologies.

Most of security policy historically has been binary rules-based, requiring a specific certain condition: Firewall rules (yes/no), port rules. These rules help reduce clutter based on known good and known bad events. They remain important, but not sufficient. DAT files keep rules up-to-date in AV, but other types of rules require ongoing tweaking and maintenance or they become irrelevant. This heavy operational burden has spawned invention of new analytics techniques.

Correlation rules help us factor more data and conditions into the decision (if A and B and C exist together in a certain condition, fire an alert or action). Correlations essentially create a rule based on a pattern. This expands what we can know for sure. Correlation rules are part of the core McAfee® Enterprise Security Manager (SIEM product), and we offer them in default sets as part of content feeds that can be customized and aggregated to help customers filter and direct their attention to events that matter to their own security and risk priorities. Threat intelligence from many sources can be ingested via STIX and TAXII into McAfee Enterprise Security Manager and (via STIX only) to TIE to enrich correlations, investigations, and decisions.

For many years, heuristics and behavioral analytics have supplemented these rules with more subtle and fine-grained interpretations and algorithms that trade off complete certainty for speed. They look for more complex and incomplete patterns of events based on experience. We have used these techniques in AV and IPS for years to handle polymorphic and emerging threats. The sophistication of the patterns we look for keeps increasing in parallel with attack complexity, malware complexity, and evolving tactics, techniques, and procedures (TTPs) of adversaries.

Risk-based correlation turns the optics inside out. Instead of looking at what is happening generically, we look at what is happening to and around a specific thing or situation that customers decide they care about (perhaps a service or application, group of user, or data type (PII)). This dynamic correlation establishes a baseline, sets a risk score, and monitors for anomalies that merit attention. It focuses resources to pinpoint events and patterns that match customer security, risk, and business priorities. This analysis can help identify areas of insider threat, both intentional and accidental, resources under attack, and past events that went unnoticed. This feature is part of the Advanced Correlation Engine for McAfee Enterprise Security Manager.

Dynamic reputational decisions help these analytics factor in what we know or suspect at a given time about a file/IP address/sender/certificate/event collection/user/time/device. McAfee Global Threat Intelligence (GTI) provides a cloud-based reputational lookup and McAfee Threat Intelligence Exchange has integrated the ability to capture, manage, and generate local threat intelligence. It concentrates on the reputations of malware analyzed by McAfee Advanced Threat Defense, application reputation, locally generated certificates, and new occurrences of files. Most of our products, including McAfee Enterprise Security Manager and McAfee Advanced Threat Defense, can consume and/or generate intelligence to enrich their decisions and activation of this service should be a default for preventative controls.

Data mining (Big Data) applies these ideas across large data sets, applying algorithms to look for bigger patterns. We use big data mining as part of our McAfee Global Threat Intelligence service and the other cloud analytics services being developed today. That “behind the scenes” data mining improves the accuracy of our scoring and interpretation, without any extra effort on your part. For that, we also enable data mining through McAfee Enterprise Security Manager’s highly scalable and responsive proprietary database as well as via Hadoop support.

Machine learning is a way for algorithms to get smarter based on what they observe. They use modeling and make data-driven predictions about an event. As we look at our Threat Defense Lifecycle, or the Gartner version, machine learning is one way for the system to feed what it learns back into protection, detection, and correction processes. It is important because it alleviates the manual rules maintenance and decision making that slows many of the previous types of analytics in this list. McAfee applies machine learning already for reputational analysis and has technologies that use these techniques in consumer products. Machine learning is a core element of our evolving portfolio of analysis products and services.

What all of these techniques have in common is that they work to reduce uncertainty and guide action. We help customers use these techniques where they make the greatest impact. Using them intelligently enables speed. It’s why our McAfee Enterprise Security Manager can enable real-time visibility and actionable intelligence. When we filter out what we know is good or bad, and interpret out what is likely good or bad, we can zero in on the unknown and potentially bad to improve resource allocation (spend your time and people where they matter) and accuracy (make the right decision). High accuracy is critical as we lead the industry to adopt automation. Customers must be confident that an automated workflow or decision or response will do more good than harm.

This set of incredibly capable analytics fed by a broad set of data (including threat and contextual intelligence) makes a difference whether you are worried about external attacks, insider threats, compliance violations, or modeling future risk. That’s why it’s so important to optimize the use of analytics as throughout the threat defense lifecycle.

To continue learning more about this topic, read this brief, Operationalizing Threat Intelligence.

About the Author

McAfee Enterprise

McAfee offers industry-leading cybersecurity solutions for all business and enterprise needs. See our blog to stay up-to-date with the latest security trends

Read more posts from McAfee Enterprise

Subscribe to McAfee Securing Tomorrow Blogs