5 Aspects to Consider When Evaluating SIEM Solutions

The SIEM category is quite mature; all Magic-Quadrant-Leader products are more than a decade old. In fact, the youngest product is 14 years old. When these products were in their prime, design requirements were different: an enterprise-class product might be expected to store 50 TB of logs; correlation rules were considered a major advance over signatures for detection; searches were judged on speed and it was acceptable to require complex search syntax; finally, the hard work of investigation and response were mostly out of scope for a SIEM.

Today, all of those requirements are outdated, and the buyer of a new SIEM must evaluate products using different factors or else be stuck with a system that doesn’t deliver value. Below are five of the most important aspects to consider when evaluating SIEM solutions:

• Cost of Collection – Log management, i.e. data collection, storage, and indexing, is a fundamental component of any SIEM solution. Most vendors license their log management products by the byte, ie. the more you collect, the more it costs. Years ago, this was seen a simple and effective option for customers. However, as data volumes have grown and customers may easily be generating hundreds of TB per week, cost-based log management has become a budget sinkhole. The rise of open source big data systems, specifically Elasticsearch and Hadoop, have changed the nature of security data collection. A new set of vendors has arisen that use these technologies to build high scale, high speed log systems that don’t charge by the byte.

Consider: For a large organization, log management costs can easily reach $1M per year. Buyers should evaluate SIEM offerings that are not licensed by the byte. The savings can be considerable and can pay for new capabilities in other areas.

• Ability to Analyze – Static (i.e. pre-built) correlation rules were once an improvement over signature-based threat detection. Correlation was seen as a better way to handle subtle combinations of actions that indicated a threat. While it was challenging to edit and create rules in response to new attacks, a fully staffed organization could handle the effort. Modern attacks change so often that static correlation rules are no longer effective. The use of machine learning to perform behavioral analytics brings new detection abilities to a SIEM. Behavioral analytics is much better suited to assessing risk of credential-driven activities.

Consider: Static correlation requires the rule creator to guess – with perfect accuracy – at the types of attacks that may come. Behavioral analytics provides a way to deal with shades of grey. Buyers should evaluate SIEM offerings that include integrated behavioral analytics.

• Complex Searching and Hunting – Threat hunting uses iterative searching and pivoting to find users, entities, or actions that meet some specific set of conditions. Obviously, all SIEMs support some level of searching; all SIEM log managers have some form of natural language search. However, analysts often need to perform contextual searches that are impossible to do with log search languages. For example, how would you search for “find me all the users who came in over the VPN from a country for the first time, then accessed a server for the first time, then my anti-malware system flagged new malware on that server”? The context, e.g. knowing that this user has never come in over the VPN from that specific country before, doesn’t exist in the log data and may be impossible to express in the search language. New capabilities exist in some SIEMs to enable proactive threat hunting and complex searching via point and click, enabling better detection on the network.

Consider: Threat hunting is more powerful than simple searching. Buyers should evaluate a SIEM’s ability to enable proactive threat hunting within the user console.

• Expertise Assistance: Investigation Timeline – SIEMs have typically been used for threat detection, and fall short in investigation and response. A fundamental step in an incident investigation is the creation of a timeline. The timeline shows, from initial access to completion, every action taken within an incident. This may involve multiple identities, machines, IP addresses, etc. It likely requires attribution of an IP to a host to an identity. These steps can take days of work. Modern SIEMs create incident timelines automatically by stitching together the events and identity information linked to an incident.

Consider: Some SIEMs can positively attribute hosts to IPs to identities and can produce coherent timelines of all sessions linked to an incident. Buyers should evaluate a SIEM’s ability to produce automatic incident timelines with minimal analyst effort.

• Ninja in a Bottle: Automatic Response – SIEMs can be hard to operate, and the shortage of security expertise available in the market is made worse when talent can’t be found. The rise of orchestration and automation response systems happened as an antidote to hiring shortages in the cybersecurity industry. Capabilities typically include the ability to create a new playbook in response to some threat (e.g. a phishing response playbook) or to ship with out of the box playbooks. These capture best practices and enable junior or new analysts to perform better and more quickly in an investigation.

Consider: Incident response automation can dramatically shorten the time to shut down a threat. Buyers should evaluate a SIEM’s ability to automate existing playbooks and a firm’s response processes.

There are many factors to consider when evaluating a SIEM; these are only a handful. However, they are critical to choosing a SIEM that isn’t overly expensive, or too difficult to use, or incapable of detecting modern threats.