A SIEM Security Primer: Evolution and Next-Gen Capabilities
Security Event and Information Management (SIEM) systems are at the core of mature security teams. Learn SIEM basics and what’s next for SIEM technology.
SIEM is a tool that allows you to monitor your network traffic and provide real time analysis of security alerts generated by the applications. SIEM struggles are typical, that is why we have created this primer to explain why SIEM products are critical for advanced attack detection, to shed light on SIEM terminology, and to explore top SIEM tools and solutions. Learn how a SIEM can improve your information security capabilities.
In this post you will learn about:
- What is SIEM security?
- What is a SIEM and How Does it Work?
- SIEM Security Evolution
- The importance of SIEM
- What is the value of SIEM
- UEBA and SOAR in modern SIEM security
- Evaluating SIEM software
- 3 best practices for SIEM implementation
- Example of a Next-Gen SIEM with UEBA and SOAR Built In
What is SIEM Security?
A security information and event management (SIEM) system is the foundation of most security processes in the modern security operations center (SOC). A SIEM saves security analysts the effort of monitoring many different systems and brings together their vast amounts of log data to form a coherent picture.
SIEM security refers to the integration of SIEM with security tools, network monitoring tools, performance monitoring tools, critical servers and endpoints, and other IT systems. A SIEM also pulls in logs and event data, analyzes them, and generates alerts when it identifies activity that might be a security incident.
Figure 1: SIEM structure and SIEM security
What is a SIEM and How Does it Work?
A security information and event management (SIEM) solution is a tool you can use for centralized alerting, logging and compliance. SIEM tools can correlate collected data to provide context for alerts and events across your systems.
SIEM tools work by collecting and aggregating logs, reports and alerts from all of your security tools and solutions. These tools then present this information in a centralized location, improving visibility and speeding incident analysis and response.
SIEM solutions follow a cycle with three stages:
- Data collection—data logs are collected from devices, applications, systems and existing security tools.
- Data consolidation—the SIEM solution normalizes and categorizes data for analysis. Categorization can include user origins, credentials used, systems accessed and processes performed.
- Data analysis—categorized data is analyzed and compared to rules defining accepted behavior. If an event is deemed suspicious, an alert is sent to your security team.
SIEM Security Evolution
Analysts identify three generations of SIEM security capabilities and technologies:
- The first generation of SIEMs, introduced in 2005, combine log management and event management systems, which were previously separate. They are limited in the scale of data they can process and in the sophistication of alerts and visualizations they generate.
- The second generation of SIEMs was better equipped to handle big data—large volumes of historical logs. Such SIEMs can correlate historical log data with real time events and data from threat intelligence feeds.
- The third generation of SIEMs, proposed by Gartner in 2017, combine traditional SIEM capabilities with two new technologies. These are user and entity behavior analytics (UEBA), which uses machine learning to establish behavioral baselines of users or IT systems, and identifies anomalies. This includes security automation, orchestration and response (SOAR) which can help analysts quickly investigate incidents and activate security tools to automatically respond to an incident.
For more details on the evolution of SIEM and SIEM security, see our in-depth guide What is SIEM, which is part of our Complete Guide to SIEM.
Over the past two decades, SIEMs have proven themselves as a powerful and effective infrastructure for security teams. At the same time, SIEMs have been notoriously expensive, challenging to implement and use, and difficult to scale. Originally, SIEMs were only an option for large, mature security organizations.
These challenges have been addressed by newer generations of technology, which are easier to adopt and use, require less computing resources, and leverage low cost storage. SIEM security solutions are also offered as a service in the cloud, and via managed security service providers (MSSP), offering multiple deployment options that balance cost with ease of implementation.
The Importance of SIEM
Organizations use SIEM technologies for:
- Log management and retention
- Continuous security monitoring and incident response
- Case management
- Policy enforcement and violations
- Comply with governmental requirements, such as HIPAA, PII, NERC, SOX, COBIT 5, PCI, FISMA
Why else is it important to have a SIEM? If you are on the receiving end of a breach and are asked to explain what happened, you want to have an answer ready.
Many organizations implement SIEM to protect sensitive data and to have proof of this protection process as a failed audit can have dramatic consequences, including loss of employees and business and large fines.
What is the Value of SIEM?
Security event correlation
SIEM analyzes the sum of all data from its log management feature for signs of a threat infiltration or data breach. For example, a failed login is generally not a concern. However, a failed login from one user on applications across the IT environment could signify a threat. You can only see the relationship between the data of these applications via SIEM facilities.
SIEM facilities include connecting to threat intelligence feeds, including the feeds of third-parties, and solution providers. Isolated feeds typically retain unique threat data, using information from many feeds can help you make optimal use of your solution.
Your SIEM should continuously update your team about possible threats, including dashboards updates, text alerts, or email alerts. If your solution does not update your team they could miss a threat, letting it reside on your server.
Value of Next-Gen SIEM Solutions
SIEM is now an established technology, and the next generation of SIEMs have new proficiencies:
User and entity behavior analytics (UEBA) – modern SIEMs surpass correlations by taking advantage of machine learning and AI methods to identify and explore typical and atypical human behavior. This insight can help organizations discover malicious activity, insider threats, and fraud.
Security orchestration and automation (SOAR) – next-gen SIEMs now include automated incident response systems. For example, the SIEM could identify an alert for ransomware and respond by automatically implementing containment steps on affected systems, before the hacker encrypts the data.
UEBA in Modern SIEM Security
User and entity behavior analytics (UEBA) is a new category of security solutions that can identify behavioral baselines and spot anomalies which might indicate security incidents. UEBA can detect security incidents that other tools can’t see, because they rely on predefined patterns or static correlation rules. Third-generation SIEM solutions come with UEBA capabilities built in.
Here are some common use cases of SIEMs with UEBA technology:
- Malicious insider—A user account with privileged access to IT systems that is abused by the account owner for personal gain. Insider attacks can be devastating and are invisible to most security tools. UEBA establishes a baseline for each user’s behavior and can detect suspicious events that might indicate malicious intent.
- Compromised insider—An attacker who gains control of a user account and uses it to perform reconnaissance, plan, or actually attack organizational systems. UEBA can identify that the user account is behaving differently from normal and alert security staff.
- Incident and alert prioritization (alert triage)—SIEM security alerts are a huge burden on security analysts and alert fatigue is a challenge. UEBA can help reduce the burden of prioritizing alerts. It does this by combining alerts and signals from many tools, ranking alerts and incidents based on the amount of anomalous behavior (their risk score), and adding layers of contextual data about the organization, for example, services or user accounts that access sensitive data.
- Data loss prevention (DLP)—DLP tools, like traditional SIEMs, create a high volume of alerts about every unusual event related to an organization’s sensitive data. UEBA tools can prioritize and consolidate DLP alerts by calculating risk scores using data from multiple tools, indicating which events represent anomalous behavior. UEBA can also place a DLP alert on an incident timeline, helping validate and investigate incidents.
For more details, see our in-depth guide to User and Entity Behavior Analytics, part of our Complete Guide to SIEM.
SOAR in Modern SIEM Security
Security orchestration, automation and response (SOAR) systems, another new technology bundled with third-generation SIEM solutions, have the following key capabilities:
- Orchestration—SOAR integrates with other security solutions, allowing them to retrieve data and also proactively perform actions. For example, it can investigate whether an email sender has a bad reputation by using a DNS tool to confirm the origin of the message.
- Automation—SOAR enables users to define security playbooks, which are codified workflows of security operations. When a known type of security incident occurs, the playbook can be activated and mitigation action can be taken automatically, such as scanning a file identified as malware and detonating it in a sandbox.
- Incident management and collaboration—When a SIEM generates a security alert, the SOAR component of the SIEM can add contextual information and evidence to help analysts investigate the issue, and organize this information in an incident timeline to make it easier to understand. They also allow analysts to collaborate and add insights or additional data that they discover as part of their investigation.
For more details, see our in-depth guide to Incident Response Automation and Security Orchestration with SOAR, part of our Complete Guide to SIEM.
Evaluating SIEM Software
We recommend the following stages in evaluating a SIEM solution:
1. Next-Gen SIEM Features
Third-generation SIEM security solutions offer the most value and also lower implementation and operation costs. Check if a solution offers:
- UEBA—advanced analytics to determine behavioral anomalies
- SOAR—automation and orchestration of incident response
- Dashboards and visualizations
- Flexible searching, querying, and data exploration
- Long term data retention and unlimited scalability
- Threat hunting interface
2. Open Source vs. Commercial and In-House vs. Hosted SIEM
Consider which type of SIEM security solution is most suitable for your organization:
- Open source vs. commercial—Open source tools offer lower upfront costs but have higher ongoing maintenance costs and more limited capabilities.
- Build vs. buy—Some organizations are creating SIEM solutions using open source tools such as the ELK stack (Elasticsearch, Logstash and Kibana). This requires major investments to implement, maintain, tune, and integrate security content, because ELK is primarily a log management infrastructure and not a security system.
- In-house vs. managed—You can choose between four deployment models: (1) self-hosted and self-managed (the traditional model); (2) hosted on the cloud but managed by in-house security staff; (3) self-hosted but managed by a combination of in-house security staff and a managed security service provider (MSSP); (4) SIEM as a service in the cloud with local security management.
3. Evaluating Total Cost of Ownership (TCO)
A SIEM is a complex piece of security infrastructure that can be costly to procure and operate. Generally speaking, a SIEM involves the following budget items:
- CAPEX budget items—Licenses, development, training, hardware and storage.
- OPEX budget items—Security analysts to review SIEM alerts, IT maintenance, integration with new IT systems, and storage costs.
Here are a few tips for accurately estimating the TCO of a SIEM implementation:
- Licensing—Check the licensing model used by available SIEM solutions, typically licensed based on ingestion volumes or velocities. Some newer entrants to the market offer user-based pricing, which may put a cap on licensing costs.
- Hardware costs and sizing—Calculate the number of events per day, using an estimate of normal event load and peak event load. The event volume will determine the number and type of servers required to deploy the SIEM (for in-house deployments).
- Storage costs—Even in cloud or managed SIEM deployments, you will typically need to pay for storage as you scale up and pay extra for historic data retention.
- In-house analysts—The largest operating expense of a SIEM is analyst time. Determine if you have the skilled manpower to review and investigate SIEM alerts, and if not, consider outsourcing to an MSSP. More modern SIEMs, which include UEBA and SOAR technology, may have lower operating costs.
Best Practices for SIEM Implementation
1. Get to Know Your Data and How it Can Work for You
This involves understanding the size, behavior, frequency, and type of your log data prior to deployment. You should know what data is available and where your data is coming from, including systems, switches and routers, and how it is being transferred.
Also specify your reason for implementing a SIEM system and the aim of your project. Will your SIEM strategy support daily operations? Is it for security and maintaining logs for threat detection purposes? Or is it for compliance adherence?
2. Establish the Rules Needed for Compliance
Establish which industry standards and regulations are applicable and how the SIEM can help with compliance audits and reports. Define baseline correlation rules to capture basic compliance requirements, but don’t stop there. Leverage next-gen SIEM technology, in particular UEBA, for better threat detection and easier ongoing maintenance.
3. Augment Correlation Rules with UEBA
Traditional SIEM correlation rules only look for what you tell them to look for. It’s important to define rules that capture basic attack scenarios, but in today’s security environment rules are not enough to capture all relevant threats. In addition, correlation rules result in a high number of false positives which puts a strain on security analysts.
We recommend defining correlation rules and monitoring to see if they create too many false positives. If they do, consider removing the rule and using UEBA to establish a behavioral baseline of the relevant system operations, and identify any significant anomaly from that baseline. For security scenarios that cannot be easily described by rules, such as insider threats, use UEBA to begin with.
Example of a Next-Gen SIEM with Built-In UEBA and SOAR
Exabeam is a third-generation SIEM platform that is easy to implement and use. It includes advanced functionality per the revised Gartner SIEM model:
- Advanced Analytics and Forensic Analysis—Threat identification with behavioral analysis based on machine learning, dynamically grouping of peers and entities to identify suspicious individuals, and lateral movement detection.
- Data Exploration, Reporting and Retention—Unlimited log data retention with flat pricing, leveraging modern data lake technology, with context-aware log parsing that helps security analysts quickly find what they need.
- Threat Hunting—Empowering analysts to actively seek out threats using a point-and-click threat hunting interface, making it possible to build rules and queries using natural language with no SQL or NLP processing.
- Incident Response and SOC Automation—A centralized approach to incident response, gathering data from hundreds of tools and orchestrating a response to different types of incidents via security playbooks. Exabeam can automate investigations, containment, and mitigation workflows.
Figure 2: Exabeam’s malware playbook
Learn more about SIEMs
- What is SIEM?
- SIEM Architecture
- Events and Logs
- SIEM Use Cases
- SIEM Analytics
- The SOC, SecOps and SIEM
- Incident Response and Automation
Learn More About SIEM Security
7 Open Source SIEMs: Features vs. Limitations
Security information and event management (SIEM) systems used to be for large organizations only, but they are increasingly adopted by medium-size and even small organizations. Open Source SIEMs are compelling for new adopters because of their low licensing cost and growing feature set. Which open source SIEMs are out there, and how do they compare to the traditional enterprise offerings?
SIEM Solutions: How They Work and Why You Need Them
SIEM solutions provide a consolidated view of security events, making them an essential component of cybersecurity. This article is relevant for anyone who does not fully understand how SIEM solutions work and why they are such a crucial component of cybersecurity. We will discuss the main advantages of using SIEM as well as some of the top SIEM vendors and why their products are unique.
Combating Cyber Attacks With SOAR
With the race between cyberattackers and cyber defense solutions evolving, many organizations are struggling to adapt to the growing threat from advanced and numerous threats. Many organizations are now turning to automated solutions like SOAR to help them identify and respond to security threats. But what is SOAR? Read on to learn what SOAR stands for, what it can do, how it differs from SIEM, and why you should use them together.
Read more: Combating Cyber Attacks With SOAR
Threat Intelligence: Threat Feeds, Tools, and Challenges
Threat intelligence is knowledge about security threats, threat actors, exploits, malware, vulnerabilities, and compromise indicators (according to SANS). The more aware you are of possible threats, and the more details you have about their mode of operations, capabilities, infrastructure, motives, and goals, the better you are equipped to defend your organization.
Battling Cyber Threats Using Next-Gen SIEM and Threat Intelligence
While SIEMs are central for SOC cybersecurity—collecting logs and data from multiple network sources for evaluation, analysis, and correlation of network events for threat detection—SIEMs are often not enough. With the growing sophistication of organized, modern cyber attackers and their highly-targeted techniques, organizations can be left with serious vulnerabilities using SIEM alone. To best identify and stop cyber attackers, organizations need a full arsenal of tools that will help them understand how attackers think, work, and what they are after.
Threat Intelligence Feeds: Keeping Ahead of the Attacker
With the ever-growing specter of cybersecurity threats, organizations need to understand how attackers can exploit vulnerabilities in their systems so they can prepare a threat mitigation strategy. However, there are several options for creating a threat intelligence solution, and it can be difficult to manage your cybersecurity needs. This article will help you navigate the available options, and understand why you need a solution that can generate and analyze threat intelligence feeds.
How a Threat Intelligence Platform Can Help You
The rise of advanced persistent threats (APTs) and the large volumes of data involved in threat detection are making the job of security analysts harder every day. The work of an information security engineer often involves sifting manually through hundreds of security alerts to find real threats. The huge amounts of data collected by any given organization nowadays make it almost impossible for usually, understaffed security teams to keep on top of threats. While security teams use resources and software tools to overcome these challenges, often the new tools they wish to introduce don’t integrate easily into the existing software infrastructure of their organization.
See our Additional Guides on Information Security
For more in-depth guides on additional information security topics, see below:
Cyber Security Threats Guide
Cyber security threats are intentional and malicious efforts by an organization or an individual to breach the systems of another organization or individual.
See top articles in our cyber security threats guide
- Information Security Threats and Tools for Addressing Them
- Drive By Downloads: What They Are and How to Avoid Them
- Cyber Crime: Types, Examples, and What Your Business Can Do
User and entity behavior analytics Guide
UEBA stands for User and Entity Behavior Analytics which is a category of cybersecurity tools that analyze user behavior, and apply advanced analytics to detect anomalies.
See top articles in our User and Entity Behavior Analytics guide
- What Is UEBA and Why It Should Be an Essential Part of Your Incident Response
- User Behavior Analytics (UBA/UEBA): The Key to Uncovering Insider and Unknown Security Threats
- Behavioral Profiling: The Foundation of Modern Security Analytics
Insider Threat Guide
An insider threat is a malicious activity against an organization that comes from users with legitimate access to an organization’s network, applications or databases.
See top articles in our insider threat guide
- Fighting Insider Threats with Data Science
- Insider Threat Indicators: Finding the Enemy Within
- How to Find Malicious Insiders: Tackling Insider Threats Using Behavioral Indicators
Security Operations Centers Guide
A security operations center (SOC) is traditionally a physical facility with an organization, which houses an information security team.
See top articles in our security operations center guide
- How to Build a Security Operations Center for Small Companies
- Security Operations Center Roles and Responsibilities
- SecOps: 7 Steps to : Taking DevOps One Step Further
DLP is an approach that seeks to protect business information. It prevents end-users from moving key information outside the network.
See top articles in our DLP guide
- Data Loss Prevention Policy Template
- Data Loss Prevention Tools
- Security Breaches: What You Need to Know
Incident Response Guide
Incident response is an approach to handling security breaches.
See top articles in our incident response guide
- The Complete Guide to CSIRT Organization: How to Build an Incident Response Team
- How to Quickly Deploy an Effective Incident Response Policy
- Incident Response Plan 101: How to Build One, Templates and Examples
Regulatory Compliance Guide
See top articles in our regulatory compliance guide