SOC Processes and Best Practices in a DevSecOps World

A security operations center (SOC) is an organizational unit with security staff and supporting personnel, who monitor enterprise systems, defend against security breaches, and proactively identify and mitigate security risks.

While in the past, a SOC was a dedicated team working in a physically separate facility, today organizations are merging development, security and operations teams into a unified model known as DevSecOps. Does the SOC still have a role in a DevSecOps organization? What will SOC processes look like in a modern, collaborative environment? Read on to find out. 

In this article, you will learn:

• 4 essential SOC processes
  • Alert triage
  • Prioritization
  • Remediation and recovery
  • Post mortem and reporting
• What is SecOps?
• How DevSecOps processes are transforming the SOC
  • The transition from a siloed SOC to DevSecOps
  • How the SOC is adapting to DevSecOps
• 6 security operations center best practices
  • Detect threats through all stages of an attack
  • Investigate all alerts to ensure nothing is overlooked
  • Gather forensic evidence for investigation and remediation
  • Leverage security automation
  • Use threat intelligence
  • Combine data across silos

4 essential SOC processes

Here are four critical processes used on a daily basis in most security operations centers.

Alert triage

A SOC collects, correlates, and analyzes log data, and analysts must have an efficient way to remove noise and identify impactful security events. Signals of an attack may be found in user activity, system events, security alerts,  firewall logs, and other data sources. In addition, combinations of events in a certain sequence and in a certain pattern can indicate incidents that require attention. 

The key to success at this stage is to have a quick way to categorize each alert. This allows analysts to prioritize and escalate critical alerts that require further investigation. Typically, Level 1 SOC analysts review alerts with the highest severity, and if they believe any of them require further investigation or represent a real breach, they escalate the issue to a Tier 2 Security Analyst.

Prioritization

In the modern threat landscape, risks are high, attacks move fast, and attackers are becoming more sophisticated and are armed with a growing range of automated tools. On the other hand, the resources available to protect the organization are limited. This requires SOC teams to focus on the events that could have the greatest impact on business operations. 

Prioritizing events relies on three pillars:

1. Knowing which assets are the most important
2. Understanding which types of events can threaten business continuity for those assets
3. Identifying threats that are likely to impact business continuity, and where that impact will be severe

SOC analysts must identify and respond first to any activity indicating that an attacker has breached the environment. Common examples of high priority events are the installation of a rootkit or backdoors, network communications between the local network and known malicious IP addresses, and compromised administrative accounts.

Remediation and recovery

The faster SOC analysts detect and respond to an incident, the more likely they are to contain damage and prevent similar attacks in the future. When investigating a case, analysts make a lot of decisions, including what level of remediation is necessary, whether the incident will be subject to criminal investigation, and whether it needs to be officially reported to comply with regulations.

It is crucial for analysts to work closely with SOC leaders and the management team, communicate clearly, and keep track of everything they do during the containment and recovery from a breach. A remediation process typically includes:

1. Re-image affected systems and restore data from backup, apply patches and updates
2. Reconfigure access controls, deleting or resetting compromised accounts, updating access control lists (ACLs), VPN access, and firewall rules
3. Ensure previously affected hosts are monitored to prevent re-infection
4. Scan affected systems for vulnerabilities and check for misconfigurations, including but not limited to the specific issues that led to the attack

Some part of this process may be handed over to other groups in the IT Ops or DevOps organization. Still, the SOC team must be closely involved, because it is their responsibility to ensure systems return to a trusted state after an attack.

Post mortem and reporting

It is always best to find and fix vulnerabilities before attackers can exploit them to gain access to your environment. The best way to do this is to conduct a periodic vulnerability assessment and review the results of these reports in detail. These assessments identify technical vulnerabilities, not procedural vulnerabilities, so you must additionally review your processes and address vulnerabilities in the SOC process itself.

For SOC team members, performing vulnerability scans and generating compliance reports are some of the most common reporting activities. In addition, SOC team members should periodically review the SOC process with auditors, to ensure compliance with policies and determine how to improve the performance and efficiency of the SOC team.

What is SecOps?

Security Operations (SecOps) is a collaboration between security and IT operations teams, where security and operations staff assume joint ownership and responsibility for security concerns. It is a set of SOC processes, practices and tools which can help organizations meet security goals more efficiently.

Before SecOps

In the past, operations and security teams had conflicting goals. Operations was responsible for setting up systems to achieve uptime and performance goals. Security was responsible for verifying a checklist of regulatory or compliance requirements, closing security holes and putting defenses in place.

In this environment, security was a burden—perceived as something that slows down operations and creates overhead. But in reality, security is part of the requirements of every IT system, just like uptime, performance or basic functionality.

After SecOps

SecOps combines operations and security teams into one organization. Security is “shifting left”—instead of coming in at the end of the process, it is present at the beginning, when requirements are stated and systems are designed. Instead of having ops set up a system, then having security come in to secure it, systems are built from the get go with security in mind.

Towards DevSecOps

SecOps has additional implications in organizations which practice DevOps—joining development and operations teams into one group with shared responsibility for IT systems. In this environment, SecOps involves even broader cooperation—between security, ops and software development teams. This is known as DevSecOps. It shifts security even further left—baking security into systems from the first iteration of development.

How DevSecOps processes are transforming the SOC

Security Operations Center processes used to be completely isolated from other parts of the organization. Developers would build systems, IT operations would run them, and security were responsible for securing them. Today it is understood that joining these three functions into one organization—with joint responsibility over security—can improve security and create major operational efficiencies.

The Transition from a Siloed SOC to DevSecOps

How the SOC is adapting to DevSecOps

Here are a few ways the modern SOC can foster a DevSecOps mentality:

• Analysts can continuously inform operations staff about threats to the organization’s systems, and actual incidents
• Analysts can proactively seek out security gaps and work with operations to close them
• Operations can come to the SOC for guidance about security implications of systems, components, vendors or changes

Here are a few ways a SOC can integrate its processes with dev and IT:

• Creating a distributed SOC with DevOps members — DevOps teams can help with incident response due to their deep knowledge of IT systems, and can learn from security staff about threats and critical vulnerabilities.
• Pairing threat hunters with DevOps team leaders — instead of discovering a threat and reporting it upwards, threat hunters can work directly with dev or ops teams to close the security gap at its source.
• Opening the SOC for guidance and advice — anyone doing work that has a security impact should have an easy path to reach the SOC and consult with the organization’s top security experts.
• Creating security centers of excellence — the SOC can work with selected dev and operations groups to implement security best practices, and then showcase these successes to the entire organization to promote SecOps practices.

6 security operations center best practices

The following best practices can help you operate your SOC and defend against threats more effectively in a modern threat environment. 

1. Detect threats through all stages of an attack

To cope with the increasing number and complexity of cyber threats, organizations have implemented security solutions that deal with specific vulnerabilities or attack vectors. Attackers in response have created sophisticated responses, using multiple techniques.

Point solutions working by themselves cannot identify the relationship between a series of events. To stop an attacker from penetrating security, security operations must:

• Deploy prevention and detection approaches throughout the entire attack chain, the IT environment, and every attack vector.
• Design the technologies to function together and communicate information.

For an example of a security tool that provides automated incident timelines, aggregating data across multiple security tools, users and devices, see Exabeam Threat Hunter.

2. Investigate all alerts to ensure nothing is overlooked

A copious number of alerts was an early driver for SIEM. SIEM systems created correlation rules to group similar events into alerts, this helped teams deal with the tens of thousands of events isolated daily. Today, organizations state that even with correlation, there are too many alerts to investigate, which leaves the organization open to risk.

Organizations need to develop solutions that not only group alerts but automatically investigate and validate them. They should try to limit the number of events that must be reviewed by human analysts.

3. Gather forensic evidence for investigation and remediation

To investigate alerts, security teams require in-depth endpoint and network activity data. This is made available by forensics solutions. However, forensics tools, specifically on the network, are known to be time-consuming and complex to use.

Organizations should find solutions for forensics that are simple to use and automated. It is important to adopt solutions that proactively combine forensic evidence into investigation procedures. An organization should also convey the results in relation to the alert or lead the data validates.

4. Leverage security automation

To more effectively analyze larger numbers of security events, identify incidents and mitigate against them, organizations can leverage security automation technologies that complement the work of skilled security analysts.

SOC capabilities depend on technological capabilities — technology can simplify and speed up security processes by collecting and aggregating data, implementing protections against security threats, and automatically responding when breaches occur. When security teams have access to data sources and tools that minimize false positives, analysts can maximize the time they spend on investigating actual incidents.

5. Use threat intelligence

When combined with threat intelligence data generated within an organization, external security data provides the SOC team with important insights into vulnerabilities and threats. Intelligence from external sources covers a wide range of data, including vulnerability alerts, incident reports, signature updates, news feeds, and threat briefs. The SOC can leverage security monitoring tools that offer integrated threat intelligence.

6. Combine data across silos

The SOC team needs to combine data from all security silos to effectively detect and respond to threats. They should receive and consolidate data from:

• The network – traffic analysis, URLs, hashes, connection details
• Endpoints – information revealed by vulnerability scanners, security intelligence feeds, intrusion prevention (IPS) and detection (IDS) systems, and endpoint protection systems
• Operating systems – reviewing logs with security significance and monitoring for anomalous processes
• Firewalls – monitoring logs and events on external-facing firewalls
• Cloud systems – today the cloud is an inseparable part of the corporate network, and cloud resources must be monitored for security misconfigurations and anomalous activity

Learn more in our article about extended detection and response (XDR)

Learn more about security operations centers

See our additional guides about key SOC topics:

• Complete Guide to Security Operations Centers
• The SOC Team
• SOC and SIEM