Looking back at the evolution of SIEM

If you're in cybersecurity, you likely know that SIEM stands for Security Information and Event Management (SIEM), is pronounced "sim" (or "seem" if you're in Europe) and that SIEM systems help security teams detect and respond to threats, manage incident response, and stay compliant. Over the last 20+ years, the SIEM market has had quite an evolution and growth explosion.

Today, SIEM accounts for approximately $4 billion of total cybersecurity spend and is expected to increase to $6.24 billion by 2027. This is easy to understand as SIEM has evolved into the data store for cybersecurity data which has been exploding as the volume of data and number of alerts is growing exponentially.

According to Ponemon Institute, the average number of cybersecurity products a company uses is 45. Some vendors claim Fortune 2000 companies have upwards of 130 tools. Each of these is generating both log files as well as alerts. But before we go into where the SIEM market goes from here, let's first take a look back at how SIEM has evolved.

Phase 1: The first SIEMs took in data and served up alerts

In the early part of the century, the first wave of SIEM vendors were the likes of ArcSight (now owned by Micro Focus) and QRadar (now owned by IBM). These early SIEMs married both log files (raw data) and security alerts (summarised events). Back then, it was about ingesting data and kicking off alerts from all the cybersecurity products that were being used –– mostly host- and network-based intrusion detection devices (ISS et al), network tools and firewalls (Check Point, Cisco et al). Endpoint and anti-virus software would come a little later.

Most of what a SIEM could really do back then was get data in, aggregate it, and send alerts to security teams. They were also used for data retention and compliance.

The most prevalent first- and second-generation SIEMs also came with very basic correlation engines, the best they knew how to do at that time. They had the ability to build correlation rules and say, "If I see X, Y and Z, then open a case in our ticketing system and send an alert to the security team".

But on-premises processing power against "unstructured" data was still quite slow, so it could take eons to query your essentially raw data and get any semblance of an answer about the root cause of an alert, security incident or otherwise.

Then the data got big

There still wasn't nearly as much data as there is today. What was being generated back then was easily parked in a database –– usually Oracle or DB2 –– and behind the scenes. With time though, enterprises continued their digital journey, and the data began to explode in volume — but all of this data was still being forced inside rigid databases.

Eventually, structured databases could not keep up with the needs of IT or security teams. They couldn't keep up with the volume, variety or velocity of the data coming at them.

Early SIEM vendors also couldn't keep up as structured databases were clearly not able to adapt — and writing new parsers to ingest new log sources took weeks or months.

Phase 2: Splunk entered the market, making search and access easy

Splunk was founded in 2003 as essentially the first-ever flexible and powerful store and search engine for big data. It introduced indexing which can search any kind of raw data – from structured to unstructured – and quickly transformed the data into searchable events.

The company's technology was a breakthrough because it made it so much easier for organisations to ingest, search, store, visualise and get insights from all of their growing data.

When they entered the SIEM market later, it changed the game for original SIEM vendors. Its first appearance as a Leader on the Gartner MQ for SIEM was in 2012. While the company's bread and butter was mostly IT operations use cases up until that point, once they introduced a SIEM, the indexing and "schema at read" capabilities allowed security teams to store, search and drill down into their data far more efficiently to get much faster SOC answers too.

Splunk's architecture was far more effective than legacy vendors, and the company had somewhat of a market lead for many years.

Phase 3: SIEM met UEBA, aka anomaly detection

At this point, the world was beginning to see more zero-day attacks: computer software vulnerabilities previously unknown until adversaries find and take advantage of them. The SIEM industry had to keep up by trying to make even more sense of the data that was being stored. Eventually, User and Entity Behavior Analytics (UEBA) was created to apply more cyber intelligence to this problem.

Most vendors were still trying to bolt some form of UEBA on top of their SIEM, but for UEBA to be at its best for anomaly detection, it needs to be able to pull data from all of the cyber data lakes that companies create.

Exabeam announced our UEBA product in 2014 in the partners pavilion at a Splunk .conf User's conference.

Around that time, most CISOs and security teams were drowning in a sea of data accompanied by too many security alerts, many of them not actionable. UEBA and alert triage tools have helped significantly, but this is still a problem today with legacy SIEMs.

Today's SIEMs cost too much

Fast forward to 2022, and what we have is a set of antiquated technology stacks that are either still on-premises or have moved to the cloud as "lift and shifts", which are super expensive to maintain. Combined with the fact that cyber data is exploding, we end up with SIEMs that cost too much.

It's not uncommon to see large organisations spend upwards of $10m per year on legacy and next-gen log management and SIEM solutions.

Some early SIEM players still have nearly 50% of their customer install base running their SIEMs on-premises, which is far more costly than the cloud. But even as more customers move to the cloud, they have woken up to the fact that SIEM costs have gotten out of control.

So where does SIEM go from here?

It's time to bring the best of what cloud-native technology can do for SIEM. Cloud is super-fast, offers inexpensive storage, instantaneous search and can integrate a threat detection engine that can catch bad actors, including the majority who are now breaking in with valid credentials.

The SIEM industry has been ripe for forward evolution for some time, and we are committed to leading that evolution.