Decoupled SIEM: Brilliant or Stupid?
Frankly, not sure why I am writing this, I get a sense that this esoteric topic is of interest to a very small number of people. But hey … LinkedIn made me do it :-) And many of those few people are my friends or at least close industry peers.
So, the topic is so-called “decoupled SIEM” (I probably made up the term, but …hey… at least this is not an acronym like EDR so YMMV). In my mind, “Decoupled SIEM” is a way to deliver Security Information and Event Management (SIEM) technology where the data management (a) and threat analysis (b) are provided by different vendors.
Actually, you can decouple even more, such as into buckets like log collection/normalization, then storage/retention, and then detection content and hunting/investigations (What about the workflow? Well, that one often gets missed by the “decouplers”… then again you can try to buy a standalone SOAR for that!)
Peculiarly, my first experience with decoupled SIEM was many years ago when a SIEM (SIM, actually) installer CD (yes, it was a thing in the Stone Age, a trained mammoth would bring you a stone tablet that you insert into your cave computer…) told me to “take me out, and insert the Oracle installer CD instead.”
This reminds us that the current fascination with decoupled SIEM is a pendulum swing! Most “original SIEM” (not to be confused with original sin…) tools that started in the late 1990s started essentially decoupled (“here, go install this RDBMS” or “here, we can helpfully install this database for you, just bring a CD”). Later on, the limits of off-the-shelf databases piled up, splunk invented log search, and most vendors slowly moved to custom-made log storage and abandoned separate databases.
To be fair, what killed this approach the first time was not its decoupled nature, but the fact that a classic relational database is abysmally bad for scalable log retention, and SQL is abysmally bad for log analysis. Then a few more years passed, and our fascination with data lakes led us straight back to “but can’t we use existing cloud-scale storage platform” theme. And so we are here discussing it!
The romantic ideal behind this approach is that scalable data management and threat analysis are dramatically different areas and should be handled by specialists while, in theory, connecting them together in the modern age should be “easy” by a client or by yet another vendor.
To be fair, this model has both advantages and disadvantages, including — theoretically — the ability to combine the best detection with the best data platform, but also the potential for complexity, difficulty of support and various problems at the seams between the security parts and other data parts (admittedly less common in the modern era).
Pros and Cons of Decoupled SIEM
Arguments in favor of the decoupled SIEM
- It allows organizations to combine the best detection with the best data platform, selected separately
- It allows organizations to make use of the existing data platform they already paid for, or the one they prefer for other reasons (some good reasons, and some bad… )
- In theory, it allows the customer to use existing skills (such as SQL, but please don’t) and avoid learning the new query language
- From the vendor side, it allows the vendor to develop one rather than two very different areas of expertise — data management and threat analysis; for smaller vendors this means better chances of excellence
- To counter for previous failures of this approach, modern API based applications and cloud services allow for easy assembly of a solution from components
Arguments against decoupled SIEM
- Use of storage technology ultimately not designed for the SIEM tasks at hand
- Complexity: a natively designed solution delivered by one vendor is easier and simpler, in theory
- Lack of a single system built for a single purpose with implied shortcuts and cut corners (From “1998: We will just use Oracle” to “2020: We will just use Snowflake”)
- Making your security dependent on whims of a storage vendor, their pricing policy, their priorities, etc
- Not having “one face to scream at” aka easier supportability of both threat and data elements
I do see other arguments online, yet things like “cost basis” (not unique to decoupled, see Chronicle), “modern storage” (same), “open formats” (still largely a myth, sorry OCSF) and a few others don’t seem decisive to me. Also most successful SIEM products today are in fact NOT decoupled, but integrated.
Now what is the conclusion to this debate? I think those of you who remember me as an analyst would remember that I really hated the “it depends” answer to pretty much any customer question. Of course, IRL most stuff “depends”, everybody knows this, but where is the value for the client in that answer?
To start, given my experience with SIEM that spans more than 20 years, I do NOT think that the future belongs to a decoupled SIEM. I also do NOT believe that the decoupled SIEM is a broken SIEM that should not exist.
I think a valuable answer here would be to outline the type of an organization that would very much prefer the integrated SIEM where data collection, storage and security functions, such as detection and investigation are tightly coupled together (A)
For example, if you’re not resourced to engineer anything and barely resourced to consume security products, go with the classics: both platform and threat components built by the same vendor.
The second part of the answer would be to outline the type of an organization that would very much prefer the decoupled SIEM where the best collection and modern storage somehow harmoniously work with the best threat detection and response content. (B)
For example, if your organization’s DNA includes engineering and practicing an engineering-led approach, decoupled SIEM may be attractive to you.
Anything else to add?
Very useful context reading:
- The Transition from Monolithic SIEMs to Data Lakes for Security Monitoring
- Security is about data: how different approaches are fighting for security data and what the cybersecurity data stack of the future is shaping up to look like
P.S. If SIEM goes in the decentralized direction(s), decoupled approach will get an extra boost…
Related blogs:
