13% of Your Detection Rules Never Fire
The figure comes from the 5th annual State of SIEM Detection Risk report by CardinalOps, which analysed more than 13,000 real detection rules from hundreds of production SIEM environments — Microsoft Sentinel among them, alongside Splunk, QRadar, and others. The result: roughly 13% of those rules are non-functional and will never fire. Not because someone wrote the query badly, but because the data source is misconfigured or the log field the rule points at is never populated in that environment. The same report puts MITRE ATT&CK technique coverage at only about 21% — most adversary techniques are simply not covered.
It is the kind of figure that is easy to nod past until you turn it around: roughly one rule in eight in your Sentinel workspace may be a placeholder that looks like protection. The dashboard says a detection exists for that technique. The audit ticks the box. The attacker walks through anyway, because the rule never fires. And the sharp part of the CardinalOps finding is the cause: missing data sources and missing fields. That is a data-and-checking problem — not one you fix by generating more rules.
A Rule That Parses Cleanly Can Still Be Dead
The dangerous part is that a dead rule is indistinguishable from a healthy one as long as you only read its text. It is valid KQL. It saves without an error. It runs on schedule, terminates cleanly, returns zero results — and zero results reads as "no suspicious activity," which is exactly how a working detection reads on a quiet day. There is no red bar, no exception, no signal that anything is broken. That is what makes this class of failure so treacherous: it fails silently, in precisely the direction nobody notices, until the incident arrives.
A rule dies this way when it filters on a table or field this workspace never populates, or when it uses a comparison that can never match the shape of the data — a condition that always evaluates to false. Both look entirely plausible in the query text. Both survive a review where someone reads the rule and nods. The only place the difference shows up is your workspace's actual schema — and nobody reads that in their head.
Two Rules, One Is Silently Dead
A concrete example. Both of the following fragments are meant to catch the same thing: a PowerShell process loading a base64-encoded payload. Both are valid KQL. An analyst — or an LLM you ask to review them — would judge either one reasonable.
// Looks correct. Parses cleanly. Silently dead in this workspace: DeviceProcessEvents // table never onboarded here → always empty | where FileName =~ "powershell.exe" | where ProcessCommandLine has "FromBase64String" // Same intent, against the table this workspace actually populates: SecurityEvent | where EventID == 4688 // classic process-creation log, onboarded | where NewProcessName endswith "powershell.exe" | where CommandLine has "FromBase64String"
The first rule references DeviceProcessEvents — a table from Microsoft Defender for Endpoint. In a workspace that only onboards the classic Windows security events into SecurityEvent and has never connected Defender for Endpoint, that table does not exist. The rule binds to nothing, never returns a row, and never fires — permanently, silently, with a green status. The second rule targets SecurityEvent, which this workspace does populate, so its references resolve and it can fire. The difference between a dead rule and a live one is not in the text. It is only in the real schema — and the one question that matters is: how do you check that across thousands of rules without opening each one?
How Do You Check Thousands of Rules — Without Asking a Language Model?
A large estate cannot be eyeballed, and re-reading the text is the exact act that misses silent rules. The obvious modern answer is to have an LLM review each rule. That is precisely where the approach fails at its core. A language model cannot be trusted to certify another language model's output — or a human's — as correct, because "looks correct" is the failure mode itself. The model will confidently wave the dead rule through for the same reason a human does: it reads as plausible.
Checking a plausible-looking artefact with a system whose native output is also "plausible, maybe wrong" adds no ground truth — it adds another layer of plausible. And an LLM that reviews rules is itself an LLM application handling security-relevant input, inheriting every known weakness of any LLM system exposed to untrusted input. What you need is a check that cannot be talked into a wrong answer. (How you build rules, rather than audit them, is a related but separate question — that is the generate-validate-refine loop in Detection Engineering with LLMs.)
Deterministic Validation Against Microsoft's Own KQL Parser
The solid answer is unglamorous: Microsoft ships the KQL engine as a library. Kusto.Language is the same parser and semantic analyser that Azure Data Explorer and Sentinel use themselves — open source and public. You can parse every rule with it; not an approximation of the grammar, but Microsoft's own tool. And then comes the step that does the real work: run semantic analysis against a state that holds the actual table-and-column schema of your workspace. Every table reference, every column, every function resolves against the real schema — or it does not.
A reference that does not resolve is not a "maybe." It is a proof that the rule cannot return the rows it claims to. That is what deterministic means: the same rule plus the same schema yields the same verdict every time, with no model in the loop and nothing that can hallucinate. You trade an opinion for a proof.
Parse, Bind, Flag What Cannot Be Proven
Concretely, the check runs in three stages. Parsing with Kusto.Language catches rules that do not even hold syntactically. Binding against the workspace schema catches exactly the cases CardinalOps names: references to tables that were never onboarded, and to fields a reduced connector configuration never delivers — the "missing data source" and the "missing log field." Type-checking catches the comparison against a column of the wrong type that always evaluates to false — as a diagnostic from the analyser, not a subtlety a reviewer has to spot.
The output is not a traffic light that says "probably fine." It is a list of rules that, as written, provably cannot fire, each with the exact reference that fails to resolve. This check scales because it is mechanical: across thousands of rules in a single pass, without a human — or a model — having to judge each one. For an MSSP maintaining rules across many tenants, the same check runs per tenant, against each tenant's own schema.
What This Check Cannot Do
Honesty is part of it, or you have merely swapped one false sense of safety for another. Deterministic validation proves structural and schema deadness: that a rule cannot return a row, or references something that does not exist. It does not prove the opposite. A rule that binds cleanly, resolves every table and column, and dutifully returns rows can still detect the wrong thing, miss the actual technique, or fire on benign traffic. Whether the detection logic matches the real adversary is a threat-modelling question — and that still needs a human who knows what this organisation has worth stealing and who is likely to come for it.
Value-level mistakes also sit outside pure schema binding — a filter on "user" where the data writes "User" is a casing bug that only a run against representative data exposes, not the parser. Deterministic validation takes an entire class of silent failure off the table cheaply and provably; it does not replace the threat model. For operators under NIS2, though, that provable, documentable evidence that your detections are alive is exactly the kind of proof that counts when it matters — more on that in NIS2 Incident Reporting in Practice.
Where Tippel Fits
This validation layer is exactly what Tippel builds for Microsoft Sentinel. Not a KQL generator — generating queries is what Microsoft's Security Copilot turned into a commodity. The defensible part is validation: parse every rule in your workspace with Microsoft's own KQL engine, resolve every reference against your live schema, and hand you the list of rules that provably cannot fire — the 13% you did not know you had. For MSSPs it runs per tenant; for an in-house SOC it runs across your own estate. It is a measurable, unglamorous discipline: validation, not more generation. What it looks like as engineering work starts deliberately narrow — one workspace, the real schema, one pass — and only scales after that.
If that lands on a problem you actually have, the AI Readiness Check is the cheapest way to prove it on your own estate, with an honest go/no-go at the end. And because this belongs in a room full of detection engineers more than in a sales call: I am happy to give a talk on it — to your SOC team, your MSSP partner day, or a security conference — the 13%, how to find them deterministically, and where the method honestly hits its limit. Get in touch and we will find the right format.