13% of Detection Rules Never Fire
The figure comes from the 5th annual State of SIEM Detection Risk report by CardinalOps, published in June 2025. It analysed more than 13,000 real detection rules from hundreds of production SIEM environments, Microsoft Sentinel among them, alongside Splunk, QRadar, and others. Averaged across all the SIEMs examined, roughly 13% of those rules are non-functional and will never fire. The queries are not badly written; the data source is misconfigured, or the log field the rule points at is never populated in that environment. The same report puts MITRE ATT&CK technique coverage at only about 21%, which leaves most adversary techniques without a rule.
If the average holds for your environment, roughly one rule in eight is dead. The dashboard still shows a detection for the technique, the audit ticks the box, and the attacker gets through because the rule never fires. CardinalOps names missing data sources and missing fields as the cause. Data and checking fix that; generating more rules does not.
A Rule That Parses Cleanly Can Still Be Dead
The text of a dead rule does not tell it apart from a healthy one. It is valid KQL and saves without an error. Sentinel runs it on schedule, it finishes without error and returns zero results. Zero results read as "no suspicious activity," which is how a working detection reads on a quiet day. No red bar, exception or other signal shows that anything is broken, and the failure goes unnoticed until the incident arrives.
A rule fails this way when it filters on a table or field this workspace never populates, or when its comparison can never match the shape of the data, so the condition always evaluates to false. Both look plausible in the query text, and both survive a review in which someone only reads the rule. Only your workspace's schema and data show the difference, and nobody holds both in their head while reading.
Two Rules, One Is Silently Dead
Both of the following fragments are meant to catch the same thing, a PowerShell process loading a base64-encoded payload. Both are valid KQL, and an analyst would judge either one reasonable, as would an LLM asked to review them.
// Looks correct. Parses cleanly. Silently dead in this workspace: DeviceProcessEvents // table present, but never populated | where FileName =~ "powershell.exe" | where ProcessCommandLine has "FromBase64String" // Same intent, against the table this workspace actually populates: SecurityEvent | where EventID == 4688 // classic process-creation log, onboarded | where NewProcessName endswith "powershell.exe" | where CommandLine has "FromBase64String"
The first rule queries DeviceProcessEvents, a table from Microsoft Defender for Endpoint. If the table is missing from the workspace altogether, the rule fails loudly. Sentinel records the run in the health log with “A table referenced in the query wasn't found”, and a rule whose target table was deleted is disabled automatically after repeated failures. The rule fails silently only when the table exists in the workspace but never receives data. Then it runs without error, returns zero rows permanently, and keeps its green status. The second rule targets SecurityEvent, which this workspace does populate, so it can fire. Whether a rule is dead or alive is therefore written in the schema and the data, not in the text.
Checking Thousands of Rules Without Asking a Language Model
Nobody checks a large estate by reading it, and reading it again repeats the method that misses silent rules. The obvious move today is to have an LLM review each rule. But a language model cannot be trusted to certify another language model's output, or a human's, as correct when "looks correct" is the failure mode itself. The model waves the dead rule through for the same reason a human does, which is that it reads as plausible.
Checking a plausible-looking artefact with a system whose own output is also "plausible, maybe wrong" yields no ground truth, only one more plausible verdict. An LLM that reviews rules is also itself an LLM application handling security-relevant input, and it inherits every known weakness of any LLM system exposed to untrusted input. The check must not be open to being talked into a wrong answer. (How you build rules, rather than audit them, is a related but separate question; Detection Engineering with LLMs describes the generate-validate-refine loop.)
Deterministic Validation Against Microsoft's Own KQL Parser
Microsoft publishes a parser and semantic analyser for KQL as a library. Kusto.Language is open source and public. It lets you parse every rule with Microsoft's own tool rather than an approximation of the grammar. Semantic analysis then does most of the work. It runs against a state that holds the actual table-and-column schema of your workspace, and every table reference, every column and every function then resolves against the real schema or fails to.
A reference that does not resolve proves that the rule cannot return the rows it is meant to. The same rule with the same schema yields the same verdict every time, because no model that could hallucinate sits in the loop.
Parse, Bind, Flag What Cannot Be Proven
The check runs in three stages. Parsing with Kusto.Language catches rules whose syntax is wrong. Binding against the workspace schema finds references to tables and columns that are missing from the workspace. That covers the cases CardinalOps names only in part. A field that is declared in the schema and that a reduced connector configuration never populates resolves without error. It shows up only in a comparison with actual population, for instance the share of non-empty values per column over the last 30 days. Type-checking reports comparisons between a column and a literal of a different type. That report comes from our own check catalogue, because Kusto.Language itself lets a comparison of a text column with a number pass without a diagnostic.
The check returns a list of rules that, as written, provably cannot fire, and names for each the reference that fails to resolve. It does not return a traffic light that says "probably fine." Because it works mechanically, it checks thousands of rules in a single pass, without a human or a model having to judge each one. For an MSSP maintaining rules across many tenants, the same check runs per tenant against that tenant's schema.
What This Check Cannot Do
Deterministic validation proves that a rule is dead structurally or by schema, meaning that it cannot return a row or references something that does not exist. It does not prove the opposite. A rule that binds without error, resolves every table and column and returns rows can still detect the wrong thing, miss the intended technique, or fire on benign traffic. Whether the detection logic matches the real adversary is for the threat model to decide, and that still needs a human who knows what is worth stealing from this organisation and who is likely to try.
Schema binding alone does not catch tables and fields that exist and stay empty; they show up only in the data. The same goes for value-level mistakes. A filter on "user" where the data writes "User" is a casing bug that the parser does not see and that only a run against representative data exposes. Deterministic validation removes an entire class of silent failure cheaply and provably, but it does not replace the threat model. Operators under NIS2 can use it to document that none of their checked rules fails on structure or schema. More on that in NIS2 Incident Reporting in Practice.