← Lab Lab · SIEM validation layer

How the validation layer for Sentinel rules works.

A language model writes the detection rule; a deterministic check decides whether it stands. This page shows the pipeline, the errors it catches and what leaves the network along the way.

Starting point

A wrong rule stays silent

Writing detection rules for Microsoft Sentinel is no longer the bottleneck: Microsoft’s Security Copilot generates the KQL queries, and many companies already hold the licence. The bottleneck sits behind the writing. A rule that looks plausible and is subtly wrong runs without error and never returns a row. It looks clean in review, and in production its silence reads as “no incidents”.

CardinalOps measured this on more than 13,000 rules from several hundred production SIEM environments, Sentinel included: around 13% of rules can never fire (2025 report, summary at Help Net Security). The validation layer starts at exactly this point. It was built for BLUEROPE Consult GmbH, where developing a query shrank from days to minutes.

key factsas of 9 october 2026
Tests192, all passing
ParserKusto.Language — Microsoft’s own library
Network access of the deterministic checknone
Modelcloud service or local via vLLM
Outputrule file, report, exit code
One run

One run: one rule, two repairs

The validation layer binds every rule to Microsoft’s own parser and additionally checks it against a catalogue of silent errors, meaning patterns with which a valid rule never returns a row.

The screenshot shows the interface of the validation layer with the first draft. The rule calculates with ticks as if they were seconds; the parser accepts it, the check catalogue flags line 7. The first repair swaps the arguments and fails on negative_duration, the second passes the check.

Interface of the validation layer: the query on the left with line 7 marked, on the right the finding timespan_without_unit with a repair hint
First draft · one error in line 7 · the check runs locally, with no model call and no network
Pipeline

Four checks see the same query

Every generated query is bound against Microsoft’s own KQL parser, the Kusto.Language library that Microsoft publishes itself. The check resolves tables and fields against the workspace schema, not against an assumption. Findings go back to the model as a repair task, and the repaired version runs through the same four checks again.

One run, end to end data the tool repairs itself you send it back in Use case REQUIRED Schema OPTIONAL Reference catalogue IF NO SCHEMA Event mapping SIGNED OFF House rules IN THE SYSTEM PROMPT Prompt one message, four parts 1 the schema — and how far to trust it 2 the binding event IDs 3 one finished query as a style example 4 the requirement itself Language model cloud or local (vLLM) KQL one draft ALL FOUR SEE THE SAME QUERY Microsoft’s parser syntax · columns · function arguments House rules hygiene · the zero-result traps Event coverage are the mapped IDs there? Second opinion advisory · requirement vs. query RESULT detection.kql best attempt Report always · findings · exit code Repair — findings go back into the model Improve — your correction re-enters the same machinery, nothing skipped
  • 01

    Binding against the parser

    Syntax, columns and function arguments are checked by Microsoft’s library itself. Validity is therefore decided by the parser and not by a second language model, which would only judge one guess with another.

  • 02

    The “unverified” mark

    Whatever could not be checked against the schema appears in the report as unverified. The check reports its own gaps instead of hiding them behind a green tick.

  • 03

    Second opinion, advisory only

    No parser can prove that the logic hunts the right threat. For that, the system queries a different model in a structured way; the decision stays with the deterministic check and with the human who holds the threat model.

Silent failures

Three errors the parser waves through

All three are valid KQL and run without error, and all three return practically no row in production. Two of them come from a single test run: a rule computed a rate from a duration it took for seconds and that was in ticks. When shown the error, the model corrected the unit and swapped the arguments of the difference in the process, leaving the rule dead again. Since then the check catalogue catches both patterns.

empty_filter_listreturns 0 rows
let Admins = dynamic([]);
SigninLogs
| where TimeGenerated > ago(1d)
| where UserPrincipalName in (Admins)
| where Location !in ("DE", "AT", "CH")
| project TimeGenerated, UserPrincipalName, IPAddress, Location

Constantly false. An empty inclusion list never matches. For exclusion lists the same is correct — the difference is structurally decidable.

timespan_without_unitreturns 0 rows
SigninLogs
| where TimeGenerated > ago(1d)
| where ResultType != "0"
| summarize FirstSeen = min(TimeGenerated),
            LastSeen = max(TimeGenerated), Attempts = count()
            by UserPrincipalName, IPAddress
| extend DurationSeconds = todouble(LastSeen - FirstSeen)
| where Attempts > 10 and Attempts / DurationSeconds > 0.5

Ticks, not seconds. One second is 10,000,000 ticks; the rate comes out seven orders of magnitude too small and never exceeds the threshold. The variable is still called “…Seconds”.

negative_durationreturns 0 rows
SigninLogs
| where TimeGenerated > ago(1d)
| where ResultType != "0"
| summarize FirstSeen = min(TimeGenerated),
            LastSeen = max(TimeGenerated), Attempts = count()
            by UserPrincipalName, IPAddress
| extend DurationSeconds = datetime_diff('second', FirstSeen, LastSeen)
| where Attempts > 10 and Attempts / DurationSeconds > 0.5

Computes the second argument minus the third. With the minimum on the left the duration is never positive and the rate never greater than zero. This error appeared while a model was repairing the ticks error: unit fixed, order reversed.

Context

What Microsoft’s kql-guard covers and what it does not

In 2026 Microsoft published kql-guard, an offline analyser for KQL. According to its documentation it checks syntax, reports unknown tables and columns when given a schema, and scores twelve expensive query shapes, from a missing time filter to unbounded mv-expand. It thereby answers whether a query runs and what it costs.

The validation layer answers a different question: whether a running, cheap query can return a row at all. kql-guard does not document the three patterns above, and both tools can run in the same pipeline. We did not run kql-guard ourselves; this assessment rests on its README as of October 2026.

The three queries verbatim, with findings →

Data outflow

What leaves the network has been measured

The deterministic check needs only the rule and the schema. It opens no connection to the workspace and none to the outside, and it runs entirely inside the operator’s network. The advisory second opinion, by contrast, is a model call and takes the same route as generation.

Generation and the second opinion can stay in-house as well: the system runs against a locally operated model, and switching takes two environment variables. If they run through a cloud service, the requirement text and the column names leave the network. We captured exactly what that covers on a real run.

For financial entities, the DORA datasheet places this mode of operation in its regulatory context.

measurement record · cloud run · one use case with schemacaptured from a real run
HTTP calls2 (generation + second opinion)
Total volume29,032 bytes
Contentrequirements document + column list
System prompt6,957 and 3,308 characters — house rules, no customer data
IP addresses in payloadnone
Event timestamps in payloadnone
Workspace connectionnone

No log data, no events, no production data. The measurement can be repeated at any time with the capture proxy that ships with the system — on the customer’s premises too.

Limit

What the check does not prove

The check catalogue knows the patterns we have found. In the run above the third version of the rule passes the check and still carries an error: count and duration are integers, the division truncates, and the rule fires only from one attempt per second instead of one every two. The correct form is todouble(Attempts) / DurationSeconds; the catalogue does not yet have this pattern.

Binding proves that a rule can technically fire. It does not prove that the rule describes the right threat, and it does not replace a test against real attack data. We name this limit because an overpromise would be exactly the kind of plausible-sounding false statement the system is built against.

Questions about the validation layer go to the engineer who built it →