Threat hunting across your endpoints

Hunting is asking a question of your own endpoint data rather than waiting for a rule to ask it for you. Has anything on my estate run an encoded PowerShell command this week? Which machines have stopped reporting? Did that file path appear anywhere other than the laptop I already know about? This article covers what you can ask, what you cannot, what stops a question becoming expensive, and five hunts worth running whether or not you consider yourself a security person.

One expectation to set first. This is not a query language and there is no text box to type SQL into. The interface is a structured document — you choose a dataset, a time range, some conditions, and optionally a grouping. That is a deliberate design choice with a real cost and a real benefit, both explained below.

Why there is no query language

Products in this space usually ship a query language, and analysts who know one are fond of it. We did not ship one, for one reason: a query language is a text-parsing surface, and a text-parsing surface in a security product is a place where injection lives. If a customer can send a string that becomes part of a database query, every future bug in the parser is a route into another tenant's data.

So a hunt is a structured document with named parts and no free-text expression anywhere. Three things are deliberately absent, and each omission is a closed door rather than an oversight: there is no projection list, because each dataset's set of returned columns is fixed; there is no raw expression field; and there is no way to override which organisation the query runs against.

The cost is that you cannot express an arbitrary question. Every column you can filter on is compiled into the product, and there is no way to register a new one at runtime. If you need a field the schema does not expose, that is a code change and a review — which is the correct price for widening the query surface of a security product, and a poor answer if you are in a hurry. We would rather say that than pretend the flexibility is there.

Two further properties are worth knowing because they close doors that are easy to leave open. A saved hunt is never stored as SQL — the stored document is parsed and compiled from scratch on every run, so it cannot become SQL even if something else wrote to that table directly. And when you name an aggregate result, that name is the only text you supply that reaches the query at all, and it is checked against a strict identifier pattern before it gets there.

What a question is made of

Dataset

One of six, listed in the next section. This is the first choice and it determines every field available to the rest of the query.

Time range

Mandatory. Either an explicit from and to, or a relative window written the way an analyst types it — 15m, 24h, 7d. There is no way to omit it.

Filter

Conditions combined with and, or and not. The available operators are eq, ne, gt, gte, lt, lte, in, not_in, contains, starts_with, ends_with, exists and not_exists.

Aggregate

Group by up to four fields and compute up to four metrics, using count, count_distinct, min, max, avg or sum. A filter on a metric refers to it by the name you gave it, not by an expression.

Order by and limit

Up to three sort fields. The default limit is 100 rows and the maximum is 1,000.

Join

Exactly one, not a list. process can be joined to activity or to detections, on the process GUID. Chained joins are how a hunt becomes a cartesian product, and the questions hunting actually asks need one hop.

The six datasets

The fields available on each are fixed and published, so the query builder in the console is generated from the same definition the server enforces. A duplicated field list would drift, and a drifted field list means an interface that offers filters the server rejects.

process      process_guid  parent_process_guid  storyline_id  sensor_id
             hostname  platform  process_name  image_path  command_line
             user_name  pid  ppid  session_id  start_time  end_time
             parent_observed

activity     process_guid  storyline_id  sensor_id  activity_type  target
             extension  size  entropy  entropy_delta  occurred_at

detections   alert_id  rule_id  process_guid  storyline_id  sensor_id
             tactic_id  technique_id  sub_technique_id  severity
             confidence  autonomy_tier  suppression_reason  detected_at

alerts       alert_id  rule_id  storyline_id  source_type  severity
             status  title  principal_id  detected_at

raw_events   sensor_id  channel  event_type  severity  process_name
             command_line  exe  parent_name  process_guid  user  hostname
             platform  path  pid  occurred_at

sensors      sensor_id  hostname  platform  architecture  sensor_version
             status  last_seen_at  enrolled_at

Two field-level notes that will save you a confusing five minutes. command_line can be filtered but not grouped — grouping on a large attacker-controlled text column is a cardinality explosion, so the schema refuses it. And on raw_events, the fields listed above are the complete set reachable inside an event's payload; the payload is not open-ended even though the underlying storage is.

In the detections dataset, autonomy_tier records the tier the detection actually fired at after any downgrade, and suppression_reason is non-empty when the precision layer softened it. That pair is what makes the downgrade-rather-than-drop design in Behavioural detections and attack coverage useful to you: nothing was thrown away, and this is how you find it.

Every bound that applies

Twelve limits are enforced and all twelve are published by the product itself, so nothing here is a surprise you discover in production. They exist because the absence of each one is an availability bug rather than a missing feature.

Two further caps apply to the request itself: the query document may not exceed 64 KiB, and the HTTP body is capped at 128 KiB before anything is read into memory. Execution runs inside a transaction with a hard fifteen-second statement timeout, so a hunt that would run for an hour is stopped rather than being allowed to compete with telemetry ingest for the same database.

Five hunts worth running

Each of these is expressible in the query builder as it stands. They are written below as the dataset, the window and the conditions, in the same words the builder uses.

1. 1

   Which machines have stopped reporting

   Dataset sensors, filter last_seen_at lt a timestamp a few hours ago, ordered by last_seen_at ascending. Note the trap: on this dataset the mandatory time range applies to enrolled_at, not to last_seen_at, so set it wide — 90d — or you will only see machines enrolled recently. This is the coverage question and it is the one most worth running on a schedule, because nothing alerts you when an endpoint goes quiet. A sensor that stopped reporting three weeks ago is a machine you are not protecting and do not know you are not protecting.

2. 2

   Encoded PowerShell anywhere on the estate

   Dataset process, window 7d, filter command_line contains -enc, grouped by hostname with a count. Encoding a command is not itself malicious — plenty of legitimate tooling does it — so read the result as a list of places to look rather than as a list of findings. A host that shows one is worth a glance; a host that shows two hundred is probably your management software.

3. 3

   Everything on one intrusion

   Take the storyline_id from any alert. Run it against alerts with storyline_id eq that value, then against process with the same filter, then activity. Three queries give you every alert raised, every process involved and every file touched in one story. This is the highest-value hunt in the product for a non-specialist and it requires nothing more than copying a string.

4. 4

   What the product decided not to tell you

   Dataset detections, window 7d, filter suppression_reason exists. Every detection the precision layer downgraded because it looked like known remote-management tooling. Read it once a fortnight. If your IT provider's tooling is in there, that is expected and correct. If something you do not recognise is in there, that is the exact scenario the downgrade-rather-than-drop design exists for.

5. 5

   The shape of ransomware, before an alert

   Dataset activity, window 24h, filter entropy_delta gte 1.5, grouped by sensor_id with a count. entropy_delta is the ransomware tell: a file that was readable and now is not. A handful of results is ordinary — installers and archives look like this. A few thousand on one machine in one hour is not, and by then a rule should already have fired.

A hunt you expect to run again can be saved, with a name, a description and optionally the ATT&CK technique and tactic it relates to. Saved hunts are validated when you save them rather than when somebody runs them — storing an invalid query would turn into a runtime failure for whoever ran it later, when the person who could fix it is no longer in the room.

Who can hunt, and what is recorded

Running a hunt requires permission to read telemetry, which a read-only analyst has. Hunting is a read across your own data, so gating it behind a write permission would mean a viewer could not investigate — and a viewer who cannot investigate is a person who has to ask somebody else to look for them.

Saving or deleting a shared hunt is a write, because a saved hunt is shared state. Verifying the hunt audit chain sits behind the detection-rules read permission rather than the telemetry one, deliberately, so that a plain analyst cannot audit their own oversight.

Every hunt execution is written to a hash-chained audit log — an append-only record where each entry is bound cryptographically to the one before it, so a deletion or an edit shows up as a break rather than as nothing. The chain can be verified on demand and answers with whether it is intact and, if not, where it broke. The person recorded as running the hunt comes from their authenticated session and the normalised source address, never from anything the request body claimed. See Roles and permissions for who holds which permission.

The errors you will actually see

A query that does not validate is refused with the reason attached, because an analyst who cannot tell why a query was refused writes a worse next one. The reasons name the schema, which the product already publishes in full, so they disclose nothing.

unknown dataset "<name>"; valid datasets are: <list>
limit <n> exceeds the maximum of 1000
time_range spans more than the maximum of 90 days
time_range.last exceeds the maximum window of 90 days

Those arrive inside an envelope of the form {"error":"invalid hunt query","reason":"..."}. Two more you may meet: request body exceeds the size limit or could not be read when the document is too large, and malformed request body when it is not valid JSON.

A failure during execution is different. It answers hunt execution failed with no detail, and that is on purpose rather than laziness: an execution error can carry database internals — a column name, a fragment of a query plan — and those should not cross the API boundary. If you hit one repeatedly, send us the time and the query and we will read the detail on our side.

If a hunt returns 403 with token carries no organization, the session is not scoped to a tenant; sign in again. A plain {"error":"insufficient permissions"} with 403 means the account lacks telemetry read.

What to do next

1. Run the sensor-coverage hunt today. It is the one whose absence is silent.
2. Read Incidents and the process tree so that a storyline_id in a hunt result is something you can follow.
3. Read Threat-intelligence feeds before assuming an imported indicator list is being matched against your traffic. It is not, and the article explains exactly what is.