Async search jobs and the hunting library

This article is an API reference, and it is written as one deliberately. Everything below — background query jobs, the 40-hunt library, bookmarks and saved-result snapshots — is built, tested and reachable over the API. None of it has a console screen yet. If you are looking for where to click, there is nowhere to click; if you are willing to call an endpoint, all of this works today.

Why a background job exists at all

A query run through POST .../siem-query/run (see The pipe query language) is capped at 1,000 rows and bound by the same statement timeout as an ordinary search. That is the right shape for an interactive question. It is the wrong shape for a query that genuinely needs to look at tens of thousands of rows — a full password-spraying sweep across a quarter, say. An async job is the same compiler and the same safety machinery, run in the background with a higher row cap and no interactive timeout pressure.

The job lifecycle

A job moves through a status machine: queued → running → completed, failed or cancelled. One background worker process polls for the oldest queued job across every tenant, claims it with a row-level lock so two workers never pick up the same job, and then pins its execution back to that job's own tenant before running it. Each run is bound to ten minutes of wall-clock time, on top of the executor's own statement timeout.

All of these are gated on the same siem:queries read/write permission as ordinary search, and they are licence-gated automatically — any path segment starting siem or siem- is matched by the platform's own licence gate, so there was no separate gate to wire up for these routes.

Results are paged rather than returned in one response: GET .../siem-query/jobs/{jobID}/rows takes offset and limit against the stored, completed result set — up to the async row cap of 100,000 described in The pipe query language.

The hunting library

Forty built-in hunting queries ship as reviewed content, mapped to MITRE ATT&CK — the same choice the detection-rule catalogue already made, and for the same reason: content that is reviewed like code needs no migration to add or change an entry. Every one of the 40 is compiled and validated as part of the platform's own test suite, so a hunt that would fail to parse or compile fails the build rather than an analyst's trust in it later.

GET .../siem-hunting/library returns the list. Each entry is a named pipe query with its ATT&CK mapping — read it, copy the query text, and run it yourself through POST .../siem-query/run or as an async job if it is a wide one.

Bookmarks

A bookmark is a per-tenant, per-analyst pointer — you can save a hunt or a query of your own for quick reuse. POST .../siem-hunting/bookmarks creates one, GET lists yours, DELETE .../siem-hunting/bookmarks/{id} removes it. There is no console list for these today; they are reachable only by calling the API directly.

Saved-result snapshots

GET .../siem-hunting/results lists your saved snapshots, GET .../siem-hunting/results/{id} reads one back in full, and DELETE .../siem-hunting/results/{id} removes it. As with bookmarks, there is no console screen for any of this yet — it is Postgres-backed and reachable only by calling the endpoints directly.

What to read next

1. The pipe query language — the language a hunt or an async job is written in, and its own safety limits.
2. Threat hunting — the equivalent, console-backed hunting workflow on the endpoint side, if you are looking for something you can actually click through today.