Collecting AWS CloudTrail

CloudTrail is AWS's own audit log: a record of every API call made in your account, whether it came from the console, a command line, an SDK or another AWS service. If you run anything in AWS — even one small workload — this is where you find out that somebody created an access key at two in the morning, or opened a security group to the whole internet, or signed in as root.

The platform reads CloudTrail from the S3 bucket that CloudTrail delivers into. This article covers how that is configured, what you get, and the several places where this source is narrower than you might assume.

What CloudTrail actually gives you

CloudTrail records management events by default — the control-plane actions that change your account. That default is the right one for security, and it is worth knowing what falls inside and outside it.

• Sign-ins, including root account sign-ins and whether multi-factor authentication was used. A root sign-in on an account that should never use root is one of the highest-value single events in AWS.
• Identity changes — users, roles and policies created, modified or deleted; access keys created; policies attached. This is where privilege escalation shows up.
• Network changes — security groups, network ACLs, route tables, internet gateways. An inbound rule opened to 0.0.0.0/0 is recorded here.
• Key and secret operations — KMS key policy changes, key deletion scheduled, secrets retrieved from Secrets Manager.
• Tampering with the audit trail itself — StopLogging and DeleteTrail. An attacker who turns CloudTrail off leaves that act in CloudTrail, and it is the last thing they leave.

What is not included by default: data events. S3 object reads and writes, Lambda invocations and DynamoDB item access are off unless you explicitly enable them on the trail. Enabling them gives you far more detail and produces an enormous volume that AWS charges you for. For most organisations of this size the honest answer is to leave data events off, or to enable them on one bucket that genuinely matters.

What the configuration needs

Six values, of which four are usually obvious and two need a decision.

`bucket` and `region`

The S3 bucket CloudTrail delivers into, and the region the bucket lives in. Both are required. The region is used for signing as well as for addressing, so a wrong region produces a signature failure rather than a not-found.

`prefix`

Optional. CloudTrail keys look like AWSLogs/<account>/CloudTrail/<region>/YYYY/MM/DD/…, so a prefix lets you target one account or one region within a bucket that holds several. If you run an organisation trail with many accounts delivering into one bucket, a prefix per account is how you keep them separate — and the stored position is scoped to the exact bucket and prefix pair, so adding a second trail does not inherit the first one's place.

`access_key` and `secret_key`

Static AWS credentials with permission to list and read objects in that bucket and prefix, and nothing else. Referenced rather than stored inline in production: the configuration holds the name of an environment variable and the worker resolves it at runtime.

`session_token`

Optional, for temporary credentials. Accepted and sent as the security token header. Read the warning about this below before relying on it.

`endpoint`

Optional override of the S3 host. It defaults to https://<bucket>.s3.<region>.amazonaws.com. The override exists for S3-compatible object stores, so a trail archived into a non-AWS S3-compatible bucket can also be read.

How the read works

Two stages, both plain signed HTTP GETs.

1. List — GET /?list-type=2&prefix=…&start-after=…, which returns an XML listing of object keys.
2. Fetch — GET /{key} for each new object, which returns a gzipped JSON document containing a Records array. It is decompressed and each record becomes one event.

Incrementality uses S3's own start-after parameter, keyed on the last object key consumed, rather than a timestamp. That is a stronger choice here than it might sound: CloudTrail keys are lexicographically ordered by delivery time, and S3 lists keys in binary order, so resume after this key is exact with no same-second ambiguity to reason about. A timestamp watermark on a source that delivers many objects per second would either skip or duplicate, and neither failure announces itself.

One poll is bounded: at most two hundred objects and twenty listing pages, with a ceiling of 64 MB on any single object. A bucket holding a year of history therefore catches up over many cycles instead of consuming the worker that serves every other tenant in the same cell. Default cadence is five minutes, adjustable per source.

The signing, and why there is no AWS SDK here

AWS requires every request to be signed. The platform implements Signature Version 4 — AWS4-HMAC-SHA256 — directly, using only the Go standard library, rather than taking a dependency on the AWS SDK.

The reasoning is recorded in the code and is worth repeating, because a customer's security reviewer will reasonably ask why: the entire S3 surface a log puller needs is ListObjectsV2 and GetObject, and both are a signed GET with no body. Pulling in a large SDK to issue two signed GETs adds a substantial dependency surface — and its own supply-chain and patching obligations — for a piece of arithmetic that is a few dozen lines of HMAC.

The honest cost of that decision is exactly the limitation described above: without the SDK, there is no credential-chain resolution, so no instance profiles, no shared credential files and no automatic role assumption. That is a deliberate trade and not an oversight.

Authorization: AWS4-HMAC-SHA256 Credential=<key>/<date>/<region>/s3/aws4_request,
               SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=<sig>
X-Amz-Date: 20260901T091422Z
X-Amz-Content-Sha256: <sha256 of the empty body>
X-Amz-Security-Token: <only when temporary credentials are configured>

A configuration with no keys is refused at validation rather than attempted: sigv4 requires access/secret keys. A configuration missing any required field is not scheduled at all, with one loud start-up message rather than a silent absence.

sigv4 requires access/secret keys
s3 puller: missing <fields>
s3 throttled: <detail>
s3 status=<code>: <detail>
decode ListObjectsV2: <detail>
decode CloudTrail object: <detail>
gunzip: <detail>
siem ingest: receiver skipped — invalid configuration

A throttled response backs off exponentially with jitter up to a cap, and an explicit Retry-After from AWS overrides our own guess. A failure on this source does not affect any other source or any other tenant.

What this source does not read

• CloudTrail delivered to CloudWatch Logs. Only the S3 delivery is read. If your trail writes only to CloudWatch Logs, add an S3 destination.
• CloudTrail Lake. Not read.
• EventBridge or SNS notification of new objects. Delivery is discovered by polling the bucket listing, not by being told.
• Anything other than CloudTrail's own document format. The fetch stage expects a gzipped document with a Records array. VPC flow logs or ALB access logs in the same bucket will not decode, and a prefix that catches them will produce decode errors rather than useful events.

What to read next

1. Connecting your first log source — the per-source status table, and which source to prioritise.
2. How your logs are normalised — what a CloudTrail record becomes once stored.
3. Log retention — how long this history is kept on our side, which is a separate question from your bucket's own lifecycle policy.