What a firewall log actually becomes

Three parsers exist specifically for network appliances: cisco_asa, fortinet and palo_alto. Collecting syslog tells you how to point a device at the platform and which parser name to ask for. This article is the next question, and the one that decides whether the collection was worth doing: once a line has been parsed, what can you actually search for?

The honest summary is that a firewall line becomes a searchable event with addresses, ports, a device-clock timestamp, the device's own severity and a usable allow/deny verdict — and a long tail of vendor-specific detail that lives in the stored document rather than in a searchable column. That last part is worth knowing before you build a process around it.

The path a line takes

Four steps, and each one can lose something.

1. 1

   The listener attributes the line to your organisation

   By source address, from a table of mapped ranges. Nothing else about the line is examined yet. If no range matches, the line is dropped — see Collecting syslog.

2. 2

   A parser is chosen for that sending address

   One listener can serve a FortiGate and a Cisco ASA on the same port, because the parser is selected per peer rather than per port. If no per-peer parser is configured, the listener's default is used, which is normally the generic syslog parser.

3. 3

   The parser is handed the whole line

   Including the syslog priority and header, which each parser peels off itself — that is where the device's clock and, on an ASA, part of its severity come from. The Palo Alto parser additionally anchors on the log-type field rather than counting positions from the start of the string, so a relay that rewrites the header cannot shift every field by one.

4. 4

   The result is normalised to OCSF, then projected back to the searchable columns

   There is no OCSF mapper written for any of these three devices, so they take the generic fallback path. That path is where most of the losses in this article happen. How your logs are normalised explains the two-stage model in general.

The timestamp is the device's clock

The timestamp column holds the time the device says the thing happened, read out of the line: the ASA syslog header, FortiGate's eventtime (or date+time+tz), the PAN-OS Generated Time field. The time we received the line is kept separately, in ingested_at. Both are on every row, and they answer different questions.

• A queued backlog keeps its real spacing. An appliance that buffers while a link is down and then flushes an hour of messages produces an hour of correctly spaced events, not a spike at the flush instant.
• Timelines line up with everything else. A firewall event and a Windows event log entry describing the same second now agree, because both use their own source's clock.
• timestamp >= is searching device time. If your appliance's clock is a few minutes out, so is its position in your search results — which is the correct behaviour, and a reason to keep NTP working.

Severity comes from the device

Each parser reads the severity the device stated and maps it onto the platform's 0-10 scale, so a severity filter is now a meaningful filter on firewall data.

One rounding to be aware of: the platform's 0-10 scale passes through the six-point OCSF severity enum on its way to the database, so the stored number is bucketed. A FortiGate warning is CIM 5 internally and stored as 6; a notice is CIM 3 and stored as 4. Filter with >= rather than = and this never bites you.

severity >= 6        # firewall warnings and above — now a real filter
severity >= 8        # matches a PAN-OS high or critical threat log
severity = 2         # informational firewall traffic, not everything

What each parser extracts

Cisco ASA

ASA does not write key=value. It writes three different address grammars depending on the message, and the parser handles each one explicitly rather than hoping for src=.

The verdict is read from the message text, which is where ASA states it: Deny becomes outcome = failure, permitted and Built become success, and Teardown is recorded as a connection close with an unknown outcome — it is the normal end of an allowed session, not a decision. A message using none of those verbs keeps outcome = unknown, which is the truth rather than a failure. The message id — 106023 — is kept as metadata.event_code and in the stored document, and remains searchable through raw_log contains.

Fortinet FortiGate

FortiGate writes genuine key=value pairs, and this is the parser that extracts most. Quoted values containing spaces are handled correctly. The field names below are from the FortiOS 7.4 Log Reference and are unchanged across FortiOS 6.4 to 7.6.

Palo Alto

PAN-OS writes comma-separated records and the parser reads them by position, against the PAN-OS 11.1 syslog field descriptions. Positions 1-31 are identical between the TRAFFIC and THREAT logs and have been stable since PAN-OS 8.0, so that shared prefix is read for both and the type-specific tail is read only after checking the log type. Values are split with quote awareness, so a rule name containing a comma does not shift every subsequent field by one.

A line that carries no PAN-OS log type in the first few fields, or that has fewer than 31 fields, is rejected rather than half-parsed. That is a deliberate change from the previous behaviour of producing an empty event: a Palo Alto token pointed at the wrong device now produces a visible ingest error instead of silence.

PAN-OS also writes Generated Time with no timezone. It is read as UTC, which is the PAN-OS factory default. If your firewall is configured to log local time, its events will be offset by that amount — the verbatim string is preserved at metadata.original_time so this is diagnosable, and setting the device to UTC is the fix.

What you can actually search on

After normalisation, every appliance event lands in the same shape. The searchable category is network for traffic, and the searchable action is a normalised verb derived from the device's verdict rather than the device's own word.

# Everything your firewalls blocked
category = network AND outcome = failure

# Blocked traffic from one address — now including Cisco ASA
category = network AND outcome = failure AND source_ip = "203.0.113.5"

# Firewall warnings and above
category = network AND severity >= 6

# All traffic through one named appliance
asset_id = "FGT-HQ"

# One ASA message id — still the raw line, but the id is also in the document
raw_log contains "%ASA-6-106023"

# A Palo Alto rule name, likewise
raw_log contains "deny-guest-to-server"

One thing that used to happen and no longer does: the reporting appliance was being written into the destination endpoint, so on an ASA message with no extracted destination dest_ip held the firewall's own address and looked like a real destination. The appliance is now recorded as the event's device rather than as its destination, and dest_ip is either the real destination or empty. Empty means no destination was recorded — which is the honest answer, and one you can filter for.

Nothing is thrown away, but not everything is reachable

Every key=value pair the parser found is preserved in the stored event, under the OCSF unmapped section. Three different audiences can reach it, and they do not have the same access:

The search language

Cannot reach it. Search is restricted to fifteen flat columns — see Searching your logs. Your only route to an unmapped value from search is raw_log contains.

A detection rule

Can reach it, at metadata.ocsf.unmapped.<field> — for example metadata.ocsf.unmapped.srcintf for a FortiGate ingress interface. Rules see the whole stored document, which is why a rule can be more precise than a search.

An export or an evidence bundle

Carries the whole document, including the raw line. Nothing described in this article as lost is lost from the record; it is lost from the index.

That distinction is the honest version of we keep everything. We do keep everything. We do not make everything searchable, and the gap between the two is exactly the set of fields above.

The wrong parser now tells you, mostly

Two of the three parsers reject a line that is not theirs, which is a change worth knowing about because it turns a silent misconfiguration into a visible one.

• Palo Alto rejects a line with no PAN-OS log type in its first few fields, or with fewer than 31 fields.
• FortiGate rejects a line carrying none of logid, type, eventtime, devid, devname or date — a FortiOS log always has at least one.
• Cisco ASA still accepts anything. ASA messages are free text with no mandatory marker, so there is no structural test that would not also reject legitimate messages. An ASA line the parser does not recognise produces an event with the raw line, the device time if the header carried one, and honestly empty structured fields.

So the check after connecting an appliance is still worth doing, and it is the same one: does a sample event have a source address in it. If events are arriving and source_ip is empty on all of them, the parser is wrong for the device — including now on an ASA, where an empty source_ip used to be normal and no longer is.

One related consequence: a replayed batch of firewall lines is stored twice. Duplicate suppression is keyed on an identifier the source supplies, and a syslog line supplies none — unlike, say, Okta System Log events, which carry a UUID and are deduplicated on it. If a collector is restarted mid-buffer, or somebody re-posts a file of captured lines, you will have two copies with different arrival times and no way to tell them apart other than the raw text.

What to read next

1. Collecting syslog from firewalls and appliances — the transport, the ports and the address-to-tenant mapping every appliance needs first.
2. How your logs are normalised — the OCSF schema these events are written into, and why there are two representations.
3. Searching your logs — the fifteen fields, and the limits of raw_log contains.