A slow or misbehaving agent run leaves behind a transcript: a sequence of messages, tool calls, and results, printed in order. That's enough to read what happened, but not enough to see why it took forty seconds instead of four, or why the same file got opened three times, or which of six nested sub-agents actually stalled. Those questions need structure the transcript doesn't have on its own — a span per operation, a timestamp on each one, and a way to nest them by who called whom. Without that structure, diagnosing a slow or looping run means re-running it while staring at it, which doesn't scale past the first few incidents.
An audit log exists to answer "what did the agent do, and can we reconstruct it for review" — a compliance and accountability record, usually append-only and retained for a fixed period (see what to log when an agent acts autonomously). A trace exists to answer a different question: "where did the time and cost go, and where did the logic branch." Traces are denser, more structural, and usually short-lived — kept for days or weeks to debug performance and correctness, not months for compliance. A system can have excellent audit logs and still be nearly impossible to debug, because an audit log tells you an action happened without telling you how long it took relative to its siblings or what triggered it.
A log line is flat: one event, one timestamp. A span has a start, an end, a parent, and a set of children — which is what lets a trace be rendered as a nested timeline instead of a scrolling list. Each model call, tool call, and sub-agent invocation should open a span when it starts and close it when it finishes, tagging it with what it was (tool name, model, sub-agent role) and what it cost (latency, tokens, dollars). The parent-child link is what turns "twelve things happened" into "this tool call happened inside this model turn, which happened inside this sub-agent, which was spawned by the root session" — a shape a flat log can't represent no matter how many fields each line has.
| Without a correlation ID | With one |
|---|---|
| Each tool call's logs sit in isolation; grouping them by session means guessing from timestamps | Every span in a session carries the same session ID, so a single query returns the whole trace in order |
| A sub-agent's spans look identical to the root session's spans | Sub-agent spans carry both the session ID and a parent-span ID, so nesting survives even across process or service boundaries |
| Retrying a failed step produces a log line indistinguishable from the original attempt | Retries carry an attempt number under the same span lineage, so three attempts read as one retried operation, not three unrelated events |
The ID needs to be generated once, at the top of the session, and threaded through every downstream call — including into sub-agents, since a sub-agent that doesn't inherit the parent's correlation ID produces a trace that looks like two unrelated sessions instead of one nested one.
Three failure modes are close to invisible in a transcript and immediate in a trace. A slow step: the transcript shows a pause with no explanation, the trace shows one span taking 8 of the run's 10 seconds and names exactly which tool call it was. A redundant call: the transcript shows the same file read twice with no indication anything is wrong, the trace shows two identical spans back to back and makes the waste visible instead of easy to skim past. A loop's origin: the transcript shows the same three steps repeating and it's tempting to assume the last one triggered it, the trace's span timestamps show exactly which earlier decision the repetition traces back to — often not the step immediately before the pattern became visible.
Tracing every span of every session in a high-volume system gets expensive fast, and most successful runs don't need a stored trace anyone will ever look at. The usual answer is sampling — trace a percentage of sessions in full. The rule that has to hold regardless of the sampling rate: any session that errors, times out, retries, or gets flagged by a downstream check must be traced in full, unconditionally, sampling decision made after the fact if needed. A sampling scheme that only decides in advance will, by construction, throw away disproportionately many of the traces someone actually needs — failures are exactly the sessions worth keeping, and they're rare enough that keeping all of them barely affects storage cost.
The test to apply: take the slowest agent run from the last week and try to answer "which single step accounted for the most time, and was it a model call or a tool call." If answering means re-running the session and watching a stopwatch, there's no trace — only a transcript. Spans with timestamps and a parent-child structure turn that into a one-query answer, and the same structure is what makes redundant calls and loop origins visible instead of merely inferable.
A trace is a historical record — useful after the fact, ignored on resume. If the process dies mid-run, what it resumes from is a checkpoint, not the trace — see checkpointing long-running agent tasks.