TL;DR
A trace is the recording of everything that happened during one agent run — a tree of spans, where each span is one unit of work with a start time, an end time, and attributes. It turns a slow or wrong answer from a black box into something you can inspect step by step.
A run is more than a reply
When your agent answers a question, it almost never does just one thing. It calls the model, maybe searches a knowledge base, maybe calls a tool, then calls the model again with the results. Each step takes time, and the LLM steps cost money.
A trace records all of that for a single run. It's a tree of spans — each
span is one unit of work (llm.complete, tool.execute, retrieval) with a start,
an end, and attributes describing what happened. A step that triggers more work (a
tool that calls the model again) nests its own children underneath it. That
parent → child nesting is why a trace is a tree, not a flat list: it tells you not
just what happened, but what happened because of what.
Analogy
A trace is an itemized receipt for a run, not just the total. The total ("4 seconds, $0.05") tells you nothing about why. The receipt breaks it line by line — this step took 3.1s, that one cost most of the money — so you know exactly where to look.
Why you care
Without a trace, a slow or wrong answer is a black box — all you see is the final reply and a spinner that lasted too long. With a trace you can answer the questions that actually let you fix it:
- Where did the time go? The longest span is your bottleneck.
- What did each step receive and return? Every span carries its inputs and outputs.
- Did anything fail quietly? A tool can error and the agent can still produce a (wrong) answer — the trace shows the failed span even when the reply hid it.
That's the checkpoint: a 4-second run isn't a mystery once you have the trace — you open it, find the longest bar, and that span is where the time went.
Watch a trace get recorded
The demo replays a real run as a trace, revealing each span as it would have been recorded — first the run wrapper, then the model call, then the tool call nested inside it. Watch the bars: their length is how long each step took.
Common mistake
Treating the final answer as the only output worth checking. The answer can look perfect while a tool silently errored or retrieval returned nothing — the model just papered over it. The trace is where those silent failures are visible. Read the run, not only the reply.
Key takeaways
Key takeaways
- A trace is the full recording of one run: a tree of spans.
- Each span is one unit of work with a start, an end, and attributes.
- Parent → child nesting makes it a tree — it shows what caused what.
- Traces turn slow/wrong answers from black boxes into inspectable steps.
Go deeper: spans, traces, and where this comes from
The trace/span model isn't specific to agents — it's borrowed from distributed tracing, the same technique used to debug microservices:
- A span has an id, a parent id (null for the root), a start and end timestamp, a name/kind, and a bag of attributes (model, token counts, inputs, outputs).
- A trace is all the spans that share one trace id, assembled into a tree by their parent ids.
- The standard most tools speak is OpenTelemetry — so the same concepts (and often the same viewers) work across LLM agents, web backends, and databases.
For agents this matters because a single run fans out across the model, tools, and retrieval — exactly the kind of multi-step, nested work distributed tracing was built to make legible. The next lesson zooms into the span itself: what's inside one, and the kinds you'll meet.