As more of our shipped code originates from agent-executed tasks rather than a human typing line by line, our incident response process needed sharper attribution than "which commit caused this." We needed to know which specification, and which specific agent run against that specification, produced the change that eventually broke something in production. That distinction turned out to matter more than we initially expected, and building the tooling to support it took most of a quarter.
Why Commit-Level Attribution Wasn't Enough
A commit hash tells you what changed. It doesn't tell you why the change was requested, what the author believed they were asking for, or whether the resulting code actually matched that intent. When a human writes code directly, those things are at least loosely recoverable from commit messages, code comments, and institutional memory about who was working on what. When an agent executes a task from a spec, the spec itself becomes the actual source of truth for intent, and if your tooling only tracks commits, that source of truth is effectively invisible during an incident review, sitting in a ticket system nobody thinks to check under pressure.
- A commit hash alone tells you what changed, not what the change was supposed to accomplish or whether it succeeded at that goal.
- The specification, not the commit message, is now the most reliable record of what a change was actually meant to do.
- An incident where code matches its spec exactly is a fundamentally different problem than one where the code deviates from spec.
- Without a link from deployment back to spec, an on-call engineer has to reconstruct intent from scratch under time pressure.
What We Built
We tag every deployment with a reference to the originating task specification, not just the commit hash it was built from, and that reference now flows through our entire observability stack: logs, traces, and alerts all carry it as a first-class field alongside the usual service and version tags. When an alert fires, an on-call engineer can jump directly from the alerting dashboard to the exact spec that produced the code currently running in production, without first digging through commit history or asking around to find out who wrote it and why.
This let us trace a production incident back to the actual intent behind a change, not just its diff, which sounds like a small distinction but changes the entire shape of an incident review. Instead of starting from "what does this code do differently than before," the review can start from "did this code do what it was asked to do," which is both a more useful question and, in our experience, a faster one to answer definitively.
- Every deployment carries a reference to its originating task specification as a first-class, queryable field.
- Logs and traces propagate that reference downstream, so a slow query or an error five services away can still be traced back to its originating spec.
- Alerting dashboards link directly from an active incident to the spec that produced the currently-deployed code.
- Postmortems now classify root cause explicitly as spec gap, agent execution error, or human review miss.
The Distinction That Actually Changed Our Postmortems
This mattered most for incidents where the code technically matched its spec but the spec itself had missed an edge case, a failure mode that's genuinely distinct from a straightforward implementation bug and one our tooling couldn't have surfaced clearly a year ago. A spec-gap incident calls for a different fix than an implementation-error incident. Patching the code addresses the immediate symptom in both cases, but only a spec-gap incident tells you that the same class of gap is probably lurking in every other spec written the same way, by the same team, for the same category of feature.
We've started explicitly classifying incidents into these buckets during every postmortem: spec gap, agent execution error that deviated from a correct spec, or human review miss where a reviewer approved a change that shouldn't have gone out. The distribution across those buckets has been more informative than we expected. Spec gaps have turned out to cluster heavily around a handful of feature categories, particularly anything involving user permissions or billing state, which told us exactly where to invest in better spec templates rather than spreading that effort evenly across the whole codebase.
Building The Attribution Chain Without Slowing Deployments
None of this was free to build, and the harder engineering problem turned out not to be the tagging itself but keeping the attribution reference intact as a change moves through our pipeline: a spec identifier that survives a rebase, a squash merge, a staged rollout across multiple regions, and eventual promotion to full production traffic, without ever silently dropping or becoming ambiguous along the way. Our first implementation attached the reference as a commit trailer, which worked until a squash merge collapsed several commits with different originating specs into one, at which point the trailer became misleading rather than simply missing.
We solved that by moving the reference out of commit metadata entirely and into a small deployment manifest generated at build time, keyed to the actual artifact being deployed rather than to any particular commit in its history. That manifest travels with the build through every environment, gets embedded into the running service's health endpoint, and is what our observability stack actually queries rather than trying to reconstruct anything from git history at incident time. It's a small piece of infrastructure, but getting it right took a full rewrite of our first attempt, and we'd recommend anyone building something similar plan for that rewrite rather than assuming the first design will survive contact with squash merges and multi-region rollouts.
What This Looks Like Day To Day
In practice, the attribution chain shows up most concretely during an active incident, when minutes matter and nobody wants to spend them digging through git blame. An engineer paged at two in the morning can now pull up the failing service, follow a single link to the spec behind the currently deployed version, and immediately see the acceptance criteria and edge cases that spec's author considered at the time. Half the time, that's enough to immediately tell whether the incident is a spec gap or something the code did wrong despite a perfectly reasonable spec, which shapes the fix before the engineer has even opened the actual code.
None of this replaces good judgment during an incident, and it hasn't made postmortems shorter in any dramatic way. What it's done is make the postmortems more accurate, and accuracy compounds. A postmortem that correctly identifies a spec-writing pattern as the root cause, rather than blaming an agent or a specific line of code, produces a fix that actually prevents the next five incidents in that category rather than just the one currently being discussed.
What We'd Still Like To Improve
The gap we're most aware of right now is that our attribution chain currently stops at the spec level, it doesn't yet capture which specific agent run, out of potentially several attempts against the same spec, actually produced the code that shipped. Most of the time that distinction doesn't matter, since a spec that gets executed cleanly on the first attempt is by far the common case. But we've had a small number of incidents where an agent's first attempt against a spec was rejected in review, a second attempt incorporated review feedback and was approved, and reconstructing exactly what changed between those two attempts after the fact required more manual digging than the rest of the attribution chain would suggest should be necessary.
We're currently working on extending the deployment manifest to reference the specific agent run and its full review history, not just the originating spec, which should close that gap. It's a smaller piece of work than the original attribution system was, mostly because the infrastructure for carrying a reference through the pipeline already exists, but it's the next thing on our list precisely because it's the one place where we've watched an engineer lose time during an otherwise well-supported incident review.
How This Interacts With Our On-Call Rotation
Attribution tooling only pays off if the people relying on it during an incident actually know how to use it under pressure, and we underestimated how much that mattered when we first rolled this out. The first few incidents after launch, engineers on call defaulted to their old habits, digging through commit history and asking around, simply because the new attribution links weren't yet part of their reflexive incident response, even though the information was sitting one click away the entire time.
We fixed that the same way we fix most unlearned-habit problems: we built the spec-attribution link into the incident response runbook as an explicit early step, and we now run it as a deliberate checkpoint during onboarding for anyone joining the on-call rotation, walking through a past real incident and showing exactly how the attribution chain would have shortened the original investigation. That kind of concrete, worked example has stuck far better than simply announcing that the new tooling exists and expecting adoption to follow on its own.
A Note On What This Doesn't Solve
It's worth being honest that none of this tooling reduces the number of incidents we have, only the time it takes to understand one once it's happened. We were careful not to oversell it internally as a reliability improvement in its own right, since conflating faster diagnosis with fewer failures would be a mistake that eventually shows up as a false sense of security. The actual reliability gains from more disciplined spec-writing are real, but they're a separate effect from the attribution tooling itself, and we track them as two distinct metrics rather than letting an improvement in one get credited to the other.