Keeping onboarding documentation in sync with a fast-moving codebase has been a losing battle for years, docs get written once and quietly go stale within a couple of months, and nobody notices until a new hire hits a wall following instructions that describe a system that no longer exists. This year we set up an agent that regenerates key onboarding docs directly from the current codebase on a recurring schedule, and it's the first documentation approach we've tried that actually kept pace with how quickly the underlying systems changed.
How the generation actually works
The agent reads the actual module structure, key configuration files, and recent architectural decisions, then drafts updated sections for a human to review before merging. It doesn't work from a prompt asking it to "write onboarding docs" in the abstract, it walks the repository the same way a new engineer would, following imports, reading configuration, checking which services actually call which others in practice rather than which ones an old diagram claims should be talking to each other. That grounding in the real, current state of the code is what separates it from docs a human wrote once with good intentions and then never revisited.
It's not fully autonomous; a human still reviews every generated update, but the review is far faster than writing the update from scratch would have been. The agent's draft gets the structure and the factual details right the overwhelming majority of the time, since those are things it can verify directly against the code. Where it needs a human's judgment is in framing, deciding which of several true things about a module are actually worth a new hire's attention versus which are implementation details that would just add noise. We treat that framing pass as the genuinely valuable part of the review now, rather than spending review time re-verifying facts the agent already got right by reading the source directly.
- The agent reads module structure, configuration, and recent architectural decisions rather than working from a static prompt.
- A human review pass focuses on framing and relevance rather than re-verifying facts already grounded in the actual code.
- Regeneration runs on a recurring schedule, catching drift automatically rather than waiting for someone to notice docs are stale.
- Sections tied to genuinely stable parts of the system regenerate to nearly the same output run over run, which itself is a useful signal of architectural stability.
What surprised us
One thing we didn't fully anticipate going in: the regeneration process itself became a quiet source of architectural feedback. When a section of the docs changed dramatically between two consecutive scheduled runs, that was often a signal that the underlying module had grown more tangled or inconsistent than anyone had consciously decided it should, since a genuinely stable, well-organized part of the system tends to produce very similar documentation run over run. We started treating large diffs in the generated docs as a lightweight architectural smell detector almost by accident, and it's caught a couple of cases where a module had accumulated inconsistent patterns across several unrelated pull requests without any single change looking alarming on its own.
We also learned that the generation schedule matters more than we expected. Running it too frequently produced review fatigue, the same reviewer approving small diffs every few days starts rubber-stamping rather than actually reading. Running it too infrequently let real drift accumulate between regenerations, defeating the purpose. We settled on a biweekly cadence for most of the codebase, with certain fast-moving modules regenerated weekly, which struck a balance between staying current and keeping the review workload sustainable for the person responsible for it.
The effect on new hires
New hires this year consistently report the onboarding docs matching what they actually find in the codebase, a small thing that used to be reliably untrue within a couple of months of any doc being written by hand. That match between documentation and reality changes the onboarding experience in a way that's hard to quantify but easy to notice: new engineers spend their early questions on genuinely non-obvious things, the "why" behind a design decision, rather than on discovering that a doc is simply wrong about how a system currently works. The trust that builds from documentation being reliably accurate compounds; once a new hire has caught the docs being right a few times in a row, they start actually relying on them instead of treating every doc as a starting hypothesis to verify against the code.
Where the limits are
The agent is good at describing what a system does and how its pieces connect. It's meaningfully weaker at capturing why a design decision was made in the first place, especially decisions that were the result of a tradeoff discussion that never got written down anywhere the agent can read. We've started supplementing the generated docs with a lightweight practice of capturing key architectural decisions in a short written record at the time they're made, specifically so the agent has that context available the next time it regenerates the surrounding documentation. That's a human discipline the tooling doesn't replace, it just makes the payoff for maintaining it more obvious, since now that context actually gets pulled forward into documentation automatically rather than sitting in a decision that only the people in the original conversation remember.
Once the onboarding docs pipeline had been running successfully for a couple of months, other parts of the organization started asking whether the same approach could apply to other kinds of documentation, API reference material for a couple of internal services, and a runbook for our on-call rotation covering common incident types. We extended the same underlying approach to both, with mixed but instructive results.
The API reference documentation extension worked almost as well as the original onboarding use case, for an unsurprising reason: API shapes are, like module structure, something the agent can verify directly against the actual code, so the same grounding advantage applied cleanly. The on-call runbook extension was less successful. Incident response guidance depends heavily on institutional knowledge about how past incidents actually unfolded and were resolved, information that doesn't live in the codebase itself and that the agent had no reliable way to access unless we explicitly fed it a written incident history, which mostly didn't exist in a structured form to begin with. That gap taught us something useful about the boundary of where this approach transfers well: it excels wherever the documentation's accuracy depends on facts derivable from the current state of a system, and it struggles wherever the documentation depends on institutional history that was never written down anywhere the agent can read.
We didn't want to rely purely on anecdotal new-hire feedback to judge whether this was worth the ongoing maintenance cost, so we started tracking a specific metric: the number of onboarding-related questions a new hire asks in their first two weeks that turn out to be answered incorrectly, or not at all, by the current onboarding docs. Before the agent-driven regeneration pipeline, that number had been creeping up steadily as the underlying codebase changed faster than anyone manually updated the docs to match. After a full quarter of the new pipeline running, that number dropped by roughly half, and importantly, it's stayed flat rather than creeping back up the way it used to, since the regeneration schedule keeps pace with the codebase automatically rather than depending on someone remembering to revisit a stale document.
That metric matters more to us than a generic sense that "the docs feel more current," since it's the kind of thing that's easy to convince yourself has improved without actually checking. Having a concrete number to point to has also made it easier to justify the ongoing review time this system requires to stakeholders who weren't directly involved in setting it up, since the value shows up as a measurable reduction in onboarding friction rather than just a qualitative impression.
We also tracked the ongoing cost of running the regeneration pipeline itself, since a recurring process that reads through a meaningful portion of the codebase on a schedule isn't free, and we wanted a clear answer before recommending this approach broadly to other teams inside the organization. The compute cost of each regeneration run turned out to be modest relative to the engineering time it saves in review, but the real cost worth watching was the reviewer's time, since a system that generates plausible-looking but subtly wrong documentation faster than a human can carefully review it would be a net negative even if the generation itself were free.
We addressed that by keeping the scope of each regeneration run narrow and predictable, rather than letting the agent range freely across the entire codebase on every run, which kept each review session focused and bounded rather than open-ended. That constraint was a deliberate tradeoff, a broader-scoped regeneration might catch more drift per run, but the corresponding review burden would have made the whole system less sustainable for the one person who owns this process alongside their other responsibilities. Keeping the loop small and frequent turned out to work better in practice than making it comprehensive and occasional.