Databases

Isolating AI agent database access with dedicated read replicas

DB

AI agents investigating a bug or exploring data during development had started running ad hoc queries directly against shared development databases, occasionally heavy enough to affect other developers' work, usually in the form of a slow, unindexed query an agent generated while trying to understand the shape of some table it hadn't encountered before. Nobody had done anything wrong exactly; an agent exploring unfamiliar data behaves a lot like a curious new engineer poking around, except it can generate and run that exploratory query far faster than a human would, and faster means more load hitting the database in a shorter window.

Routing agent access through dedicated read replicas solved it cleanly, and in retrospect it's a fairly obvious application of a pattern we'd already used for analytics workloads long before agents entered the picture: give the workload that doesn't need write access, and doesn't need to be protected from, its own isolated copy of the data to hammer on. Agents get the freedom to run exploratory, occasionally expensive queries without any risk to shared infrastructure or to whatever a human developer happens to be doing on the primary database at the same time.

The replicas are cheap enough at our scale that the isolation cost almost nothing to add, which made this an easy decision once we'd identified it as the actual problem rather than something we spent much time debating. We did add one additional safeguard beyond the replica itself: a query timeout specific to the agent-facing connection, tighter than what we'd apply to a human developer's session, since an agent that generates a genuinely pathological query has no instinct to notice it's taking too long and cancel it the way a human watching a spinning cursor eventually would.

The replicas have also turned out to be a convenient place to keep slightly stale or intentionally scrubbed data for agents working in contexts where the very latest production state isn't necessary and a small lag is actually preferable from a safety standpoint. That wasn't the original motivation for the change, but it's become a genuinely useful side benefit as more of our agent-assisted workflows have matured beyond simple read-only investigation into things that benefit from a slightly more controlled view of the data.

We also apply data scrubbing on a subset of these replicas specifically for agent access, masking obviously sensitive fields like full payment details or unhashed personal identifiers even though the agent's connection is already read-only and already isolated from production write paths. That's arguably a belt-and-suspenders measure given the other safeguards already in place, but it reflects a broader principle we've settled on for agent-facing infrastructure generally: isolation from the ability to cause harm is necessary but not sufficient on its own, and reducing the sensitivity of what an agent can see in the first place is worth the modest extra engineering effort even when the direct risk of a leak is already low.

Setting this up also gave us a template we've since reused for other non-human workload types, internal tooling that runs scheduled reports, third-party integrations that need read access for reconciliation, anything that isn't a human developer sitting at a keyboard now defaults to its own isolated replica rather than sharing a connection pool with production traffic or with active development work. What started as a fix for one specific friction point with agent-generated queries turned into a general pattern for how we think about non-human database access across the board.

← Back to the journal

Have a project in mind?
Let’s talk.

Tell us where you are and where you want to go. We'll map the fastest route between the two.

Currently accepting new clients