Databases

Adding schema validation to a MongoDB collection that grew organically

DB

MongoDB's schema-less nature is a feature during early development and a liability a few years later. One of our longer-running client projects had a core collection with years of inconsistent document shapes, fields added by different developers at different times, some renamed and never fully migrated, some typed as a string in older documents and a number in newer ones, so we finally added JSON Schema validation rather than continuing to patch around the inconsistency in application code.

Why we waited as long as we did

Adding validation to a collection this size wasn't something we could do casually, because the risk of an overly strict validator rejecting a legitimate write in production is real, and a collection with years of undocumented shape drift is exactly the case where you can't fully predict what a strict validator will reject until you actually run it against real data. The collection in question backs a customer-facing feature with meaningful write volume, so a validator that's even slightly too aggressive doesn't fail loudly in a test environment; it fails quietly in production as a spike in write errors that someone has to notice and diagnose under pressure.

Turning on validation in warning mode first, rather than strict mode, surfaced a long tail of malformed documents without breaking the application while we cleaned them up. MongoDB's validationAction setting lets you log schema violations without actually rejecting the write, which turned what would otherwise have been a leap of faith into a measured rollout. We ran in warning mode for roughly three weeks, long enough to capture a full billing cycle's worth of write patterns, since one of the collection's write paths only fires monthly and we didn't want to miss it.

About 15% of documents needed a migration script before strict validation could be enabled, mostly from fields that had quietly changed type over several feature releases: a `quantity` field that used to store a string like "5 units" and later stored a plain integer, a `status` field with three different casing conventions depending on which service had written it last. The migration script itself was less risky than the validator rollout, since it ran against a snapshot first and its diffs were reviewed before touching the live collection.

  • Turning on validation in warning mode first, rather than strict mode, surfaced a long tail of malformed documents without breaking the application while we cleaned them up.
  • About 15% of documents needed a migration script before strict validation could be enabled, mostly from fields that had quietly changed type over several feature releases.

What the schema actually covers

We deliberately kept the validator looser than a strict interpretation of "schema validation" might suggest. Required fields and correct types for the handful of fields the application logic actually depends on being present and correctly typed, yes. But we didn't try to close off every additional field or enforce enum values on every categorical field, because doing so would have made the validator brittle against future legitimate schema evolution, and the entire point of choosing MongoDB originally was to allow the document shape to evolve without a migration for every new field.

That middle ground, strict on the fields the application actually reads and writes against, permissive everywhere else, has held up well operationally. New optional fields can still be added by a feature team without touching the validator, but the fields that matter for correctness can no longer silently drift into an unexpected type, which was the actual problem we were solving rather than schema purity for its own sake.

The process change that mattered more than the schema itself

The validator solved the immediate problem, but the more durable change was procedural: any new collection created on this project now gets a JSON Schema validator defined at creation time, even a permissive one, rather than adding validation reactively years later once the cost of doing so has compounded. Schema-less databases don't mean schema-less applications; every collection has an implicit schema the application code assumes, whether or not the database enforces it, and the only real choice is whether that implicit schema is written down and checked or left to erode silently one feature branch at a time.

What the migration script actually had to handle

Writing the migration script safely meant treating it with the same care as a production schema change in a relational database, even though MongoDB doesn't force that discipline the way an SQL migration framework would. We ran it first against a full snapshot of production data restored into a staging environment, diffed the before-and-after document counts and a sample of transformed documents by hand, and only then scheduled the actual production run during a low-traffic window with monitoring specifically watching write error rates for the hour afterward. The type-coercion logic itself needed several edge cases we hadn't anticipated from just reading the schema drift report: a small number of documents had a `quantity` field that was neither a string nor a number but an empty object, apparently the result of a partially failed write years earlier that had never been cleaned up, which needed its own explicit handling rather than falling through the general type-coercion logic and silently producing a nonsensical value.

Ongoing monitoring after the validator went live

Enabling strict validation wasn't the end of the project; we added dashboard monitoring specifically for schema validation rejection events, since a validator that starts rejecting a new pattern of writes months later, because an upstream service changed its output format in a way nobody flagged as related to this collection, is exactly the kind of silent failure the whole project was meant to prevent in the first place. That monitoring caught one real issue within the first quarter: a new internal service, unaware of the schema constraints, had started writing documents with a field type the validator correctly rejected, and the alert let us fix the writing service's output before it accumulated the kind of long-tail drift the original collection had taken years to build up.

Communicating the change to the engineering teams who write to this collection

Because several different feature teams write to this collection for different purposes, rolling out validation required more cross-team communication than a typical schema change on a service one team owns end to end. We documented the new required fields and their types in a shared internal doc, presented it in a brief session open to any team whose service touches the collection, and gave a two-week notice window before switching from warning mode to strict mode in production, specifically so any team with an in-flight feature that might conflict with the new constraints had a chance to flag it before enforcement began rather than discovering a conflict through a production incident.

A near-miss that reinforced the value of the warning-mode rollout

About a week into the warning-mode period, we found a batch job, run monthly and easy to overlook because of its infrequent schedule, that was writing documents missing a field the validator would have required. Had we gone straight to strict mode, that job would have started failing silently until whoever depended on its output noticed something was missing downstream, likely well after the fact given how few people had that job's schedule in mind day to day. Catching it during the warning period, with time to fix the job's output before enforcement began, was exactly the kind of near-miss the staged rollout was designed to prevent, and it's the concrete example we now point to internally whenever someone suggests skipping the warning-mode step to save time on a future validator rollout.

Extending the approach to newer collections

The confidence gained from this project changed how the team approaches every collection created since, not just this one. Two newer collections started with a JSON Schema validator defined on day one, deliberately looser than this collection's eventual validator but still enforcing the handful of fields the application logic actually depends on, and both have so far avoided accumulating anything like the multi-year drift this project had to untangle after the fact. It's a small amount of upfront discipline that's easy to skip when a collection is brand new and every document still looks the same, which is exactly why it's worth insisting on before that uniformity starts to erode.

Adding validation after the fact is more work than designing it in from day one, since retrofitting it means reckoning with years of accumulated drift all at once instead of catching each inconsistency as it's introduced. But it's far better than living without it indefinitely, and for any collection old enough that nobody on the current team can confidently describe its actual shape from memory, the audit alone, even before writing the validator, is usually worth doing on its own.

← Back to the journal

Have a project in mind?
Let’s talk.

Tell us where you are and where you want to go. We'll map the fastest route between the two.

Currently accepting new clients