DevOps

Standardizing CI across a dozen repos with reusable GitHub Actions workflows

OPS

Over several years, each of our roughly dozen active repositories had grown its own CI pipeline, mostly copy-pasted from whatever template existed at the time and then diverged, one small tweak at a time, into something nobody could confidently describe from memory. GitHub Actions' reusable workflows finally gave us a clean way to consolidate all of that into a single maintained source, and the migration turned into one of the more satisfying infrastructure projects we've run this year.

Why this matters

Extracting the common lint, test, and build steps into a single reusable workflow file, called from each repo with a few input parameters, removed most of the duplication in one pass. The shared workflow accepts inputs like the Node version, whether to run an additional integration test job, and which package manager the repo uses, which covered the overwhelming majority of variance we found once we actually catalogued it. Repos with genuinely unusual requirements, a Python service among mostly Node repos, for instance, still needed their own workflow, which was the right call rather than forcing a bad abstraction onto a pipeline that didn't fit the shared shape.

Cataloguing the variance turned out to be the most useful part of the whole exercise. We had assumed each repo's pipeline was meaningfully different because each one looked different, but most of the differences were incidental, a job name here, a slightly different caching key there, rather than reflecting any real difference in what the repo needed from CI.

What we changed

Updating the shared workflow now propagates a security patch or a caching improvement to every repo at once instead of requiring a dozen separate pull requests, one per repo, each reviewed and merged on its own schedule. That used to mean a single dependency-scanning update took the better part of a week to land everywhere; now it's one pull request against the shared workflow repo and a version bump in each caller, which we did as a single batch pull request the day the shared workflow shipped.

The migration itself took about two weeks spread across the team, mostly because every repo's existing pipeline had some small undocumented quirk that needed to be understood before it could be safely replaced. One repo had a build step that silently depended on a stale cached artifact from a previous run, which nobody had noticed until we removed the caching behavior and the build started failing intermittently. Untangling that took longer than migrating the other eleven repos combined.

  • Extracting the common lint, test, and build steps into a single reusable workflow file, called from each repo with a few input parameters, removed most of the duplication in one pass.
  • Repos with genuinely unusual requirements, a Python service among mostly Node repos, for instance, still needed their own workflow, which was the right call rather than forcing a bad abstraction.
  • A handful of repos needed a short-lived compatibility shim during migration, since a couple of downstream deploy scripts referenced job names from the old pipeline directly.

Versioning and rollout mechanics

We version the shared workflow using tagged releases rather than pointing every caller at the default branch, which avoids the failure mode where an in-progress change to the shared workflow breaks a dozen repos simultaneously. Each caller pins to a major version tag, and we bump that pin deliberately as a reviewed change rather than letting it float. This adds a small amount of friction, every improvement needs an explicit rollout rather than landing everywhere instantly, but the alternative, an untested change silently propagating to production pipelines across the company, felt like an unacceptable risk for the convenience it would save.

We rolled the migration out gradually rather than all at once, starting with two low-traffic internal tools as a pilot before touching anything client-facing. That let us catch the caching quirk and a couple of smaller issues, secret naming mismatches, output variable differences, in a context where a broken pipeline meant a minor inconvenience rather than a blocked release. By the time we migrated the higher-traffic repos, the shared workflow had already stabilized.

What we'd do differently

Reusable workflows won't fit every repo, and we kept a few genuinely custom pipelines rather than forcing them into the shared template. For the majority that were just doing the same thing slightly differently, consolidating saved real maintenance time almost immediately. If we were doing this again, we'd document the shared workflow's inputs and expected repo structure before starting the migration rather than discovering the requirements repo by repo, since several of the surprises we hit were things a clear contract would have surfaced up front.

We'd also set up the versioned-tag rollout process from day one instead of adding it after the first shared-workflow regression scared us into it. The two weeks the migration took would likely compress to about one week with that groundwork done in advance, and the ongoing maintenance savings, we estimate somewhere around a few engineer-hours per month across the team, have made this one of the higher-leverage infrastructure investments we've made this year.

Handling secrets across a dozen repos

One problem we hadn't fully anticipated going in was secret management. A reusable workflow can accept secrets from the calling repo, but it can't assume every repo names its secrets the same way, and ours didn't; one repo called its deployment token `DEPLOY_KEY`, another called an equivalent value `PROD_DEPLOY_TOKEN`. Rather than forcing every repo to rename its secrets to match the shared workflow's expectations, which felt like a needlessly disruptive change to make purely for the convenience of the new pipeline, we had the shared workflow accept secret values as explicit inputs from the caller, with each repo's own workflow file responsible for mapping its locally named secret onto the shared workflow's expected parameter name. It's a small amount of boilerplate in each caller, but it kept the migration from turning into a secrets-renaming project on top of everything else.

Monitoring the shared workflow itself

Because the shared workflow now sits on the critical path for every repo's deploys, we added monitoring specifically for it: alerting if the shared workflow's own success rate drops, and a lightweight internal dashboard showing which version tag each repo currently points to. That dashboard turned out to be more useful than we expected during the rollout, since it made it immediately obvious which repos still needed to migrate and which had already adopted the latest tag, information that used to require manually checking each repo's workflow file one at a time.

Testing changes to the shared workflow itself

Changing a workflow that a dozen repos depend on needed a testing story beyond "merge it and see," since a mistake here doesn't fail quietly, it breaks CI everywhere at once. We set up a small internal test harness, a throwaway repo that calls the shared workflow the same way a real caller would, and any proposed change to the shared workflow has to pass a run against that harness covering each of the input parameter combinations we actually use in production before it gets tagged as a new version. It's caught a handful of changes that worked fine for the common case but broke a less-used input combination, the Python-adjacent repo's slightly different caching key, most memorably, before those changes ever reached a tagged release that repos could pull in.

The compatibility shim, in more detail

The short-lived compatibility shim mentioned earlier is worth explaining, since it's a pattern we'd reuse on a future migration like this one. A couple of deploy scripts referenced the old pipeline's job names directly, `build-and-test` rather than whatever the new shared workflow called the equivalent job, and rewriting every downstream script before the migration could proceed would have meant blocking the whole rollout on scripts owned by a different team with their own priorities. Instead we had the new shared workflow expose job outputs under both the old and new names for a transition period, letting downstream consumers migrate on their own schedule rather than in lockstep with the CI migration itself, and we removed the old names about six weeks later once every consumer had confirmed they'd switched.

Cost and runner minutes after consolidation

We also tracked GitHub Actions runner minutes before and after the migration, expecting consolidation to be roughly neutral on cost since the same lint, test, and build steps still had to run somewhere. Total runner minutes actually dropped by a modest amount, mostly because the shared workflow's caching configuration was more consistently applied than the patchwork of caching setups it replaced; several of the older per-repo pipelines had let their dependency caching quietly stop working months earlier without anyone noticing, since a slow but passing build doesn't trigger the same urgency as a failing one.

Feedback from the rest of the team

The reception from other engineers was mostly positive once the migration settled, though a few people initially missed being able to tweak their own repo's pipeline freely without touching a shared file that other teams also depend on. We addressed that by documenting a clear escalation path for proposing a change to the shared workflow, a short design doc and a review from at least one other repo owner, which added a small amount of process but avoided the alternative failure mode where the shared workflow becomes something people are afraid to touch at all, which would have undone much of the point of consolidating in the first place.

← Back to the journal

Have a project in mind?
Let’s talk.

Tell us where you are and where you want to go. We'll map the fastest route between the two.

Currently accepting new clients