← All posts

Deploys: 20 minutes to 6

Measured: 20m37s per deploy, 12m16s of it Next.js building on a 2-vCPU box. Now the image is built once on a CI runner, tagged by the core commit, skipped when the tag exists, and pulled by the box. A workspace-only deploy is 6m27s end to end; a core change about 17. Plus the twenty-minute rollback that taught the pipeline to refuse a superseded commit.

Vocion Team

Status: shipped and measured. On the instance that runs Vocion for Metacto: metacto-vocion-agents#179 (the CI build and the pull, merged 2026-09-24 19

), #181 (registry cache), #182 (the superseded-run guard), #188 (cache only the deps stage). The pipeline is the instance's; the shape is generic and documented at Deploys: CI-built image, the box pulls.

The measurement

2026-09-24: "Why are deployments taking so long? Prioritise a fix." One run, timed on the box:

StepTime
next build --webpack on 2 vCPUs12m16s (60%)
tooling npm ci before the workspace applies1m48s
migrations, including the pre-migration pg_dump2m00s
export + applies + git~3m30s
Total on the box20m37s

The build was the deploy. And the build ran on every push — a one-line change to a skill rebuilt Next.js from source on a machine that is sized to serve, not to compile.

What we rejected

A bigger box. It pays every month for a build that should not be there at all.

Turbopack. The core next.config has a webpack hook for Temporal's proto files; switching the build is a separate experiment, not a deploy fix. It stays on the list.

The shape

Build once, on a GitHub runner, where the cores and the cache are. Tag the image by what it was built from — the core commit and a hash of the brand assets baked into the client bundle: core-<sha12>-<brand8>. Before building, docker manifest inspect the tag; a workspace-only push does not move the core pin, so its tag already exists and the build job finishes in seventeen seconds with image exists — nothing to build. The box gets the image reference and a run-scoped registry token, pulls with one retry, tags it vocion-app:latest, and prints a receipt line: [deploy-receipt] image=pulled … duration=…. A manual run on the box with no image reference still builds locally and prints image=built, exactly as before.

The receipts

First CI-built deploy (2026-09-24 20

, run 36048699280): box step 7m12s, was 20m37s. Image pulled in about two minutes, migrations two, swap four seconds, applies about three. The build job, cold: npm ci 1m30s, next build 6m29s (was 12m16s on the box), push 42s — and 8m10s exporting the build cache to the GitHub cache. Moved to a registry cache (#181).

Workspace-only deploy (2026-09-24 20

, run 36052079253): 6m27s end to end. Build job 17s. Box 6m07s: git sync and pull 48s, migrations 2m12s, swap 8s, applies 2m53s — of which the tooling npm ci is still 1m48s.

Core change (2026-09-24 21

, run 36055982755, 9f56c82e): 17m18s end to end — build 9m56s (deps cached; next build 5m49s; push 13s; cache export 68s), box 7m19s including the tooling volumes' first install, reused from then on. #188 exports only the deps stage's cache afterwards: exporting the whole build's cache cost 3m43s a run to the registry, gigabytes of a builder stage the next commit invalidates anyway. Expected build about eight minutes from here.

So: workspace-only 20 → 6.5 minutes; core change 20–24 → about 17, with the next two levers being the box's — bake the tooling image (−1m48s) and overlap pg_dump with the pull (about −1m) — and the runner's, for the six-minute Next build.

The incident the speed caused

20

the same evening. A run that had started at 19
— before the pipeline change, with an older core pin — finished its box step after the new core had gone live, and put the old one back. Production was on the previous core for about twenty minutes.

Cause: cancel-in-progress: false, which is right (a deploy must never be killed mid-swap), plus a ten-minute build job, which is a window main can move in. Two correct deploys, standing on each other.

Fix (#182): right before the SSH, the deploy job reads refs/heads/main from the remote and compares it with the commit that triggered the run. If main has moved past it, the run marks itself superseded and does not deploy — and says so as a warning, not a failure. The guard was exercised within the hour: the core build of fa2c6ed8 correctly skipped its deploy because main had already moved to 9f56c82e.

What this changes for verifying a deploy

Nothing, and that is the point. The health gate was already reading version.txt from the public URL and asserting the pin this commit carries is what is serving — the previous container answers on 443 perfectly well, which is how a deploy that never swapped used to go green. The receipt lines now say image=pulled or image=built beside the migrate, drain, swap and prove lines from Two deploys in one, and the forty-second gap. A green run is a claim; version.txt and a grep inside the container are the evidence.

And the pin bump is still manual: merging core deploys nothing until the instance moves its submodule pointer and pushes. That is the deploy that takes six minutes now.