Multi-Environment Infrastructure — One Setting, Three Environments, No Divergence
Photo by ian .R on Pexels
Part 2 claimed that the same build artefact runs in every environment and that the differences all live in configuration. This article is about what that actually takes, because the failure mode it introduces is a release that builds successfully, deploys successfully, reports success, and then runs with a setting nobody intended.
The names, values and topology below are illustrative. And a note on vocabulary before anything else: in this article deployment revision means an immutable snapshot of a deployed application, its image plus its configuration. That’s a platform concept, and it has nothing whatsoever to do with the document revisions in our running example.
Table of Contents
- A Green Release With the Wrong Setting
- Three Environment Classes and What They Isolate
- Base Configuration Plus an Environment Overlay
- Following One Setting Into a Manifest
- Who Owns Which Resource
- Secret References Are a Chain
- Values Known Only at Deployment Time
- Deploy Verify Switch
- What Revision Routing Does Not Isolate
- Rollback and Its Limits
- The Release Checks Worth Having
- What’s Next
A Green Release With the Wrong Setting
Our document application has a setting that enables the shared result-set cache from Part 9. It should be on in production and off in preview environments, where editors need to see content changes immediately.
Now, consider the ways that can go wrong without anything at all turning red. The variable gets defined for production and forgotten for the preview overlay, so previews inherit a value that makes editorial review baffling. Or it’s defined in the pipeline under one name and read by the application under another, so the application quietly uses its default. Or it’s set to the string "false", which is truthy in a naive parse, so the cache is on everywhere and an editor files a ticket saying “preview shows old content”.
Not one of those is a build error. Every one of them is a configuration decision that existed in two places and matched in neither, which is the same problem Part 1 described about data contracts, relocated into deployment. So the question here is the same question: which of these agreements can a pipeline check, and which does it merely transport?
Three Environment Classes and What They Isolate
| Class | Purpose | What is actually isolated |
|---|---|---|
| Preview | Review one proposed change | Dedicated application instances and cache; the platform, network and identities may be shared |
| Integration | Exercise changes together | A shared deployment with controlled updates |
| Production | Serve real traffic | Separately managed infrastructure and release policy |
That second column is where the useful precision lives, because a preview deployment is usually nothing like a private copy of the world:
A dedicated cache per preview stops key collisions between two reviews, and that’s well worth having given how much of this series hangs on cache identity. What it doesn’t do is isolate a shared database, and it doesn’t give each preview its own share of a downstream rate limit, so two active previews plus an integration run can happily exhaust a quota that production also depends on.
Two operational notes that are easy to get wrong. Previews get created per pull request rather than per push, which keeps the number bounded, though the effective trigger also depends on your repository provider and branch policies, so it’s worth verifying rather than assuming. And cleanup is a reconciliation job rather than a side effect of deleting a branch: derive the set of deployments that should exist, compare it against what does exist, and remove only the confirmed orphans. That requires the naming rules used for provisioning and for cleanup to be identical, which is exactly the kind of agreement that drifts.
Base Configuration Plus an Environment Overlay
The model here is deliberately small: one base file, one optional overlay per environment, then reference resolution.
values.yaml
→ values.{environment}.yaml, if present
→ resolve {{ configuration.references }}
→ generate deployment artefacts
The merge semantics are the part to really internalize, because that’s where the surprises come from:
- objects merge recursively, so an overlay adds or changes keys without restating the whole tree;
- arrays, scalars and explicit
nullreplace the base value. An array override is not an append, which is what people expect roughly half the time; - an explicit
nullis useful precisely because it removes: a preview can null out an inherited secret mapping in order to supply its own instead.
References resolve after the merge. A value consisting entirely of a reference keeps the resolved type, so a boolean stays a boolean, whereas a reference embedded in a longer string becomes text, which is usually what you want for connection strings and never what you want for a flag. Missing references and reference cycles have to fail generation, because partially substituted output that looks plausible is worse than no output at all.
Following One Setting Into a Manifest
Base configuration:
nodes:
spa:
env:
NUXT_DOCUMENT_CACHE_ENABLED: false
Production overlay:
nodes:
spa:
env:
NUXT_DOCUMENT_CACHE_ENABLED: true
Generated manifest entry:
{
"name": "NUXT_DOCUMENT_CACHE_ENABLED",
"value": "true"
}
Notice the quotation marks in that last step: container environment variables are strings, whatever the YAML happened to say. So the application still needs a runtime configuration property with a matching name and a parse that treats "false" as false. The generator transports a value. It can’t infer what a new variable is supposed to mean, and this is the exact seam where the third failure from the opening section lives.
What generation does buy you is that one source produces every artefact mentioning the setting, so application manifests, infrastructure parameter files, pipeline variables and a schema for editor completion, which means those artefacts can’t disagree with each other. And a reference validator can check that every pipeline variable a manifest refers to is actually defined somewhere, which catches the missing wiring failure.
But be clear about the limit here: those checks catch names that don’t resolve. They don’t catch a perfectly valid-looking endpoint pointing at the wrong environment, and that’s why the release verification later in this article matters.
Who Owns Which Resource
| Owner | Examples | Typical change |
|---|---|---|
| Infrastructure deployment | Container environment, network, identities, monitoring resources, managed cache | Platform change |
| Application deployment | Image reference, environment variables, probes, scaling rules | Release change |
| Secret management | Secret values, access grants, rotation | Credential lifecycle |
The point of that split is blast radius: an ordinary release changes the middle row and nothing else, so it can’t accidentally reconfigure a network. It isn’t a claim that infrastructure changes are safe or that application manifests are harmless. Both need review, and anything touching a shared resource needs a compatibility plan, because that shared resource is shared with production.
Secret References Are a Chain
A secret sitting in a container environment variable is usually not the secret at all. It’s a reference to an application-level secret, which is itself a reference into a vault, resolved using a managed identity that has to have permission:
Every link in that chain can break independently, and the interesting failures are at the two ends. Provision the identity and its permissions separately from the release, and verify that the deployed workload can actually resolve the reference, because a manifest referencing a secret it can’t read starts up and then fails in a way that looks exactly like an application bug.
Three consequences worth planning for. A reference protects configuration storage and does absolutely nothing about application code logging the resolved value, so the redaction discipline from Part 15 still applies in full. Rotation needs an explicit procedure, because changing a vault value doesn’t instantly update every running process: something has to restart or re-resolve, and “we rotated it” and “everything uses the new one” are two different states. And removal happens in two stages, because retained deployment revisions still reference the application secret. Stop mapping it into new revisions first, and delete the definition only once no retained revision needs it. Skip that ordering and you break the release you were keeping for rollback, which you’ll discover at the worst possible moment.
Values Known Only at Deployment Time
Some values simply can’t exist at the moment configuration is generated: the image tag being deployed, the environment’s assigned domain, monitoring connection details from freshly provisioned resources. Those get explicit placeholders which are substituted during deployment.
Substitute structurally, meaning operate on the parsed JSON rather than running a text replacement over a manifest, and verify before applying that no required placeholder survived the process. An unsubstituted placeholder that reaches a running container turns into a literal string inside a URL, and it fails in some confusing place rather than at the step which should have caught it.
Keep an auditable association between the source commit, the image identity and the generated configuration. When something is wrong in production, “which configuration is this instance running?” ought to be answerable from records rather than by going and looking.
Deploy Verify Switch
And now the mechanism Part 2 put off explaining. A deployment revision is an immutable snapshot of a deployed application: this image with this configuration. Creating one doesn’t send it any live traffic, and each one is reachable at its own internal address, which is what makes verification before cutover possible at all.
So for shared environments the candidate release consists of several revisions which have to talk to each other rather than to the current live versions, addressed by revision-specific internal hostnames:
The user-facing cutover happens at the proxy, and the internal default routes get updated afterwards. Those are separate control-plane operations rather than one transaction, so a failure between them leaves behind a state somebody has to work out. Which is a good argument for making the sequence explicit in your pipeline rather than leaving it implicit in some deployment tool’s defaults.
The verification step deserves a blunt caveat, because this is where release gates are routinely far weaker than they look. A smoke check that fetches the candidate URL and accepts any successful or redirect response proves that a process is listening. That’s all. If the application redirects unauthenticated requests to a sign-in page, then that check passes cheerfully while rendering is broken, the gateway is misconfigured and the API is unreachable. A gate worth the name asserts expected content on a representative page and exercises one real application operation against the candidate. And if that stronger check is planned rather than implemented, then it should be described that way.
What Revision Routing Does Not Isolate
Revision routing isolates application versions. Here are four things it doesn’t isolate, every one of which has caused bad releases:
- Shared data. Both revisions read and write the same database, so a schema migration takes effect for the old revision too, the moment it’s applied. Rename
RevisiontoRevisionNumberin one migration and the previous release, the one you were carefully keeping for rollback, starts failing on every query that selects the old name, even though nothing about that release changed at all. This is why migrations have to be backward-compatible with the release you intend to roll back to: add the new column, write both, migrate the readers, and drop the old column in a later release, once rolling back that far isn’t something you’d ever do. - Cache formats. A candidate that writes a new shape into the shared cache leaves behind entries the previous release can’t read. Version the cache keys, which is Part 3’s mechanism, so a rollback doesn’t go consuming the new revision’s data.
- External side effects. Messages published, emails sent and webhooks delivered by the candidate during verification don’t get undone by switching traffic back.
- Browser clients from the previous release. Somebody out there is holding a bundle from the old version and still sending its operations, which is Part 5’s compatibility point stated operationally: a schema or endpoint change that sails through today’s checks can break yesterday’s client for as long as those sessions last.
Rollback and Its Limits
Retaining the previous revision means a rollback doesn’t need a rebuild, and that’s a genuine improvement over the alternative. It is not instantaneous recovery, though: the retained revision may need reactivating, it has to pass readiness checks, and routing changes take time to propagate. Select the rollback target explicitly by identity, because assuming the most recent non-current revision is the right one is how you roll back to a broken release from two hours earlier.
For the initial migration, an edge gateway can switch between the old and the new system without touching public DNS, which makes the cutover reversible in minutes rather than hours. The same qualifications apply: configuration propagation takes time, cached responses at the edge outlive the switch, the origin needs to be warm, and requests already in flight are already in flight. So rehearse both directions, and keep the old system compatible with any shared-data changes the new one has made. Otherwise your rollback path exists on paper only.
The Release Checks Worth Having
Before you switch traffic: every configuration reference resolves, the workload can read the secrets it maps, no placeholders survived substitution, the candidate serves the expected content and completes one real operation, and the rollback target has been identified and is known to work.
Probes deserve a line of their own, since “the probe is green” is what most release gates actually rest on. A platform normally distinguishes two of them: a liveness probe answers “should this container be restarted”, and a readiness probe answers “may this instance receive traffic”. Point both at an endpoint that returns 200 unconditionally and what you’ve told the platform is that a process which can’t reach the API, the cache or the identity provider is nonetheless ready to serve visitors. Readiness should reflect the dependencies a request genuinely needs. Liveness should stay minimal, because making it depend on a downstream service turns a dependency outage into a restart loop. So check that each probe means what you intended before you lean on it as a gate.
After you switch: watch user-facing errors and latency rather than the deployment command’s exit code. A successful deployment only means the platform accepted your manifest and started the containers. It says nothing at all about whether the application serves correct pages.
The rule this article leaves you with is the same one it opened with, applied to infrastructure: for each setting that differs between environments, name the single file that defines the difference and the check that fails if it’s wrong. A setting whose correctness depends on somebody remembering which environment they’re looking at isn’t configured. It’s merely deployed.
What’s Next
- Part 17: Memory, Stability, and PM2 — Sizing the process that runs all of this, and why three memory limits are not one.
- Part 18: From a Latency Alert to a Root Cause — The evidence needed to see whether a release is actually healthy.
Munir Husseini is a software architect specializing in full-stack TypeScript, .NET, and cloud-native architectures.
Category: Nuxt in Production