A/B Testing at the SSR Level — One Assignment per Request

Hero image credit: Photo by Francesco Ungaro on Pexels

Part 11 of the Nuxt and .NET series.

Part 10’s visibility rules consumed an experiment assignment without saying where it comes from. This article supplies it — and deals with the failure that makes experiments on a server-rendered site harder than on a single-page application: the server and the browser can disagree about which variant the visitor is in.

The examples use the fictional document application. The statistical guidance at the end is deliberately conservative and not a report of any measured result.

Table of Contents

• Two Renderers Can Disagree About the Variant
• Assign Once Before Anything Renders
• Why a Cookie and What It Does Not Do
• Reusing the Assignment During Hydration
• Test Definitions as Content
• Cache Identity and Variants
• Forcing a Variant for QA
• Measuring the Experiment Is a Separate Discipline
• What This Design Guarantees
• What’s Next

Two Renderers Can Disagree About the Variant

In a client-rendered application, an experiment is three lines: read a flag, pick a variant, render. There is one renderer, so there is nothing to disagree with.

Server-side rendering removes that comfort. The HTML is produced on the server, which means the variant has to be chosen before any component renders. And then the same components run again in the browser during hydration, where — if they choose the variant by the same procedure rather than reading the server’s choice — they can choose differently. A random assignment evaluated twice is two assignments. Even a deterministic one can differ if it depends on anything the browser sees differently: the current time, a cookie the server wrote in the response that is not yet readable, the viewport.

The visible results are a flicker as the layout switches under the visitor, a hydration mismatch warning, or the quieter and worse outcome where the DOM keeps the server’s markup while the component’s state believes something else. The measurement consequence is worse still: the visitor was counted in one variant and shown the other, which does not merely add noise — it biases the result.

So the requirement is narrow and strict: one assignment per request, made before rendering, and reused rather than recomputed in the browser. Everything below follows from that.

Assign Once Before Anything Renders

Assignment happens in server middleware, before the page’s components run. For each active test, the middleware determines the visitor’s variant, stores the complete map in request-scoped state, and makes it available through a composable:

Components then consume it without knowing any of this:

const { variant } = useAbTest('layout')

<DocumentLayoutA v-if="variant === 'A'" />
<DocumentLayoutB v-else />

Two details in the assignment step are worth deciding explicitly. Prefer a deterministic assignment — a hash of a stable per-visitor identifier combined with the test id, bucketed by the configured split — over calling a random number generator. Deterministic assignment produces the same answer if it is ever computed twice, which turns a whole class of the disagreements above into non-events, and it makes the assignment reproducible when you are investigating one visitor’s session. And assign every active test at once, at request start, rather than lazily when a component asks. Lazy assignment means the set of assignments depends on which components rendered, which makes exposure data depend on page structure.

Why a Cookie and What It Does Not Do

The assignment is persisted in a cookie — abt=layout:A,teaser:B or similar — for three good reasons. It is present on the request, so the server can read it before rendering, which server-side session lookup cannot promise without a round trip. It survives page loads, so a visitor does not get reassigned on every navigation. And it requires no server-side state, so any number of replicas behave identically without sharing a store.

Two things a cookie does not do, both of which have bitten people.

It does not make a shared HTTP cache serve different content. A CDN or reverse proxy keys on the URL and whatever it has been told to vary on; a cookie it was never told about is invisible to it. Put variant-specific HTML behind such a cache without varying on the assignment and every visitor gets whichever variant was rendered first — an experiment that measures nothing while appearing to run.

It is not a credential. The value is client-controlled: a visitor can set abt=premium:B as easily as reading it. That is harmless when a variant selects a layout and unacceptable when someone uses the same mechanism to gate a capability. Entitlements are a server-side authorization decision, which is Part 15’s subject; experiment assignment is a rendering decision that happens to be persisted client-side.

Reusing the Assignment During Hydration

The composable reads from Nuxt’s request-scoped state, which Nuxt serializes into the payload. During hydration the browser therefore finds the map rather than producing one, and the one rule for anything downstream is that nothing recomputes it. No client plugin re-hashing the identifier, no component defaulting to 'A' when the map is unexpectedly empty — a default is a silent disagreement, and failing loudly in development is more useful.

This is a specific instance of the general rule Part 12 develops: a decision made on the server is transferred, not re-derived. Experiment assignment is the clearest case because the cost of getting it wrong is both visible (flicker) and invisible (biased data) at the same time.

Test Definitions as Content

Experiments are defined as CMS entries rather than in application code, so that starting, scheduling and ending one is not a release:

The application loads the active configuration and refreshes it on a publish webhook, which inherits the properties Part 9 described for redirects: the refresh is per worker and not atomic, so “the test is live” means every worker has processed it. A worker that missed the message keeps assigning according to the previous configuration, which shows up as a split that does not match the intended one — see the sample-ratio check below.

The activation window has the same interaction with caching that Part 10’s date-range conditions do: it changes the correct behaviour without any event occurring in the system. A test that is supposed to start at midnight starts when the caches holding pre-midnight renders expire. That is fine if everyone knows it, and it is a support ticket if they do not.

Cache Identity and Variants

This is the part that most repays precision, because the honest answer is “it depends on what the variant changes.”

If a variant only decides which components render, and both variants consume the same content, then the data cache from Part 9 — query results keyed by URL, locale and preview mode — can be shared between variants. It stores content, not markup, and the content is genuinely identical. That is a real efficiency, and it is worth designing experiments to stay on this side of the line where possible.

The line is crossed in three situations, and each one forces the assignment into a cache identity:

• The variant changes which data is fetched — a different query, a different collection, a different ordering. Then the content cache key must include the assignment, or variant B renders variant A’s data.
• Rendered HTML is cached anywhere: a page cache, a reverse proxy, a CDN. Markup differs per variant by definition, so that cache must vary on the assignment or be bypassed for pages under test.
• A Part 10 condition consumes the assignment. Then the visibility of a section depends on it, which makes the rendered output variant-specific even when the data is not.

The practical rule: the assignment belongs in every cache identity that spans a decision the assignment influenced. Writing that sentence down for a specific page takes two minutes and prevents the most embarrassing possible outcome, which is an experiment whose results are an artefact of a cache.

Forcing a Variant for QA

Nobody can test variant B by refreshing until randomness cooperates, so a query parameter overrides the assignment — ?abt=layout:B — and the middleware merges it into the cookie so the override persists across subsequent navigation. A debug panel, gated to non-production per Part 6, lists the active tests, the current assignment and a control to switch, which is how QA verifies both variants deliberately.

The consequence to plan for is data quality. Overridden sessions are not random samples; they are developers and testers repeatedly viewing one variant, often on internal networks, often without converting. If those sessions reach the same analytics stream as real traffic, they contaminate the result of every experiment being tested. Mark the session as overridden when the parameter is used and exclude such sessions from experiment analysis — and be aware that the override lives on in a cookie, so a colleague who forgot they pinned variant B a month ago is still pinned.

Measuring the Experiment Is a Separate Discipline

Rendering the right variant reliably is what this article solves. Deciding which variant is better is a different problem, and the architecture only contributes the inputs: an exposure event recording that this visitor saw this variant of this test, emitted once per relevant view, and the conversion events already collected.

The rest is method, and the parts that matter most are the ones easiest to skip:

• Decide the sample size, the minimum effect worth detecting, and the stopping rule before starting. Elapsed time does not establish significance, and neither does a dashboard that currently shows a difference. Checking repeatedly until a difference appears will eventually produce one from noise.
• Check the observed split against the intended one. A 50/50 test delivering 54/46 exposures signals something mechanical: stale configuration on some workers, a cache serving one variant more often, bot traffic assigned but never converting. A sample ratio mismatch invalidates the comparison regardless of how good the conversion numbers look, and it is the check most likely to catch the bugs this article warns about.
• Exclude overridden and non-human traffic, and confirm the exposure event fires for both variants under the same conditions — an event emitted in one branch and forgotten in the other produces a beautifully clean and completely false result.

When a decision is made, the lifecycle stays in the CMS: send all traffic to the winner, or deactivate the test if the existing behaviour won. Neither requires a deployment, which was the point of defining tests as content — while the losing variant’s component eventually should be deleted, which does.

What This Design Guarantees

Concretely: every visitor sees one variant per test, chosen once per request, identical in the server-rendered HTML and after hydration, persistent across navigation, overridable for QA, and recorded in analytics as an exposure.

What it does not guarantee is that your experiment is valid. Caches can leak a variant across visitors if their identities are wrong, configuration propagation is eventual rather than instant, overrides pollute data unless excluded, and the statistics are only as good as the plan made before the test started. The architectural part of A/B testing is finished when the rendering is stable; the difficult part is the conclusion someone draws, and no amount of SSR correctness makes an underpowered test conclusive.

The diagnostic I would run on any SSR experiment before trusting its numbers: request a page twice with the same assignment cookie and confirm the HTML matches, request it with different assignments through every cache in front of the application and confirm the responses differ, then compare the observed exposure split to the configured one. If all three hold, the mechanism is sound and the remaining risk is statistical.

What’s Next

• Part 12: SSR and Hydration — The general form of the disagreement this article avoided, and how to debug it.
• Part 13: Performance Optimization Across the Stack — Where the time in a server-rendered page actually goes.

Munir Husseini is a software architect specializing in full-stack TypeScript, .NET, and cloud-native architectures.