A/B Testing at the SSR Level — One Assignment per Request
Photo by Francesco Ungaro on Pexels
Part 10’s visibility rules happily consumed an experiment assignment without ever saying where it came from. So this article supplies it, and deals with the failure that makes experiments on a server-rendered site harder than on a single-page application: the server and the browser can disagree about which variant the visitor is in.
The examples use the fictional document application. The statistical guidance at the end is deliberately conservative, and it isn’t a report of any measured result.
Table of Contents
- Two Renderers Can Disagree About the Variant
- Assign Once Before Anything Renders
- Why a Cookie and What It Does Not Do
- Reusing the Assignment During Hydration
- Test Definitions as Content
- Cache Identity and Variants
- Forcing a Variant for QA
- Measuring the Experiment Is a Separate Discipline
- What This Design Guarantees
- What’s Next
Two Renderers Can Disagree About the Variant
In a client-rendered application, an experiment is about three lines of code: read a flag, pick a variant, render. There’s one renderer, so there’s nothing around to disagree with.
Server-side rendering takes that comfort away. The HTML gets produced on the server, which means the variant has to be chosen before any component renders at all. And then the same components run again in the browser during hydration, where, if they choose the variant by the same procedure instead of reading the server’s choice, they can perfectly well choose differently. A random assignment evaluated twice is two assignments. Even a deterministic one can come out differently if it depends on anything the browser sees differently: the current time, a cookie the server wrote into the response that isn’t readable yet, the viewport.
What you see as a result is a flicker as the layout switches under the visitor, a hydration mismatch warning, or the quieter and considerably worse outcome where the DOM keeps the server’s markup while the component’s state believes something else entirely. The measurement consequence is worse still: the visitor got counted in one variant and shown the other, which doesn’t just add noise, it biases the result.
So the requirement is narrow and strict: one assignment per request, made before rendering, and reused rather than recomputed in the browser. Everything below follows from that one sentence.
Assign Once Before Anything Renders
Assignment happens in server middleware, before any of the page’s components run. For each active test, the middleware works out the visitor’s variant, stores the complete map in request-scoped state, and makes it available through a composable:
Components then consume it without knowing a thing about any of the above:
const { variant } = useAbTest('layout')
<DocumentLayoutA v-if="variant === 'A'" />
<DocumentLayoutB v-else />
There are two details in that assignment step worth deciding explicitly.
Prefer a deterministic assignment to calling a random number generator. Deterministic here means the variant gets computed from values that don’t change: hash the visitor’s stable identifier together with the test id, take the hash modulo 100 to get a number from 0 to 99, and read the variant off the configured split, so 0–49 is A and 50–99 is B for an even split. Including the test id in the hash is what stops the same visitor landing in bucket 3 of every single experiment, which would leave two concurrently running tests overlapping in exactly the same population. Deterministic assignment gives you the same answer if it ever does get computed twice, which turns a whole class of the disagreements above into non-events, and it makes the assignment reproducible when you’re investigating one visitor’s session. Also, assign every active test at once at request start, rather than lazily when some component asks for one. Lazy assignment means the set of assignments depends on which components happened to render, which makes your exposure data depend on page structure.
Why a Cookie and What It Does Not Do
We persist the assignment in a cookie, something like abt=layout:A,teaser:B, for three good reasons. It’s present on the request, so the server can read it before rendering, which a server-side session lookup can’t promise you without a round trip. It survives page loads, so a visitor doesn’t get reassigned on every navigation. And it needs no server-side state at all, so any number of replicas behave identically without sharing a store between them.
Now, two things a cookie doesn’t do, both of which have caught people out.
It doesn’t make a shared HTTP cache serve different content. A CDN or reverse proxy stores one response per cache key, and that key is the URL plus whatever the response told it to distinguish on, typically via the Vary header or an explicit rule in the CDN’s configuration naming the cookie or header to split on. A cookie nobody mentioned is simply invisible: as far as the cache is concerned, two requests for /documents/42 are the same request. Put variant-specific HTML behind a cache like that without varying on the assignment and every visitor gets whichever variant happened to render first. That’s an experiment which measures precisely nothing while appearing to run beautifully.
And it isn’t a credential. The value is client-controlled, so a visitor can set abt=premium:B just as easily as they can read it. That’s entirely harmless when a variant picks a layout and completely unacceptable when somebody uses the same mechanism to gate a capability. Entitlements are a server-side authorization decision, which is Part 15’s subject. Experiment assignment is a rendering decision that happens to be persisted client-side.
Reusing the Assignment During Hydration
The composable reads from Nuxt’s request-scoped state, which Nuxt serializes into the payload. So during hydration the browser finds the map rather than producing one, and the single rule for everything downstream is that nothing recomputes it. No client plugin re-hashing the identifier, and no component quietly defaulting to 'A' when the map turns out to be empty, because a default is a silent disagreement and failing loudly in development is far more useful.
This is one specific instance of the general rule Part 12 develops: a decision made on the server gets transferred, not re-derived. Experiment assignment is the clearest case of all, because the cost of getting it wrong is visible (the flicker) and invisible (the biased data) at the same time.
Test Definitions as Content
We define experiments as CMS entries rather than in application code, so that starting, scheduling and ending one isn’t a release:
| Field | Example | Purpose |
|---|---|---|
testId |
layout-display |
Stable identifier used in assignment and analytics |
variants |
["A", "B"] |
Available variants |
trafficSplit |
[50, 50] |
Share per variant |
startDate / endDate |
2026-06-01 / 2026-07-01 |
Activation window |
targetPages |
["/documents/*"] |
Where the test applies |
The application loads the active configuration and refreshes it on a publish webhook, which inherits all the properties Part 9 described for redirects: the refresh is per worker and it isn’t atomic, so “the test is live” really means every worker has processed it. A worker that missed the message carries on assigning according to the previous configuration, which shows up as a split that doesn’t match the intended one. See the sample-ratio check below.
The activation window interacts with caching exactly the way Part 10’s date-range conditions do: it changes the correct behaviour without any event happening in the system at all. A test that’s supposed to start at midnight actually starts when the caches holding pre-midnight renders expire. Which is fine if everybody knows that, and is a support ticket if they don’t.
Cache Identity and Variants
This is the part that repays precision the most, because the honest answer is “it depends on what the variant changes”.
If a variant only decides which components render, and both variants consume exactly the same content, then the data cache from Part 9, meaning query results keyed by URL, locale and preview mode, can be shared between the variants quite safely. It stores content rather than markup, and the content genuinely is identical. That’s a real efficiency, and it’s worth designing your experiments to stay on this side of the line wherever you can.
You cross that line in three situations, and every one of them forces the assignment into a cache identity:
- The variant changes which data gets fetched, so a different query, a different collection, a different ordering. Then the content cache key has to include the assignment, or variant B ends up rendering variant A’s data.
- Rendered HTML is cached anywhere at all: a page cache, a reverse proxy, a CDN. Markup differs per variant by definition, so that cache has to vary on the assignment or be bypassed entirely for pages under test.
- A Part 10 condition consumes the assignment. Then the visibility of a section depends on it, which makes the rendered output variant-specific even where the data isn’t.
So the practical rule is that the assignment belongs in every cache identity spanning a decision the assignment influenced. Writing that sentence down for one specific page takes about two minutes and heads off the most embarrassing outcome available here, which is an experiment whose results turn out to be an artefact of a cache.
Forcing a Variant for QA
Nobody is going to test variant B by refreshing until randomness finally cooperates, so a query parameter overrides the assignment, ?abt=layout:B, and the middleware merges it into the cookie so the override survives subsequent navigation. A debug panel, gated to non-production per Part 6, lists the active tests, the current assignment and a control to switch between them, and that’s how QA verifies both variants deliberately.
The consequence you have to plan for here is data quality. Overridden sessions aren’t random samples at all. They’re developers and testers viewing one variant over and over, often on internal networks, and often without converting. If those sessions reach the same analytics stream as real traffic, they contaminate the result of every experiment being tested. So mark the session as overridden whenever the parameter gets used and exclude such sessions from experiment analysis. And bear in mind that the override lives on in a cookie, so a colleague who forgot they pinned variant B a month ago is still pinned today.
Measuring the Experiment Is a Separate Discipline
Rendering the right variant reliably is what this article solves. Deciding which variant is actually better is a completely different problem, and the architecture only contributes the inputs: an exposure event recording that this visitor saw this variant of this test, emitted once per relevant view, plus the conversion events you already collect.
The rest is method, and the parts that matter most are unfortunately the easiest ones to skip:
- Decide the sample size, the minimum effect worth detecting, and the stopping rule before you start. Those three are linked together: the smallest difference you care about determines how many visitors you need in order to tell it apart from noise, and the stopping rule is your commitment to run until you’ve got them. Skipping the arithmetic is how a test ends up underpowered, meaning too few visitors to detect the effect it was looking for, so a genuine improvement reads as “no difference”. Skipping the stopping rule is worse in the opposite direction: if you check the dashboard every morning and stop the day it shows a winner, you will find winners in pure noise, because a difference large enough to look convincing turns up by chance fairly often in a small sample. Elapsed time on its own establishes nothing whatsoever.
- Check the observed split against the intended one. A 50/50 test delivering 54/46 exposures is telling you something mechanical: stale configuration on some workers, a cache serving one variant more often than the other, bot traffic that gets assigned but never converts. A sample ratio mismatch invalidates the comparison no matter how lovely the conversion numbers look, and it’s the check most likely to catch the bugs this article has been warning about.
- Exclude overridden and non-human traffic, and confirm that the exposure event fires for both variants under the same conditions. An event emitted in one branch and forgotten in the other gives you a beautifully clean and completely false result.
Once a decision has been made, the lifecycle stays in the CMS: send all the traffic to the winner, or deactivate the test if the existing behaviour won. Neither of those needs a deployment, which was the whole point of defining tests as content. Deleting the losing variant’s component eventually does need one, mind you.
What This Design Guarantees
Concretely, this: every visitor sees one variant per test, chosen once per request, identical in the server-rendered HTML and after hydration, persistent across navigation, overridable for QA, and recorded in analytics as an exposure.
What it doesn’t guarantee is that your experiment is valid. Caches can leak a variant across visitors if their identities are wrong, configuration propagation is eventual rather than instant, overrides pollute your data unless you exclude them, and the statistics are only ever as good as the plan somebody made before the test started. The architectural part of A/B testing is finished once the rendering is stable. The hard part is the conclusion somebody draws, and no amount of SSR correctness will make an underpowered test conclusive.
Here’s the diagnostic I’d run on any SSR experiment before trusting its numbers. Request a page twice with the same assignment cookie and confirm the HTML matches. Request it with different assignments through every cache sitting in front of the application and confirm the responses differ. Then compare the observed exposure split to the configured one. If all three of those hold, the mechanism is sound and the risk that’s left is statistical.
What’s Next
- Part 12: Hydration Mismatches — The general form of the disagreement this article avoided, and how to debug it.
- Part 13: Kinds of Waiting — Where the time in a server-rendered page actually goes.
Munir Husseini is a software architect specializing in full-stack TypeScript, .NET, and cloud-native architectures.
Category: Nuxt in Production