Sooner or later somebody is going to ask whether the new stack is faster than the one it replaced. It’s a perfectly fair question, and answering it honestly turns out to be harder than running a load test, because the two systems were measured by different tools, at different boundaries, over different populations of requests.

So this article is about the method, and about what a result of this kind can and can’t support. I’ve deliberately left out traffic distributions, route names, exact measurements and resource allocations. The observations appear as broad ranges rather than as a fingerprint of any particular deployment.

Table of Contents

A Fast Response on a Laptop Proves Nothing

A developer’s document page renders quickly on a warm cache, with one visitor, no competing renders, a local dependency and no proxy sitting in front of it. Every single one of those conditions is doing work that production will not be doing for you.

So validating a migration needs three things, and they’re usually assembled in the wrong order: a workload that resembles reality, acceptance criteria written down before the run, and statistics defined precisely enough that a comparison actually means something. The third is where most published performance claims fall apart, so it goes first.

Define the Statistic Before Computing an Improvement

Metric What it means The usual mistake
Median request duration The middle observation of the request-duration distribution Calling an average of per-page medians “the median”
P95 The boundary below which about 95% of observations fall Treating it as an upper bound on the slowest 5%
Error rate Failures divided by requests in the same population Comparing one route’s historical rate with an all-route result
Delivered throughput Work actually completed per unit time Reporting the requested arrival rate as achieved capacity
Maximum tested load The highest load this test exercised Calling it the system’s capacity

Three of those rows deserve expanding on, because they’re the ones that produce confidently wrong statements.

P95 is a boundary, not a bound. “95% of requests were faster than some threshold” tells you absolutely nothing about the remaining 5%. They might be marginally over it or thirty times worse, and the p95 reads identically either way. It tells you where the tail starts rather than how bad the tail gets, which is why a p99 and a maximum are worth reporting right alongside it.

Delivered throughput is what completed, not what you asked for. A load generator configured for 500 requests per second only achieves that if the system keeps up. When it doesn’t, responses take longer, the generator’s virtual users are still sitting there waiting on their previous request, and the rate actually applied falls below your target. Reporting the configured number as though it were achieved capacity describes the test’s intent rather than the system’s behaviour, and that’s why the dropped-iteration count matters every bit as much as the latency figures.

A traffic-weighted average of per-page medians is a genuinely useful summary:

sum(page median × page traffic weight) / sum(page traffic weight)

It is not the median of the combined request population, and you can’t relabel it as one. A small example shows why. Suppose one page takes 90% of the traffic and is fast, while another takes 10% and is ten times slower. The weighted average sits noticeably above the fast page’s median, because the slow page’s median drags it upward. The median of the combined observations doesn’t move nearly as far, because 90% of the individual requests are still fast ones. A median counts observations; a weighted average of medians counts pages. Group medians simply don’t carry the information you’d need to reconstruct the distribution they came from, so this number should always travel with its definition attached to it.

And an error rate is a property of a population. A historical figure computed over one route class and a test result computed over all routes are two different quantities. Divide one by the other and you get a ratio with no meaning at all, however impressive it looks.

Why the Headline Ratio Is Not Available

This is the part no stakeholder enjoys hearing, and it’s the honest position.

Historical production aggregates and a fresh load-test run measure different things. Server-side processing duration, client-observed HTTP duration and the total time of a multi-request navigation are three separate boundaries, and the old system’s records and the new system’s test typically don’t share any one of them. Nor do they share an aggregation method, a traffic mix, a cache state or a measurement window.

So this series doesn’t claim a whole-system speedup ratio or an error-rate reduction factor out of those mixed summaries. What’s actually available is described below: broad ranges for the scale of the observations, and behaviour under tested load. A qualitative statement you can defend is worth considerably more than a precise-looking number you can’t, because that number only survives until the first person asks how it was calculated.

And if a like-for-like ratio genuinely matters to somebody, then it has to be produced deliberately: the same statistic, at the same boundary, over the same route population, on both systems, ideally by running the same harness against both of them.

Modelling Traffic Without Publishing It

The load scripts select from a bounded set of routes using weights derived from historical traffic. That gives us synthetic HTTP traffic informed by production observations, which is a rather different thing from replaying real browser sessions.

flowchart LR M["Private workload model"] --> S["Weighted route selection"] S --> A["Frequently visited document pages"] S --> B["Less frequent collection views"] S --> C["Infrequent paths"] A --> H["HTTP requests"] B --> H C --> H

Keep the actual route list and the weights private, because they describe a business rather than a technique. For internal reproducibility, store everything needed to repeat the run alongside the test artefact: test revision, target release, runtime versions, cache state, region, load-generator placement, authentication mode and workload parameters. A result whose conditions nobody recorded can’t even be compared with its own successor.

And one scope limitation to state up front: a GET-only test doesn’t exercise writes at all. State-changing operations need separate, authorized scenarios, and they must not run destructively against real data.

Sustained and Stepped Runs

Two run shapes answer two different questions here.

A sustained run ramps up to a chosen arrival rate and then holds it, which is how you find out whether the system is stable at a level resembling normal traffic. The warm-up policy belongs in the report, because it determines cache state, and given how much of this architecture leans on the caches from Parts 3 and 9, a run against a fully warm cache and a run against a cold one are measuring two different systems. Both are legitimate. Conflating them is not.

A stepped run increases offered load in stages with a hold at each level:

flowchart LR W["Warm-up, declared policy"] --> B["Baseline hold"] B --> I["Increase offered load"] I --> H["Hold and observe"] H --> D{"Acceptance criteria<br/>still met?"} D -->|Yes| I D -->|No| S["Stop escalating and investigate"]

The holds are the valuable part of that. They show you whether the system actually settles after a step, with new replicas started, caches refilled and queues drained, or whether it merely survived the transient. A ramp with no holds in it tells you very little indeed.

Two things a stepped run doesn’t establish. Stopping at some chosen multiple of baseline because the criteria still passed doesn’t locate a ceiling. It locates where you stopped. And a fifteen-minute step is not a soak: retention effects and cache-eviction behaviour need a longer sustained run, which is the same point Part 17 made about memory conclusions drawn from short windows.

What the Runs Support

Stated at the level this series is able to publish: the observations span seconds-scale response summaries for the legacy stack and hundreds of milliseconds for many requests in the revised stack, and the revised application was exercised at several multiples of the selected baseline while meeting the criteria then in force.

Both of those statements are deliberately modest. The first describes the scale of the observations without asserting any like-for-like ratio, for the reasons above. The second describes tested behaviour for one workload on one configuration, and not a capacity multiplier over the previous system, which was never run through the same harness at all.

The most transferable finding here is methodological: a stable median can coexist quite happily with a degrading tail. Most requests can stay fast while a minority wait through a cold cache, a slow dependency, a retry chain or a worker startup, and the users in that minority are exactly the ones who write support tickets. A result reported only as a median can hide precisely the population that motivated the work in the first place.

Tails Failures and Thresholds

So report the whole set rather than the headline: median, tail percentiles, failure rate, delivered throughput, and dropped iterations, meaning the ones the generator couldn’t issue, which are how you tell “the system handled the load” apart from “the load generator gave up”.

And look at route-level distributions as well as the aggregate. A consistently slow page with a small traffic weight vanishes into a weighted summary, and it’s still a bad page.

Two warnings about apparent passes. Exploratory scripts usually carry permissive thresholds, so a zero exit code means “the thresholds configured for exploration were met”, which is not a production-readiness statement. Release criteria should come from user needs and error budgets, and they should live separately from your stress-test limits. And an HTTP success status doesn’t mean a useful page: a sign-in redirect, an error page or a partial render can all return 200-range responses, which is exactly the weakness Part 16 identified in a smoke check. If content correctness is part of the test’s purpose, then assert content.

Sizing Experiments in the Same Harness

The load harness is also the thing that makes Part 17’s sizing procedure possible at all, because changing a memory or worker setting only means anything under a repeatable workload:

flowchart TB BL["Stable allocation with headroom"] --> CH["Change one resource or worker setting"] CH --> RN["Run the same workload"] RN --> CM["Compare tail latency, errors,<br/>memory and restarts"] CM --> OK{"Within acceptance criteria?"} OK -->|Yes| SK["Soak and exercise reloads"] OK -->|No| DG["Restore baseline and diagnose"]

What experiments like that support is a trade-off rather than a number: reducing memory and CPU improves your utilization right up until garbage-collection pressure, request queuing or slower startup starts undermining reliability. There’s no universal minimum container size for an SSR application, and anybody quoting you one is describing their own component tree.

And keep the three memory quantities firmly distinct while you’re doing this, so container working set, process resident set and V8 heap, because a percentage of container memory is not heap utilization, and attributing a failure to memory pressure means correlating GC and probe events rather than pattern-matching a graph.

Browser Performance Is a Different Experiment

HTTP load scripts don’t execute page JavaScript, don’t hydrate components, and don’t fetch subresources the way a browser does. So they can’t measure anything at all from Parts 13 and 14: deferred hydration, font behaviour, image delivery, interaction readiness. A load test that got faster says precisely nothing about whether the revision switcher responds any sooner.

Those questions need browser measurements, with a declared device profile, throttling method and cache state, and repeated rather than run once. And a lab audit stays a lab observation: field responsiveness and accessibility are separate assessments rather than corollaries of a score.

The parity rule from Part 13 applies here as well. If functionality is disabled for audit user agents, then the resulting number describes a configuration no visitor experiences, and it shouldn’t be published as though it did.

Cost Is a Different Experiment

Smaller application replicas don’t produce a proportionally smaller bill. Dedicated node capacity has a minimum of its own, retained revisions for rollback occupy resources, and managed caches, monitoring, networking and storage are all line items that don’t scale with replica count.

If cost is the question being asked, then compare actual spend over equivalent periods and workloads. And for publication, the useful content is the trade-off, so baseline capacity against scaling headroom against operational complexity, rather than a figure.

Reporting Rules

Before you publish any measurement from work like this:

  1. State the metric definition, and confirm that the populations being compared really are the same.
  2. Keep the exact evidence in private records rather than in public screenshots or attachments.
  3. Use broad ranges or orders of magnitude consistently, in the text, the tables and the diagrams alike.
  4. Remove exact workload distributions, allocation combinations and identifiable route names.
  5. Label synthetic examples as examples rather than as observations.
  6. Say what you didn’t measure, and avoid causal claims without a controlled comparison.

The rule underneath all six of those: a credible performance result is not the most dramatic ratio available to you, it’s the statement whose scope and limitations a reader can go and check. “The revised stack served several multiples of baseline traffic with stable tail latency under these conditions, and we did not run a like-for-like comparison against the old system” is a sentence that survives scrutiny. A single impressive multiplier usually isn’t.

What’s Next

Part 20: The Full Picture goes back to Part 1’s one-field change and asks whether all the machinery built up over these twenty articles was actually worth owning.

Munir Husseini is a software architect specializing in full-stack TypeScript, .NET, and cloud-native architectures.