Get started

Taking an assessment rather than buying one? This page is written for employers. Here is the page for candidates.

Quality engineering and test · Senior level

How to assess a Performance Test Engineer

Performance testing is the role where the wrong answer looks most like a result. A tool-recall interview establishes that a candidate can drive the load generator, which is the easy half; every expensive failure in this role is a failure of experimental design, and it produces a confident number rather than an error. Load-test a system with a warm cache and one repeated user identity and you will measure the cache. Run the generator from one undersized client and you will measure the client. Use a test dataset of ten thousand rows against a production table of forty million and you will measure a query plan that does not exist in production. Report the mean when the business cares about the slowest one percent and you will report success on a system that is failing its worst-affected users. In each case a report is delivered, a number is quoted, a release is approved, and the defect ships — and unlike the QA analyst's missed defect or the automation engineer's flaky suite, this failure actively supplies false confidence rather than merely withholding assurance. No screen that asks for a script observes any of it, and an assistant writes a clean load script instantly, so the artefact carries no signal.

Performance testing is an experimental science practised by people usually hired as tool operators. The tool is a week's learning. The science — constructing a load that resembles reality closely enough that the result means something, and knowing when it does not — takes years, and it is the entire value of the hire.

The first thing a strong performance engineer does is refuse to start. They ask what the traffic actually looks like: how many users, arriving how, doing what in what proportion, with what think time, hitting what proportion of cold versus warm data, from where. A load model built without those answers produces a precise measurement of something nobody cares about. The most common version of this failure is the single-user-identity test, which pulls one row into cache and then measures the cache at ten thousand requests per second — a spectacular result and a meaningless one.

The second is instrumenting both ends. A load test that only reports client-side latency cannot distinguish a saturated database from a saturated test client from a network limit, and the default assumption is always that the system under test is at fault. A strong engineer watches CPU, connection pools, garbage collection and disk on the target while watching the generator's own resource use, and will happily report "I could not push it hard enough to find the limit" rather than report a limit that was theirs.

The third is reading the right statistic. Averages hide the failure. If the mean is 180 milliseconds and the 99th percentile is nine seconds, one user in a hundred is having an experience that loses accounts, and the mean says everything is fine. Tail latency is where performance work meets reality, and a candidate who talks in percentiles without prompting is telling you they have watched a real system degrade. The field data supports treating this as consequential rather than academic: HTTP Archive's 2024 Web Almanac, drawing on Chrome UX Report field measurements, found 43 percent of mobile loads and 54 percent of desktop loads achieving good Core Web Vitals, with 59 percent of mobile loads passing LCP. Most real systems are slower than their owners believe, largely because their owners are measuring in the lab and on the average.

The fourth competence is the one that determines whether the work has any effect: turning a result into a decision. A performance report that says the system degrades non-linearly above 400 concurrent users is data. A performance report that says the current infrastructure supports the launch if traffic stays below the projected peak, becomes unpredictable above it, and can be made safe for roughly this cost by changing this one thing, is a decision. Uptime Institute's 2026 outage analysis found 57 percent of surveyed operators saying their most recent major outage cost more than $100,000 and around one in five above $1 million — which is the currency in which a capacity finding should be expressed if anyone is going to act on it.

The assessment is built to catch design errors rather than tooling errors. The sandbox provides a running service, an existing and deliberately flawed load script, a dataset far smaller than the described production one, and a monitoring view showing both the target and the generator. There is a bottleneck, and there is a second, false bottleneck in the harness. The candidate investigates and writes the report a delivery manager will read. Then the conversation: your throughput plateaued at 900 requests per second — what convinced you that was the service and not your own client? Your script uses one login for every virtual user; what does that do to the result? The 99th percentile is nine seconds and the mean is fine — which number goes in the summary, and why? We launch in two weeks; given what you found, what is your recommendation and what is the confidence in it? An engineer who designed the experiment answers with the reasoning that produced their own numbers. One who ran a tool has only the numbers.

What the job actually needs

How people fail in this seat

What most employers do instead

CV screen for JMeter, k6, Gatling or LoadRunner, an interview on tool configuration and the definition of throughput versus latency, and occasionally a request to produce a sample test script.

Performance testing is the role where the wrong answer looks most like a result. A tool-recall interview establishes that a candidate can drive the load generator, which is the easy half; every expensive failure in this role is a failure of experimental design, and it produces a confident number rather than an error. Load-test a system with a warm cache and one repeated user identity and you will measure the cache. Run the generator from one undersized client and you will measure the client. Use a test dataset of ten thousand rows against a production table of forty million and you will measure a query plan that does not exist in production. Report the mean when the business cares about the slowest one percent and you will report success on a system that is failing its worst-affected users. In each case a report is delivered, a number is quoted, a release is approved, and the defect ships — and unlike the QA analyst's missed defect or the automation engineer's flaky suite, this failure actively supplies false confidence rather than merely withholding assurance. No screen that asks for a script observes any of it, and an assistant writes a clean load script instantly, so the artefact carries no signal.

The assessment

About 67 minutes end to end.

The systems it runs in

A containerised service under load from a script the candidate did not write, with dashboards over both the system under test and the load generator's own host, so the generator's saturated CPU is observable to anyone who thinks to look at it rather than hidden by the fixture. The raw per-request distribution in t3 is the run's own output — a CSV or summary export with timestamps — analysed by the candidate rather than handed to them pre-summarised, which is what makes the warm-up exclusion a decision they have to disclose.

This is the one role in this batch where no dominant tool can honestly be asserted, and the file says so rather than picking one and implying it is the standard. No survey with a published methodology establishes a single load generator as the industry default: JMeter is the most widely deployed in enterprise estates, and k6, Gatling and Locust are what developer-owned pipelines are built on. The corpus will not invent a share for any of them. The fixture therefore ships as both a JMeter test plan and a k6 script and the buyer chooses; LoadRunner or NeoLoad buyers supply their own script and the fixture is rebuilt around it. Nothing in the design depends on the choice, because the false plateau is a property of the generator's host rather than of the tool running on it.

Working speed is not scored. The correct result in t2 is the absence of a number. A candidate who reports being unable to push the system hard enough to find its limit is right, and one who quotes the plateau has produced a confident wrong figure faster and with less work. A speed criterion would rank false confidence above honest inconclusiveness, which is the exact failure this role exists to prevent — and it supplies false assurance rather than merely withholding it, which is worse. Time to first result is not reported at all.

What the candidate actually does

TaskWhat happens
The intake
live_call · 8 min		A live call opening the engagement. An AI delivery manager says the launch is in two weeks and asks the candidate to confirm the system can handle the load. He volunteers one number, roughly five thousand users, and does not say whether that is registered, daily, or concurrent. He has answers to most of the questions that matter — arrival pattern, the mix of actions, think time, how much of the data is cold, where traffic originates, what response time the business would consider failure — but he supplies them only when asked, and one of his answers is a guess he presents with the same confidence as the facts.
The fork. Accepting the number and going away to test produces a precise measurement of something nobody specified. Establishing the load model before touching a generator, and identifying which of his answers is a guess that needs checking against real traffic data, is the work.
The harness that lies
coding_sandbox · 25 min		A monitored sandbox with a running service, an existing load script written by someone else in the buyer's own tool, a dataset far smaller than the production one the delivery manager described, and dashboards covering both the target system and the host the load generator runs on. The script authenticates every virtual user as the same account and has no think time. There is a genuine bottleneck in the service, an unindexed lookup that only degrades above a data volume the sandbox dataset does not reach. And there is a second, false bottleneck — the generator saturates its own CPU, so the throughput curve flattens at a number that is a property of the test client and not of the system under test.
The fork. The plateau is a clean, confident, quotable number and reporting it as the system's capacity is the failure this role exists to prevent, because it supplies false confidence rather than merely withholding assurance. Watching the generator's own resource use, noticing it is the thing that ran out, and reporting an inability to find the limit is the correct result and looks like a worse one.
Which number goes in the summary
data_task · 10 min		The raw per-request latency distribution exported from a completed run, with timestamps and no summarisation. The mean is comfortable. The upper percentiles are not, and the first minute of the run contains a cold-cache spike that shifts several statistics if it is left in. The candidate produces the summary figures they would put in front of the business and says what they excluded and why.
The fork. Reporting the mean and the maximum is conventional, quick, and hides both the tail experience and the reason the maximum exists. Reporting the upper percentiles, stating the warm-up exclusion explicitly, and translating the tail into how many real users per hour that represents, is the version that changes a decision.
The capacity recommendation
written_artifact · 12 min		The one-page document the delivery manager and his director will read. It carries a recommendation about the launch, the evidence for it, and the limits of that evidence.
The fork. A number with no confidence attached will be quoted back for a year as though it were established. A recommendation that says what the experiment could establish, what it could not because the dataset was unrepresentative and the generator was undersized, and what would have to be done to close each gap, is harder to write and is the only honest artefact available given what the session actually measured.
Was that the system or your client
live_call · 12 min		A conversation about the candidate's own run. Your throughput plateaued — what convinced you that was the service and not your own client? Your script uses one login for every virtual user; what does that do to the result, and in which direction? Your dataset is a fraction of production — which of your findings survives that and which does not? The launch is in two weeks and my director wants a yes or a no; give me one, with your confidence.

The mark scheme

Each criterion is scored 1 to 5 against written anchors, and every score is reported with the excerpt that earned it. A criterion marked floored is reported as a finding rather than averaged into the total. The first is open; open any other to read its anchors in full.

Builds a load model before generating loadweight 0.25Establishes arrival pattern, mix, think time, data volume and the response threshold the business would call failure; identifies which supplied answer…
1 Accepts the volunteered user count, begins testing, and never establishes arrival rate, action mix, think time or cache state.
3 Asks several of the right questions and assembles a partial model, but does not challenge the guess presented as a fact or state what the model is missing.
5 Establishes arrival pattern, mix, think time, data volume and the response threshold the business would call failure; identifies which supplied answer is a guess, says how it would be checked against real traffic data, and states what the test will therefore not represent.
Suspects the instrument before the systemweight 0.25Watches both ends throughout, identifies the generator as the binding constraint, states plainly that the system's limit was not found, and either rem…
1 Reports the throughput plateau as the system's capacity, with no reference to the generator's own resource use.
3 Monitors the generator, notices something is wrong with the ceiling, but cannot separate the client limit from the service limit and reports both as one uncertain figure.
5 Watches both ends throughout, identifies the generator as the binding constraint, states plainly that the system's limit was not found, and either removes the constraint or reports the inability as the finding rather than dressing it as a result.
Reports the distribution rather than its centreweight 0.2Reports upper percentiles, declares the warm-up exclusion and its justification explicitly, and translates the tail into the number of real users an h…
1 Summarises with the mean, or the mean and the maximum, and does not mention warm-up or exclusions.
3 Reports upper percentiles correctly but leaves the cold-cache period in without comment, or excludes it without saying so.
5 Reports upper percentiles, declares the warm-up exclusion and its justification explicitly, and translates the tail into the number of real users an hour who would experience it.
Attaches confidence and stated limits to every numberweight 0.2Says for each conclusion whether the experiment supports it, names the unrepresentative dataset and the undersized generator as the specific reasons f…
1 Presents figures as established capacity with no qualification, so a reader would reasonably quote them as fact.
3 Qualifies the findings generally, with caveats that do not identify which specific conclusions are and are not supported.
5 Says for each conclusion whether the experiment supports it, names the unrepresentative dataset and the undersized generator as the specific reasons for the gaps, and states what would close each and roughly what it would take.
Produces a result someone can act onweight 0.1Gives a clear recommendation on the launch, the conditions under which it holds, the single change that would most improve the position, and what it c…
1 Delivers data with no recommendation, or a recommendation with no stated consequence of being wrong.
3 Makes a recommendation but does not connect it to a decision the business is actually taking.
5 Gives a clear recommendation on the launch, the conditions under which it holds, the single change that would most improve the position, and what it costs to be wrong.

How it is scored

Weighted mean of the five criteria, each scored 1 to 5 against the anchors, reported with the excerpt that earned it. One observation is reported separately and is not scored on a scale, because it is close to binary and because it carries more information than the mean does — whether the candidate ever attributed the throughput ceiling to the load generator, and at what point in the session. A candidate who reached that conclusion and reported no capacity figure at all has done the job correctly.

Integrity

The log describes what happened. It does not produce a cheating verdict — the follow-up conversation is the control, because a statistical accusation is not something we would ask a reviewer to defend.

What you receive

Who decides

Required, because the correct outcome of this design is a report that contains no capacity number, and no automated ranking handles that gracefully. The reviewer is asked, before seeing any score, to read the candidate's recommendation and answer one question in writing — if I quoted this to my director, what would I be claiming, and is it true? A candidate who reports that the limit could not be found, explains why, and says what would be needed to find it, should rank above a candidate who produced a confident figure, and a mechanical reading will do the reverse. Reviewers confirm or override each criterion with a written reason.

What this does not measure

This is an assessment of experimental design conducted at a scale that is not production. The sandbox has a small dataset, a single generator and a service with no real users, and it therefore cannot observe the things that make performance work hard in reality — a multi-day soak, a test window negotiated across three teams, a result that contradicts what a principal engineer has already told the board. It does not observe on-call temperament, and it does not observe behaviour during a genuine multi-hour outage where a capacity limit is being discovered live with revenue moving; performance engineers are frequently in that room and nothing here approximates the stakes. It also cannot show whether someone will hold a no-go recommendation across three weeks of pressure rather than across a twelve-minute conversation, which is the attribute that determines whether the function has any effect. Narrower cautions. The generator and service stack must be ones the candidate declared, since scoring experimental judgement while someone learns an unfamiliar tool measures the tool and penalises engineers whose background is in another one; the design deliberately supplies an existing script rather than asking for one partly for this reason. The data task must not depend on statistical vocabulary beyond percentiles, and a candidate who reasons correctly about the tail in plain language must be able to score five without using the standard terms. Both live calls are scored strictly on the content of the questions and the recommendation, never on accent, register, fluency or how confidently the candidate sounds — which matters unusually here, because the design explicitly rewards saying I could not establish that, and reviewers must not read appropriate hedging as weakness or read misplaced confidence as authority. Extended time should be available on request, applied before the session and not surfaced to the reviewer. Monitor score distributions by declared stack and by requested adjustment.

Performance testing is the one role in this family where the wrong answer arrives as a result. A QA analyst who misses a defect has withheld assurance; an engineer in test who ships a flaky suite has degraded trust slowly. A performance engineer who runs a badly designed experiment produces a number, a report, an approval and a shipped release, and the number is wrong in a direction that reassures everybody. That asymmetry is the reason this design scores the experiment rather than the tooling, and it is the reason the correct outcome of the session is a document that declines to state a capacity.

The false bottleneck is the mechanism. The sandbox's generator is sized so that its own CPU saturates before the service does, which means the throughput curve flattens cleanly at a specific figure. Everything about that figure looks like a finding: it is stable, it reproduces, it appears in a graph with a plateau in it. It is a measurement of the test client. The default assumption in this role, universally, is that the system under test is at fault — the whole point of pointing a load generator at something is to find its limit, and finding one feels like success. The candidate who instead watches their own generator's resource use, sees it pinned, and reports that they could not push hard enough to find the service's limit has produced the more useful and less satisfying result, and the rubric and the human review instruction are both written specifically so that this does not get ranked below a confident wrong answer.

The intake call is placed first, before any tooling exists, because the most decisive act in this role happens before the generator starts. The delivery manager's five thousand users is not a load model; it is not even a unit. A strong candidate spends the eight minutes turning it into arrival rate, action mix, think time, cold and warm data proportions, origin and a failure threshold, and separates what he knows from what he is guessing. The single-login script waiting in the sandbox is the consequence of not doing this: every virtual user authenticating as the same account pulls one row into cache and then measures the cache at thousands of requests a second, producing a spectacular result about nothing.

The data task exists as its own scored surface because choosing a statistic is a decision with an ethical component, not a calculation. The mean is comfortable and the tail is where the business lives. A candidate who reports the upper percentiles, states the warm-up exclusion rather than performing it silently, and converts nine seconds at the ninety-ninth percentile into a count of real people per hour has done the translation that makes a performance finding actionable. Declaring an exclusion is scored explicitly because quietly trimming a distribution is the most common way an honest engineer produces a dishonest number.

The written recommendation is where all of it either lands or does not. The useful artefact is not a measurement; it is a bounded claim — this is what I established, this is what I could not establish and precisely why, this is my recommendation and this is what it costs if I am wrong. Everything the sandbox withholds, the small dataset and the undersized generator, exists so that the honest version of this document has to contain real limitations rather than ritual caveats.

What this design does not attempt is any claim about production scale. Nothing observed here happened at volume, over days, against real traffic. The claim is narrower and defensible: this session shows whether the candidate designs an experiment whose answer would mean something, and whether they can tell a director that they do not yet know.

Sources

Every figure on this page is traceable. Where a claim could not be sourced it is stated qualitatively instead.

  1. US Bureau of Labor Statistics, Occupational Outlook Handbook, Software Developers, Quality Assurance Analysts, and Testers, 2025, https://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm
  2. HTTP Archive, Web Almanac 2024, Performance chapter, based on Chrome UX Report field data, https://almanac.httparchive.org/en/2024/performance
  3. Uptime Institute, Annual Outage Analysis 2026, press release, survey of data centre and IT operators, https://uptimeinstitute.com/about-ui/press-releases/uptime-announces-annual-outage-analysis-report-2026

See what the employer actually receives. A full report for one role, with every score shown beside the excerpt that earned it, conduct findings reported rather than averaged, and a reviewer sign-off required before any decision. No form.

Read a sample reportOr talk to us about this role