Get started

Taking an assessment rather than buying one? This page is written for employers. Here is the page for candidates.

Software engineering · Entry level

How to assess a Graduate Software Engineer

This is the band where the screen carries the entire load, and it is the band where the screen is weakest. There is no work history to fall back on, so a bad automated assessment is not one signal among several — it is the only one. And the automated assessment is now the least defensible artefact in hiring: a self-contained, precisely specified puzzle with a known answer is the exact shape of problem an assistant solves perfectly, sat by candidates who have the least professional exposure to why that would be a bad idea and the least to lose by trying. The deeper failure is that the test was never measuring the right thing even when it worked. A junior engineer does not fail in their first year because they cannot invert a binary tree. They fail because they spend three days blocked without telling anyone, or change a function without checking who calls it, or cannot describe a bug precisely enough for someone senior to help. Every one of those is a behaviour under observation, and none of them appears in a score out of 100.

This is one role file covering junior backend, junior frontend and junior full-stack hiring, and that is a deliberate decision rather than a shortcut. At entry level the specialism changes the language and the repository the candidate is dropped into; it does not change what is being measured. Nobody hires a graduate for their opinion about database isolation levels. They hire them for the rate at which the person converts supervision into independent output, and that rate is specialism-agnostic. Where the senior files in this family diverge sharply by specialism, the entry-level design differs only in the fixture.

What the job actually is, in the first year, is bounded work with a known answer somewhere in the building. A ticket says a date is rendering in the wrong timezone for users in one region. The fix is four lines. Finding those four lines requires reading code somebody else wrote, forming a hypothesis, checking it, and — critically — knowing at what point to stop and ask. The competence being purchased is not knowledge. It is a particular kind of tractability: makes progress, notices when progress has stopped, says so early, and does not repeat the same mistake after being told.

The separation between the top quartile and the median at this level is almost entirely about the blocked state. Everyone gets stuck; graduates get stuck constantly, and they should. The strong ones get stuck in a legible way. They can say: I expected this function to return a list of three, it returns an empty list, I have checked that the query is running, I think the filter is wrong but I do not know why. That sentence is worth more than any amount of syntax fluency, because it converts one person's blockage into a thirty-second answer from someone senior. The weak ones say "it's not working", or worse, say nothing for two days and then present a large, confident, wrong change.

The second separator is reading before writing. A junior who opens the file, changes the thing, and runs the test is doing the job in the wrong order. A junior who first finds the three other call sites and asks whether they depend on the current behaviour is already operating a band above. This is directly observable in a monitored sandbox — the sequence of files opened before the first edit is a genuine signal, and it is not available from a submitted diff.

Hiring managers should be honest about the market context here. The Stanford Digital Economy Lab's payroll analysis is the most credible evidence available that this specific band is contracting relative to more senior ones, with the gap for 22-to-25-year-olds in AI-exposed occupations widening from roughly 15 to roughly 19 percent over a year. Whatever one concludes about cause, the practical consequence for a hiring team is more applications per opening, more of them assisted, and a higher cost to getting the screen wrong in either direction. Screening harder is not the answer; screening for something the assistant cannot supply is.

That is what the follow-up conversation does at this level, and the questions are gentler than the senior versions but no less diagnostic. Show me the first thing you looked at and tell me why. You tried this approach for six minutes and abandoned it — what made you stop? Here is a change I would like to make to your solution; talk me through what would break. And the one that separates most reliably: explain this line to me as if I were another graduate. Someone who reasoned their way to the answer explains it fluently in plain language. Someone who obtained the answer explains it in the vocabulary of the answer, and cannot rephrase it. Nobody can complete that conversation on a candidate's behalf, which is the entire point.

What the job actually needs

How people fail in this seat

What most employers do instead

A CV or degree filter, then a timed online assessment of algorithm puzzles administered at scale, then a technical interview.

This is the band where the screen carries the entire load, and it is the band where the screen is weakest. There is no work history to fall back on, so a bad automated assessment is not one signal among several — it is the only one. And the automated assessment is now the least defensible artefact in hiring: a self-contained, precisely specified puzzle with a known answer is the exact shape of problem an assistant solves perfectly, sat by candidates who have the least professional exposure to why that would be a bad idea and the least to lose by trying. The deeper failure is that the test was never measuring the right thing even when it worked. A junior engineer does not fail in their first year because they cannot invert a binary tree. They fail because they spend three days blocked without telling anyone, or change a function without checking who calls it, or cannot describe a bug precisely enough for someone senior to help. Every one of those is a behaviour under observation, and none of them appears in a score out of 100.

The assessment

About 70 minutes end to end.

The systems it runs in

A TypeScript and Node repository with a React front end and a PostgreSQL schema, opened in a browser-hosted VS Code inside the sandbox, with the full git history present and the failing case reported by the project's own test runner. The specialism swap changes which directory the candidate lands in; it does not change the tooling around them. The instruments that matter at this band are the editor's find-references and project-wide search, and git log and git blame on a file the candidate has never opened.

Python with pytest, Java with JUnit in IntelliJ IDEA, or C# with .NET in Visual Studio; the timezone defect and its three callers keep their shape in all of them. The default is chosen because JavaScript, SQL and Python are among the most reported languages, PostgreSQL the most reported database and VS Code the most reported editor in the Stack Overflow Developer Survey 2025, so it is the environment the largest share of graduate applicants have already seen. Where a buyer supplies their own repository, the bug is planted in it and the fixture is theirs.

Working speed is not scored. The whole of t1 is a measurement of the twenty seconds before the first keystroke — whether the candidate searched for callers or edited at the line the stack trace named. The second is faster and wrong. A speed criterion would score it higher, which inverts the design. Time to first edit is still recorded and reported as a raw number beside the score, annotated as not a positive signal.

What the candidate actually does

TaskWhat happens
The unfamiliar repository
coding_sandbox · 25 min		The candidate is dropped into a repository they have never seen, with one bug report written the way bug reports are actually written: "the weekly digest shows yesterday's date for users in Australia." A failing test points at one function. That function has three other callers, two of which depend on the current behaviour. The fixture is swapped by specialism — a service, a component, or a small full-stack app — but the defect has the same shape in all three, because what is being measured is not the stack. AI assistance is available inside the sandbox and its use is expected, permitted and logged. The fork is at the first edit: the cheap path changes the line the test points at and turns the suite green, which is exactly what an assistant handed the stack trace will propose; the correct path finds out who else calls this before touching it. The sandbox records which files were opened before the first character was typed, and that ordering is the primary evidence for this task.
The question you cannot avoid asking
written_artifact · 10 min		Fifteen minutes in, the task requires a fact that is genuinely not anywhere in the repository and cannot be derived from it: which timezone the business treats as authoritative for a user who has never set one. There is no answer to find, and searching harder will not produce it. The candidate is told they may send exactly one message to a senior engineer who is in a meeting and will reply in twenty minutes, and that they should keep working meanwhile. Two things are scored: the informational content of that message, and what the candidate did with the twenty minutes. This task is deliberately unwinnable by persistence, because the blocked state is where graduate hires are actually lost.
The correction
coding_sandbox · 15 min		A short review message arrives in the sandbox from a fictional senior engineer, making one specific and correct criticism of the work so far — for example that the candidate has compared timestamps as strings in one place. The same mistake exists in a second place the reviewer did not mention. The candidate has fifteen minutes and no further contact. What separates candidates is whether the correction is applied once, applied everywhere it applies, or resisted; and whether a candidate who genuinely disagrees says so with a reason rather than silently complying or silently ignoring.
Explain your own change
live_call · 20 min		A live spoken conversation about the diff the candidate just produced, with the code on screen. The questions are gentle and entirely first-person. Show me the first file you opened and tell me why. You spent six minutes on this approach and abandoned it — what made you stop? Point at the line you were least sure about. I want to change the requirement: the digest now runs hourly instead of weekly — walk me through what breaks. And the one that separates most reliably: explain this line to me as if I were another graduate on your intake. Someone who reasoned their way to the change explains it in plain language and can rephrase it a second way. Someone who accepted a generated change explains it in the vocabulary of the change itself and cannot restate it.

The mark scheme

Each criterion is scored 1 to 5 against written anchors, and every score is reported with the excerpt that earned it. A criterion marked floored is reported as a finding rather than averaged into the total. The first is open; open any other to read its anchors in full.

Reads before changingweight 0.2Searches for callers before the first edit, names the two that depend on the current behaviour, and either preserves them or states in the submission …
1 First edit lands in the file named by the stack trace, within two minutes, with no other file opened. Discovers the two broken callers only when a test fails, or not at all.
3 Opens the calling code at some point before submitting and adjusts for it, but after the first edit rather than before.
5 Searches for callers before the first edit, names the two that depend on the current behaviour, and either preserves them or states in the submission that they were changed on purpose.
Legibility when blockedweight 0.25Message states what was expected, what was observed, what has already been ruled out, and what the candidate will assume until told otherwise — and th…
1 Message says the equivalent of "it doesn't work" or "what should I do here", or no message is sent and the twenty minutes are spent guessing at the unknown.
3 Message states the problem accurately but leaves the senior engineer to do the framing; some useful work continues in parallel.
5 Message states what was expected, what was observed, what has already been ruled out, and what the candidate will assume until told otherwise — and the work continues on that stated assumption, marked as such.
Ownership of the reasoningweight 0.25Rephrases any line on request, names at least one thing in the diff they would do differently before being asked, and traces the changed requirement t…
1 Cannot restate a line of their own diff in different words; explains it using the same phrasing on each attempt; answers the changed-requirement question with a general principle rather than a consequence in this file.
3 Explains most of the diff in their own words, hesitates on one section and says so.
5 Rephrases any line on request, names at least one thing in the diff they would do differently before being asked, and traces the changed requirement to specific lines that would break.
Response to correctionweight 0.15Applies it, finds the second instance unprompted, and says in one line what the underlying rule is — or gives a specific reason for disagreeing rather…
1 Ignores the review message, or applies it only at the exact line quoted while leaving the identical mistake untouched twelve lines away.
3 Applies the correction where it was pointed out and acknowledges it.
5 Applies it, finds the second instance unprompted, and says in one line what the underlying rule is — or gives a specific reason for disagreeing rather than complying.
Correctness of the changeweight 0.15Fixes the reported behaviour and leaves the dependent callers correct, with the intent legible from the diff.
1 Submitted change does not fix the reported behaviour, or fixes it by breaking something else without noticing.
3 Fixes the reported behaviour; one dependent case is left inconsistent.
5 Fixes the reported behaviour and leaves the dependent callers correct, with the intent legible from the diff.

How it is scored

Weighted mean of the five criteria, each scored 1-5 against the anchors, and reported with the excerpt or transcript line that earned it. Correctness carries the smallest weight in the set, which is deliberate: at this band a four-line fix is not the scarce thing, and any assistant will supply one. Scores are reported as a band with the evidence, not as a percentile against other candidates, because rank-ordering graduates on a 15-percent-weighted correctness score is exactly the false precision this design exists to avoid.

Integrity

The log describes what happened. It does not produce a cheating verdict — the follow-up conversation is the control, because a statistical accusation is not something we would ask a reviewer to defend.

What you receive

Who decides

Required, not merely recommended, at this band. A human reviewer confirms or overrides every criterion with a written reason, and the ownership criterion is never scored from the transcript alone — a reviewer who was on the call signs it. Entry-level candidates have no work history behind the screen, so the screen carries the whole decision and must never be the whole decision.

What this does not measure

This design deliberately does not measure algorithm recall, typing speed, prior exposure to the fixture's language or framework, degree class, or institution. It does not measure how much code someone produces per hour. Two risks need active monitoring by the deployer. First, the blocked-state task rewards a particular directness in writing, and directness is unevenly distributed by education, culture and confidence rather than by ability — score the informational content of the message, never its register, politeness, grammar or fluency in English, and give reviewers that instruction explicitly. Second, the correction task can penalise candidates who have never seen a code review before, which correlates with route into the industry rather than with aptitude; the anchors are written about the change made, not about knowing review etiquette, and reviewers should hold that line. The spoken interview is a real accessibility surface: offer it with captions, with extra time, in writing, or asynchronously on request, and score the content of the explanation, never the accent, pace or fluency of delivery. Finally, this assessment predicts a first year of supervised work. It does not predict who will still be strong at senior level, and should not be used to.

Every other role in this hub gets an assessment shaped by its specialism. This one covers junior backend, junior frontend and junior full-stack with a single design, and the reason is not economy. At entry level the specialism decides which fixture the candidate opens and nothing else. Nobody hires a graduate for an opinion about isolation levels or bundle budgets; they hire for the rate at which supervision turns into independent output. That rate is the same measurable thing whether the timezone bug is in a serialiser, a component or both, so the fixture swaps and the rubric does not.

The design is built around one uncomfortable fact: the bug in t1 is four lines, and any assistant handed the stack trace will produce those four lines instantly. That is fine, and it is assumed. The four lines are not the assessment; they carry fifteen percent of the score. What the sandbox is actually watching is the twenty seconds before the first keystroke. A candidate who searches for callers before editing has done the job in the right order. A candidate whose first action is an edit at the line the error named has done it in the wrong order, and will keep doing it in the wrong order in week three of employment, on code that matters. The submitted diff cannot distinguish those two people. The edit timeline can, which is the entire argument for a monitored sandbox over a take-home at this band.

The second task exists because the most expensive graduate failure is not a bad change, it is silence. A junior who is stuck for two days and says nothing costs more than one who is stuck for two days and says so in the first hour, and the difference is visible in a single sentence. So the design manufactures a question that must be asked — a fact that is not in the repository and cannot be inferred from it. Candidates who grind at an unanswerable problem reveal themselves, as do candidates who stop entirely rather than proceed on a stated assumption. The strongest response is not the politest one; it is the one that tells a busy senior engineer what was expected, what happened, what has been eliminated, and what the candidate is assuming meanwhile. Reviewers are instructed to score exactly that content and to ignore the writing.

The third task tests the thing that actually determines whether a graduate is worth supervising: what happens after they are told they were wrong. The planted correction is real and the candidate has made the same mistake twice. Applying the fix once is the median. Finding the second instance without being told is the top quartile, because it means the candidate extracted a rule rather than accepting a patch. And a candidate who disagrees with a reason — who says the comparison is safe here because these strings are always ISO-8601 — scores as highly as one who complies, because the failure mode being screened out is not disobedience, it is passivity.

The interview is where the delta gets measured. It is deliberately the gentlest follow-up in this hub, because the questions do not need to be hard to work. Asking someone to explain their own line to a peer, in plain language, twice, in two different ways, is a test that only fluency-of-reasoning passes. Assistance can produce the change; it cannot sit in the chair and rephrase it. The requirement change — weekly digest becomes hourly — is asked because a candidate who understood their own fix can trace it to specific lines within a sentence or two, and a candidate who accepted it will answer about caching or timezones in general. That gap opens up in under a minute and it is the most reliable signal in the whole design.

Sources

Every figure on this page is traceable. Where a claim could not be sourced it is stated qualitatively instead.

  1. Stack Overflow, 2025 Developer Survey, Developers section, roughly 49,000 respondents worldwide, developer-type question answered by 43,560, https://survey.stackoverflow.co/2025/developers
  2. Brynjolfsson, Chandar and Chen, Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence, Stanford Digital Economy Lab, August 2026 update, based on US ADP payroll microdata, https://digitaleconomy.stanford.edu/news/canariesaug26/
  3. Stack Overflow, 2025 Developer Survey, AI section, roughly 49,000 respondents worldwide, https://survey.stackoverflow.co/2025/ai

See what the employer actually receives. A full report for one role, with every score shown beside the excerpt that earned it, conduct findings reported rather than averaged, and a reviewer sign-off required before any decision. No form.

Read a sample reportOr talk to us about this role