Taking an assessment rather than buying one? This page is written for employers. Here is the page for candidates.

Data engineering and analytics engineering · Mid level

How to assess a BI Developer

The SQL test measures greenfield authoring, and this role is almost never greenfield. The characteristic day is opening a stored procedure written six years ago by someone who has left, working out which of four similar reports the business actually uses, and changing one thing without breaking the other three. That is a comprehension and archaeology skill, and it is close to the inverse of what a timed authoring test rewards — a fast writer with no patience for reading existing logic is precisely the wrong hire here and precisely the profile the standard screen selects for. The vendor keyword screen makes it worse by filtering on tool names, which are the most transferable part of the job, while ignoring the least transferable part, which is the temperament to work inside somebody else's undocumented system.

BI developer and ETL developer are the same seat under two names, and it is the seat that keeps the existing reporting estate alive. Most of the organisations hiring it are not building a modern warehouse from scratch; they are running one that accumulated over a decade, in a vendor stack, with several hundred scheduled jobs and reports of which perhaps forty matter and nobody knows which forty. The work is maintenance, extension and careful surgery, and it is undervalued precisely because the successful version of it is invisible.

The defining competency is reading. A BI developer is handed a number that is wrong in a report and has to walk it backwards: which dataset, which view, which procedure, which staging table, which source extract, which upstream system, and at which of those seven layers the logic diverged from what the business believes. Candidates who cannot do this comfortably compensate by rebuilding, which is the most expensive failure available in this role — a rebuild takes three months, reproduces the original's undocumented edge cases only partly, and the discrepancies surface one at a time for a year afterwards.

The second competency is reconciliation. Incremental loads are the backbone of this work and they drift: a late-arriving record, a source row updated after extraction, a failed run half-applied, a watermark advanced past unprocessed data. A strong developer builds a control total that compares the target back to the source and alerts on divergence, because the alternative is discovering the gap when someone notices the annual figure does not add up. A weak one monitors whether the job succeeded, which is a different question entirely.

The third is writing for the successor. This role turns over, and every undocumented job left behind is a liability priced into the next hire. A short written handover — what this job does, what it assumes, what breaks it, what to check first — is a genuine and assessable deliverable, and it is one of the few places where a candidate's habits are visible in a single artefact.

The boundary with the BI analyst is worth stating because job adverts routinely merge them. The analyst answers for the meaning of a number and for whether the business believes it; the developer answers for the machinery that produces it on schedule and without drift. The boundary with the analytics engineer is generational rather than functional: both own transformation logic, but the analytics engineer works in a version-controlled, tested, modular layer and the BI developer usually does not have that luxury, which makes their change-safety discipline a matter of personal practice rather than of tooling. Assessing this role with a modern-stack exercise flatters candidates who have only ever worked in one and tells the buyer nothing about how they behave inside the estate they are actually being hired to inherit.

What the job actually needs

reading and safely changing code you did not write
tracing a number backwards through a stack of jobs
incremental load and reconciliation logic
documenting for the next owner
performance tuning against a real workload

How people fail in this seat

rewrites rather than reads
changes a shared job without finding its other consumers
builds an incremental load with no reconciliation against source totals
leaves a scheduled job whose logic exists nowhere but in the job
tunes for the query that was slow today and breaks the one that runs at month end

What most employers do instead

CV screen for a vendor stack (SSIS, Informatica, Power BI, SSRS, Qlik), a SQL test on joins and window functions, and an interview about past projects and tool versions.

The assessment

About 70 minutes end to end.

The systems it runs in

Somebody else's estate, named. A Power BI semantic model published to a workspace by a developer who has left, four reports built on it, and behind it a Microsoft SQL Server warehouse loaded by a T-SQL stored procedure on a SQL Server Agent schedule. The candidate builds nothing here and inherits everything, which is the fixture's whole argument. Two properties of that arrangement do real work in the rubric. The consumers are discoverable the way they are discoverable on the job — the workspace subscription list says who is sent what and the warehouse query history says who actually runs it — rather than being stated in the brief. And the shared view the candidate is tempted to change is referenced by two other views and by the semantic model's source query, so the third dependent is not in the database at all and is invisible to anyone who checks dependencies inside SQL Server and stops there.

Microsoft SQL Server
T-SQL stored procedure
SQL Server Agent
Power BI semantic model
Power BI Service workspace subscriptions
SQL Server query history

Tableau, where the estate is a published data source with four workbooks and subscriptions on Tableau Server; or Looker, where the shared object is a LookML view and the dependents are explores. Oracle, Snowflake or PostgreSQL in place of SQL Server, and SSIS or Azure Data Factory in place of the Agent job. The estate is rebuilt inside the buyer's own tool wherever they can provide a sandbox, because inheriting a model in the tool you will actually run it in is the point. What must survive any swap is that the semantic model and the reports were built by somebody who is no longer available to explain them.

Working speed is not scored. Not scored, and this is the seat where scoring it would do the most damage. The raw number reported here is time to first edit, and a long one is the good sign — the discriminating candidate reads the chain end to end before touching anything, while the fast candidate rewrites the stored procedure, reconciles August by accident through a full reload, and leaves the watermark in place to reopen the gap in September. A speed criterion would rank that submission first. Time to first edit and the number of distinct objects opened before it are reported raw and outside the mean instead, which also keeps a long read from becoming a number candidates can pad.

What the candidate actually does

Task	What happens
The report that is under the ledger coding_sandbox · 40 min	An inherited estate, deliberately unpleasant, and none of it is the candidate's. A source extract, three staging tables in SQL Server, a two-hundred-line T-SQL stored procedure written years ago by somebody who has left and run nightly by a SQL Server Agent job, four views with similar names, and a published Power BI semantic model carrying four reports of which the business genuinely uses one. The ticket is one sentence — August recognised revenue in the monthly pack is slightly below the ledger, find out why and fix it. Nothing says which of the four reports finance reads; that is discoverable from the workspace's subscription list and the warehouse's query history, both of which are in the fixture. The real cause is the incremental watermark: the stored procedure takes `MAX(extracted_at)` from the target, so a source row updated after the last extraction of the month is never collected. A decoy sits closer to the surface — a currency conversion view that rounds at the wrong step and accounts for a trivial part of the gap, is easy to find, is genuinely a defect, and does not explain August. The fork. The procedure is ugly enough that rewriting it is the obviously satisfying move, and it is rewarded twice over. A rewrite is what an assistant produces when asked to fix a stored procedure, and a rewrite makes August reconcile — because a full reload happens to pick up the rows the watermark skipped. It looks exactly like a fix, it passes the only check available, and the gap reopens next month. Fixing the rounding decoy also produces a real, defensible, verifiable improvement to a number, which is the second way to spend forty minutes and be wrong.
The note for whoever has this next written_artifact · 12 min	A handover for the job the candidate has just changed — what it does, what it assumes, what breaks it, what to check first when the number is wrong again. Graded against what is actually in the fixture and what the candidate actually changed, not against a template. The fork. A description of the procedure's steps is easy to write, reads as thorough, and is worth nothing to the next owner, who can read the procedure. The valuable note is the part that exists nowhere in the code: which assumption the whole thing rests on, and what the candidate did not have time to check.
Interview on the trace live_call · 18 min	A spoken conversation with the estate on screen. Walk me through the order you opened things in and what each one ruled out. You changed a view that three other objects reference — which three, and how did you establish that, given that one of the three is not in the database at all? The source team is about to start back-dating corrections up to ninety days; does your fix still hold, and what does it do to a period that has already been reported? Which of the four reports did you decide finance actually reads, and what told you? The fork. A candidate who rewrote can describe the new procedure fluently and cannot say what the old one did on the last day of the month, because they never found out. The back-dating question is answerable only by someone who understood the watermark rather than replaced it.

The mark scheme

Each criterion is scored 1 to 5 against written anchors, and every score is reported with the excerpt that earned it. A criterion marked floored is reported as a finding rather than averaged into the total. The first is open; open any other to read its anchors in full.

Reads before writing, and does not rewriteweight 0.25Reads the chain end to end before changing anything, makes the smallest change that addresses the actual cause, and can state what the original logic …

1 First substantive action is authoring. Replaces the procedure or large parts of it. Reconciles August and cannot say what the original did differently.

3 Reads a fair amount of the existing logic, then rewrites the section containing the fault rather than changing it, without establishing whether the rewritten section had behaviour that mattered elsewhere.

5 Reads the chain end to end before changing anything, makes the smallest change that addresses the actual cause, and can state what the original logic was doing and why somebody might have written it that way.

Traces the number to the layer that caused itweight 0.25Names the watermark as the mechanism, states the exact condition that loses a row — a source update after the last extraction of a period — demonstrat…

1 Stops at the rounding decoy and reports it as the cause, or fixes the symptom by reloading without identifying any cause at all.

3 Reaches the incremental load and identifies that rows are missing, without isolating the watermark expression as the mechanism or saying which rows are lost and when.

5 Names the watermark as the mechanism, states the exact condition that loses a row — a source update after the last extraction of a period — demonstrates it against the data, and separately reports the rounding defect as real but not the cause.

Reconciliation rather than success monitoringweight 0.2Proposes a control total that compares the target back to the source on an amount rather than a count, on a schedule that would have caught this gap, …

1 Proposes alerting on job failure, or nothing. Treats a successful run as evidence the data is right.

3 Proposes a row-count comparison between source and target, without addressing updated rows, which is the failure in front of them.

5 Proposes a control total that compares the target back to the source on an amount rather than a count, on a schedule that would have caught this gap, and says what tolerance is acceptable and what happens when it is breached.

Finds the consumers before changing a shared objectweight 0.15Establishes both — the objects that reference the changed view, and which report finance genuinely uses, from the distribution list and the query hist…

1 Changes a view several objects depend on without looking for dependencies, and cannot name them when asked.

3 Checks dependencies within the database and finds the objects, without establishing which of the four reports is actually read or by whom.

5 Establishes both — the objects that reference the changed view, and which report finance genuinely uses, from the distribution list and the query history — and says what would move for each consumer.

The handover is usable by the next personweight 0.15Names the load-bearing assumption in one sentence, names what breaks it, says what to check first when the number is wrong, and states plainly what th…

1 Restates what the code does, in prose, at a level a reader could get from the code. Names no assumption and no failure mode.

3 Documents the job's purpose, schedule and inputs accurately, with a general note that the incremental logic is fragile.

5 Names the load-bearing assumption in one sentence, names what breaks it, says what to check first when the number is wrong, and states plainly what they did not have time to investigate.

How it is scored

Weighted mean of five criteria, 1-5 against the anchors, each reported with the excerpt, query or transcript line behind it. Two raw numbers are reported beside the score and outside the weighted mean, per the sequence pattern: time from session start to the first edit of any file, and the number of distinct objects opened before that first edit. The gap is the artefact. This is the one seat in the corpus where a long time to first edit is the good sign, and reporting it raw rather than scoring it keeps that from becoming a number candidates can pad. Also reported and annotated as not a positive signal: whether August reconciles at the end of the session. It reconciles for the candidate who found the watermark and for the candidate who rewrote the procedure and got lucky, and a buyer reading that column as the result would rank them identically.

Integrity

Monitored sandbox with file-open order, read time per object, and full statement log in execution order
Edit timeline with paste-versus-typed provenance, which is what distinguishes a rewrite from a change
AI assistance permitted and logged; the fixture is designed so that assistance produces a clean rewrite that appears to fix the month
Same-day spoken interview on the candidate's own trace, with the estate on screen
No automated cheating verdict is produced. The log shows what was opened and in what order and what was typed against what was pasted; the interview about the original logic is the control, because only someone who read it can describe it.

The log describes what happened. It does not produce a cheating verdict — the follow-up conversation is the control, because a statistical accusation is not something we would ask a reviewer to defend.

What you receive

File-open order and read time per object
Diff of what was changed, with edit timeline and paste provenance
The handover note as submitted
Interview recording and transcript
Per-criterion score with the excerpt or quotation behind it
Time to first edit and objects opened before it, reported raw

Who decides

Recommended. The rewrite criterion needs a human, because the boundary between a legitimate targeted rewrite and an avoidance of reading is a judgment call that depends on what the reviewer would have done with the same procedure. A reviewer confirms or overrides each criterion with a written reason.

What this does not measure

This design names a stack because an inherited estate has to have been built by somebody in something, but it does not measure vendor-tool fluency and that distinction should be held firmly. No criterion here is about DAX, about M, or about a T-SQL dialect, and none rewards knowing where a menu is; the anchors are written about tracing, reconciliation and blast radius, and a candidate who has never opened Power BI can score at the top of every one of them. Tool names are the most transferable part of this job and the standard screen already over-weights them. A buyer who specifically needs SSIS package authoring or deep DAX modelling should establish that separately and should not read it into this score. It does not measure performance tuning against real volumes, nor the multi-year accumulation of knowledge about a specific estate, which is most of what makes an incumbent in this seat valuable and which by definition cannot be assessed before the hire. It also does not measure stamina for maintenance work over months, which is the temperament question underneath this role; a session can show that someone reads carefully once. Fairness notes: a candidate who has only ever worked in a modern version-controlled transformation layer may find the fixture unfamiliar in ways unrelated to competence, so the anchors are written about tracing, reconciliation and blast radius rather than about any dialect. The interview is spoken, frequently in a second language, and is scored on content and never on fluency or register. Offer captions, extra time, or the interview in written form on request.

Almost every screen for this role is an authoring test — write this query, use a window function, model these dimensions — and almost none of the job is authoring. The characteristic day is opening a stored procedure written six years ago by somebody who has left, working out which of four similar reports the business actually reads, and changing one thing without breaking the other three. That is archaeology, and it is close to the inverse of what a timed authoring test rewards. A fast writer with no patience for reading somebody else's undocumented logic is exactly the wrong hire for this seat and exactly the profile the standard screen selects.

So the fixture is deliberately unpleasant, and the fork is the rewrite. This is the strongest fork available in this role because the rewrite is not a lazy move — it is an intelligent, defensible, well-motivated move that a good engineer might make, and in this fixture it is wrong for a reason the candidate cannot see from inside it. Reloading the affected period is what a rewrite does as a side effect, and reloading picks up the rows the watermark skipped, so August reconciles. The check goes green. The candidate has strong evidence they have fixed it. They have not: the gap will reopen at the end of September, because the mechanism that loses a source row updated after the last extraction of a period is still there. It is very hard to construct a task where being wrong feels this much like being right, and that is the whole value of it.

The rounding decoy exists for a different reason. It is a real defect. Fixing it is a genuine improvement and a candidate who reports it has found something true. It is placed nearer the surface than the watermark because the seat's actual discriminator is not whether somebody can find a problem in old code — most competent people can find one — but whether they keep going after finding one that plausibly accounts for the symptom. The top anchor on the tracing criterion requires both: name the watermark as the cause and report the rounding as real but not the explanation.

The handover note is the second deliverable and it is here because it is one of the very few places where this role's habits become visible in a single artefact. Every undocumented job left behind is a liability priced into the next hire, and the note that is worth something is not a description of the steps — the next owner can read the steps. It is the sentence naming the assumption the whole thing rests on, and the honest admission of what was not checked. A candidate who writes "I did not verify whether the other three reports use this view, and that is the first thing I would confirm" has told the buyer more about how they will behave in an inherited estate than any amount of correct SQL.

The interview is calibrated as the mid-band instrument: a changed requirement that invalidates an assumption rather than a harder version of the same question. Back-dated corrections up to ninety days do not make the fix harder to write; they make the fix's premise false, and only somebody who understood the watermark rather than replaced it can say so. That is what the whole design is built to separate, and it is worth stating plainly to a buyer that the two candidates it separates will often submit work that looks equally finished.

Sources

Every figure on this page is traceable. Where a claim could not be sourced it is stated qualitatively instead.

US Bureau of Labor Statistics, Occupational Outlook Handbook, Database Administrators and Architects, 2025, https://www.bls.gov/ooh/computer-and-information-technology/database-administrators.htm
US Bureau of Labor Statistics, Occupational Outlook Handbook, Computer Systems Analysts, 2025, https://www.bls.gov/ooh/computer-and-information-technology/computer-systems-analysts.htm

See what the employer actually receives. A full report for one role, with every score shown beside the excerpt that earned it, conduct findings reported rather than averaged, and a reviewer sign-off required before any decision. No form.

Read a sample report Or talk to us about this role