Taking an assessment rather than buying one? This page is written for employers. Here is the page for candidates.

Data and analysis · Entry level

How to assess a Business Intelligence Analyst

A dashboard portfolio is the least verifiable artefact in analytics hiring: it is a picture, it may be a template, it may be a team's work, and it says nothing about how the numbers behind it were defined. The chart-building exercise tests the part of the job a tool now largely automates and a model fully automates. Neither touches the actual competency, which is definitional custody — being the person who knows that active customer means one thing in the finance pack and another in the growth dashboard, and who refuses to ship a third meaning. Nor does anything in the standard process test the seat's most frequent live failure: being asked why a number moved, and distinguishing a real movement from a reporting artefact under time pressure.

The BI analyst owns a surface that other people look at every week without thinking about it, which is a stranger job than it sounds. The data analyst is asked a question and answers it once. The BI analyst is responsible for a set of numbers that will be read repeatedly, by people who will not check the methodology, and who will make decisions on the assumption that this week's figure is comparable to last week's. Comparability over time is the actual product.

That makes the defining competency definitional rather than technical. In most organisations of any age there are already three definitions of revenue, two of an active user, and a churn figure whose denominator nobody can reconstruct. A strong BI analyst treats every new request as a chance to reuse an existing definition and only invents a new one deliberately and with a note explaining why. A weak one is helpful: they build exactly what was asked for, name it something reasonable, and add a fourth revenue number to the estate. The damage is not visible for a year, and when it becomes visible it is attributed to the data being bad rather than to the hire.

The second differentiator is diagnostic under pressure. The recurring live moment in this job is a stakeholder pointing at a tile and asking why it dropped. There are two families of answer — the business changed, or the reporting changed — and they demand completely different responses. A filter that now excludes a segment, a source system that started sending nulls, a timezone boundary, a backfill that rewrote history, a definition someone edited on Friday: these produce movements indistinguishable from real ones on the surface. The candidate worth hiring says "before I explain it, let me check whether anything upstream changed," and knows what to check. The candidate to avoid produces a confident narrative on the spot, because a confident narrative is what the room wants.

The third is restraint. dbt Labs' 2025 practitioner survey found 57 percent of respondents spend most of their workdays maintaining or organising datasets rather than producing new ones, which is the same tax at a different layer: every artefact shipped is a maintenance obligation. A BI analyst who cannot decline a request, or cannot propose amending an existing report instead of adding one, becomes the bottleneck they were hired to relieve.

The boundary with BI developer is worth drawing hard. The BI developer builds and maintains the delivery machinery — the semantic model, the refresh, the extract job. The BI analyst is accountable for what the numbers mean and whether the business trusts them. Small organisations combine the two seats; when they do, they should assess for both and know which one they will forgive weakness in.

What the job actually needs

owning a recurring reporting surface
metric definition discipline
noticing when a number moved for a reporting reason rather than a business reason
visual clarity
saying no to a new dashboard

How people fail in this seat

ships a dashboard that duplicates an existing metric with a different definition
presents a change as a business trend when a pipeline or a filter changed
lets the dashboard estate grow until nobody knows which one is authoritative
builds what was requested instead of what would be looked at
cannot explain what a tile on their own dashboard is measuring

What most employers do instead

CV screen for tool names (Power BI, Tableau, Looker, Qlik), a portfolio of dashboard screenshots, sometimes a short exercise building a chart from a supplied clean extract.

The assessment

About 47 minutes end to end.

The systems it runs in

A Power BI workspace somebody else owns, sitting on a modelled warehouse. The three existing meanings of "active customers" are three DAX measures with the same display name, written by three people in three reports, all published, all in use and none flagged — which is the form this problem actually takes, because a measure name is what a stakeholder sees and the filter context behind it is what nobody reads. The candidate can open all three before building anything, and the report-open log records whether they did. The tile that fell nine percent is a visual on a report the candidate did not build, whose page filter excludes blanks on a column the source has begun sending empty, so a slice of the population silently drops out of the count without the visual changing or erroring. The warehouse is queryable directly during the call, which is what makes checking before explaining possible inside the ninety seconds the fork costs.

Power BI Desktop
Power BI Service workspace
DAX measures
Power BI page and visual filters
warehouse SQL

Tableau, where the three definitions are three calculated fields of the same name across three workbooks on a published data source; Looker, where they are three LookML measures across explores; Qlik or Metabase on the same pattern. Power BI is the default only because it is the platform a buyer filling this seat is most likely to already run, and nothing in the rubric depends on that choice — the estate is what matters and the vendor is not, and the fixture is rebuilt in the buyer's own tool wherever they can provide a workspace. What cannot be swapped away is that the reports, the definitions and the broken tile all belong to somebody else.

Working speed is not scored. Not scored, and in this file scoring it would directly punish the behaviour the design exists to observe. The second task's fork is whether the candidate spends ninety seconds checking upstream before explaining a movement to somebody who is short of time and wants an answer now; the fast path is a confident business narrative assembled from two plausible explanations sitting in the fixture, and it is wrong. The first task rewards opening three existing reports before writing a fourth definition, which costs minutes and produces nothing to show for them. A speed criterion would rank the candidate who did neither above the candidate who did both. What is reported instead, beside the score and outside the mean, is the plain fact of whether any existing report was opened before the candidate wrote their own, with a timestamp.

What the candidate actually does

Task	What happens
Build me active customers by segment data_task · 25 min	A request from a stakeholder preparing for a quarterly review — a view of active customers by segment, by month, for the last year. The candidate has a modelled warehouse and access to the existing reporting estate. That estate already contains the phrase "active customers" three times, in three reports, with three different definitions embedded in the queries behind them: one counts anyone with a login in the period, one counts anyone with a paid subscription on the last day of the month, one counts anyone with a paid subscription at any point in the month including trialists. All three are in use. None of them is flagged. The brief asks for the view and says nothing about definitions. The fork. Building it directly takes fifteen minutes, produces something clean, and the stakeholder is delighted. Whichever reasonable definition the candidate writes becomes the estate's fourth meaning of active customer, and the damage is invisible for a year, at which point it is attributed to the data being bad rather than to this decision. An assistant given this brief writes the fourth definition immediately and well, because the existing three are in the estate rather than in the prompt.
Why did that tile fall live_call · 12 min	A live spoken call. An AI stakeholder is looking at an existing dashboard — not the candidate's — and a tile that fell nine percent last week. They want an explanation now, because they are presenting in an hour. The cause is a reporting artefact: a source system began sending nulls in a field the tile's filter depends on, so a slice of the population silently dropped out of the count. The candidate can query during the call. The fork. A confident narrative is what the room wants and there are two plausible business explanations sitting in the fixture that would satisfy the stakeholder completely. Saying "before I explain it, let me check whether anything upstream changed" costs ninety seconds of an urgent call and makes the candidate look slower in front of somebody who is short of time. The stakeholder does not object either way and thanks them either way.
Explain your own tile live_call · 10 min	A short spoken conversation on the candidate's own work. Which definition of active customer did you use, and why that one? Somebody in finance opens this next to the board pack and gets a different number — what do you tell them? If I asked you to add churn to this view tomorrow, what would you want to know first? The band instrument here is ownership of the reasoning rather than depth: can the candidate restate their own choice in somebody else's vocabulary, on demand, without the query in front of them. The fork. A candidate who chose a definition by accident can describe what their query does and cannot say why that reading rather than another, because there was never a decision to describe. The finance question is the separator: it has an answer only for somebody who knows their number differs from somebody else's and by what.

The mark scheme

Each criterion is scored 1 to 5 against written anchors, and every score is reported with the excerpt that earned it. A criterion marked floored is reported as a finding rather than averaged into the total. The first is open; open any other to read its anchors in full.

Reuses an existing definition, or invents one deliberatelyweight 0.3Looks at what exists before building, finds that the phrase already means three things, picks one on a stated reason, labels the tile with the definit…

1 Writes a fresh definition without looking at the estate. Names it active customers. Ships a fourth meaning and does not know there were three.

3 Finds one or two of the existing definitions, adopts one, and does not record anywhere which one was adopted or that alternatives existed.

5 Looks at what exists before building, finds that the phrase already means three things, picks one on a stated reason, labels the tile with the definition it uses, and flags that the other reports disagree.

Separates a reporting artefact from a business movementweight 0.25Says the cause is not yet known, names the specific checks — did the population change, did an upstream field change, was history rewritten, did a fil…

1 Produces a business explanation on the spot, from the fixture's plausible candidates, with nothing checked. Confident, coherent, wrong.

3 Says the movement should be verified before being explained, and either does not know what to check or checks the wrong layer, ending with no answer and no route to one.

5 Says the cause is not yet known, names the specific checks — did the population change, did an upstream field change, was history rewritten, did a filter or definition change — runs at least one of them, and finds or credibly narrows to the nulls.

Restraint about what gets added to the estateweight 0.15Proposes amending or pointing at an existing report where that would serve, or builds the new one and says explicitly what it replaces or what should …

1 Builds exactly what was asked, adds it to the estate, and proposes nothing else. Helpfulness with no cost attached.

3 Mentions in passing that similar reports exist, without proposing an alternative to adding another one.

5 Proposes amending or pointing at an existing report where that would serve, or builds the new one and says explicitly what it replaces or what should be retired, or declines a part of the request with a reason.

Can explain their own numbers to somebody who did not build themweight 0.2States in one sentence what the tile counts and what it excludes, says where their number will differ from the board pack's and why, and names what th…

1 Describes the mechanics of the query. Cannot say in plain words what the tile counts, or gives an account that does not match what they built.

3 Explains the definition accurately in technical terms and does not translate it for the finance reader or anticipate the disagreement.

5 States in one sentence what the tile counts and what it excludes, says where their number will differ from the board pack's and why, and names what they would need in order to reconcile the two.

The output is readable by the person who asked for itweight 0.1The comparison the stakeholder asked about is the thing the eye lands on, and the definition and the date of last refresh are visible on the artefact …

1 Unlabelled axes, no definition on the tile, a chart type that obscures the comparison being asked about.

3 Clear and conventional, with the definition available only in the underlying query.

5 The comparison the stakeholder asked about is the thing the eye lands on, and the definition and the date of last refresh are visible on the artefact itself rather than in a document beside it.

How it is scored

Weighted mean of five criteria, 1-5 against the anchors, each reported with the query, artefact detail or transcript quotation behind it. Reported beside the score and outside the mean: whether the candidate opened any existing report before writing their own, as a plain yes or no with a timestamp. It is the single most informative fact this session produces about how somebody will behave inside an existing estate, and it is reported rather than scored because the scored criterion is what they did with what they found, not whether they clicked something.

Integrity

Monitored sandbox with statement log in execution order and a record of which existing reports and queries were opened, and when
Edit timeline and paste-versus-typed provenance
AI assistance permitted and logged; the fixture is designed so that assistance produces a clean fourth definition, because the existing three live in the estate rather than in the brief
Same-day spoken interview on the candidate's own artefact
No automated cheating verdict is produced. The log describes what was opened and built; the spoken question about the finance disagreement is the control, because it is answerable only by somebody who knows what they chose.

The log describes what happened. It does not produce a cheating verdict — the follow-up conversation is the control, because a statistical accusation is not something we would ask a reviewer to defend.

What you receive

The built view or dashboard as submitted
Statement log and report-open log with timestamps
Both call recordings and transcripts
Per-criterion score with the excerpt or quotation behind it

Who decides

Recommended. The restraint criterion in particular rewards a candidate for building less than they were asked for, which is a judgment a hiring team has to be willing to stand behind — some estates genuinely want a fourth report built quickly. A reviewer confirms or overrides each criterion with a written reason, and should read the definitional choice before reading the score.

What this does not measure

This design does not measure what it takes to hold a reporting surface stable over a year, which is the real job: the weekly discipline of noticing drift, the slow negotiation that retires a report somebody still opens, the accumulated knowledge of which stakeholder means what. A session shows the instinct once. It does not measure tool fluency in any particular BI vendor, deliberately, because tool names are the most transferable part of this seat and the most over-weighted part of the standard screen — a buyer who specifically needs deep Power BI or Looker skill should establish that separately. It does not measure design taste beyond legibility, and the visual criterion carries only a tenth of the weight for that reason. As an entry-band assessment it is also worth saying plainly that the estate-awareness this design rewards is partly a function of having worked in an estate before, and a candidate coming from a first job or a bootcamp may score low on definitional custody for reasons of exposure rather than of judgment; reviewers should weigh the follow-up interview heavily for that group, because a candidate who did not think to look but immediately understands why it mattered is a different prospect from one who does not see the problem when it is explained. The spoken segments are frequently in a second language and are scored on content, never on fluency, accent or register; no criterion here is named clarity or professionalism. Offer captions, extra time, or the calls in written form on request.

The BI analyst is not a junior data analyst, and assessing them as one is the mistake this design exists to avoid. A data analyst is asked a question and answers it once. A BI analyst owns a set of numbers that will be read repeatedly by people who will never check the methodology and who assume this week's figure is comparable to last week's. Comparability over time is the product. That makes the defining competency definitional rather than analytical, and definitional competence is invisible in every part of the standard screen — a dashboard portfolio is a picture, and a chart-building exercise tests the part of the job the tool already does.

So the fixture is an estate rather than a dataset. Three reports, three definitions of active customer, all in use, none flagged, and a request that would make a fourth. The request is ordinary and reasonable and the stakeholder is grateful for whatever gets built. Nothing in the brief hints that there is anything to find, because in the real version of this moment nothing hints either — the fourth definition is not created by a careless person, it is created by a helpful one. That is the fork, and it is the most expensive recurring decision in this seat: the damage does not appear for a year, and when it appears it is blamed on the data rather than on the hire.

The second scored moment is the one that happens most often on the job and least often in an interview. A stakeholder points at a tile and asks why it dropped. There are only two families of answer — the business changed, or the reporting changed — and they call for completely different responses. In this fixture the cause is a source system that began sending nulls into a field the tile's filter depends on, which silently removed a slice of the population from the count, and the fixture also contains two entirely plausible business explanations that would satisfy the stakeholder completely. The behaviour worth hiring is "before I explain it, let me check whether anything upstream changed", followed by knowing what to check. It costs ninety seconds in front of somebody who is short of time, which is exactly why it is rare and exactly why it has to be observed live rather than asked about.

The follow-up interview is deliberately the entry-band instrument. It does not escalate difficulty; it asks for ownership of the reasoning. Which definition did you use and why that one, and what do you say when finance opens this next to the board pack and gets a different number. A candidate who arrived at their definition by accident can describe what their query does and cannot describe a decision, because there was not one. This is also the fairest question in the set for an entry hire: it does not require prior estate experience to answer well, only that the choice was actually made.

One consequence of that fairness point deserves stating to buyers directly. The definitional criterion carries the largest weight, and part of what it measures is exposure — somebody who has worked in a messy estate has learned to look first, and somebody in their first analytics job has not. The interview is the correction. A candidate who did not think to check the estate but who, when shown the three definitions, immediately sees the problem and can say what they would have done, is a good entry hire. A candidate who does not see the problem after it is pointed out is not, and the session distinguishes them clearly.

Sources

Every figure on this page is traceable. Where a claim could not be sourced it is stated qualitatively instead.

US Bureau of Labor Statistics, Occupational Outlook Handbook, Database Administrators and Architects, 2025, https://www.bls.gov/ooh/computer-and-information-technology/database-administrators.htm
dbt Labs, 2025 State of Analytics Engineering Report, 459 data practitioners and leaders surveyed 8 October to 27 December 2024, https://www.getdbt.com/resources/state-of-analytics-engineering-2025

See what the employer actually receives. A full report for one role, with every score shown beside the excerpt that earned it, conduct findings reported rather than averaged, and a reviewer sign-off required before any decision. No form.

Read a sample report Or talk to us about this role