Job family

Assessing data engineering and analytics engineering

Roles that build and own the pipelines and models everyone else's numbers come out of.

This family is distinct from data analysis, and conflating the two is a common and costly hiring mistake. Analysts answer questions. Data and analytics engineers build the substrate the answers come from: ingestion, transformation, dimensional models, tests, orchestration, and the contracts that stop an upstream schema change from silently corrupting a board metric. The BLS projections capture the shift plainly — database architect roles growing 9 percent to 2035 while database administrator roles sit at 0 percent — and dbt Labs' 2025 survey of 459 practitioners shows a population now dominated by analytics engineers and data engineers rather than analysts.

What separates a good hire here is not tool familiarity, which is cheap and teachable, but a specific kind of paranoia about correctness at scale. A strong data engineer notices that the join they were about to write will fan out because the source table has a soft-delete pattern; declines to make a transformation idempotent-by-accident and instead makes it idempotent on purpose; writes the test that catches a duplicate primary key before the dashboard shows revenue doubling. The weak hire ships a pipeline that works on Tuesday's data. dbt Labs found poor data quality was the most frequently cited challenge, named by 56 percent of respondents — that is the tax the industry pays for hiring this family on tool keywords.

Screening has not kept up. The dominant formats are a SQL quiz, a whiteboard question about normal forms, and an unmonitored take-home pipeline exercise. All three are now fully solvable by a model: dbt Labs found 70 percent of respondents already using AI for analytics development and 80 percent using AI in their day-to-day workflow, up from around 30 percent a year earlier. A take-home that asks for a working transformation is, in 2026, a test of whether the candidate has an account with an AI vendor.

The design that still discriminates puts the candidate in a monitored sandbox with a deliberately dirty dataset and a modelling brief, then interviews them about their own output. The sandbox shows the order of work — did they profile the data before writing, or write first and discover the duplicate key by accident? The interview asks what breaks when the source system changes, which row-count assertion they would add and why, and what they would do if this pipeline failed at 3am. A candidate who assembled their solution can produce the code but not the failure analysis, because the failure analysis was never in the prompt they wrote.

Why this work can be assessed

The deliverable is code against data — a transformation, a model, a backfill — which a monitored sandbox with a real dataset reproduces directly, and which a follow-up interview can probe for the assumptions buried in the joins.

Roles in this family

Sources

Every figure on this page is traceable. Where a claim could not be sourced it is stated qualitatively instead.

1. US Bureau of Labor Statistics, Occupational Outlook Handbook, Database Administrators and Architects, 2025, https://www.bls.gov/ooh/computer-and-information-technology/database-administrators.htm
2. dbt Labs, 2025 State of Analytics Engineering Report, 459 respondents surveyed 8 October to 27 December 2024, https://www.getdbt.com/resources/state-of-analytics-engineering-2025
3. US Bureau of Labor Statistics, Occupational Outlook Handbook, Data Scientists, 2025, https://www.bls.gov/ooh/math/data-scientists.htm