Azure data engineering: what to hand a consultancy, what to keep, and what it should cost

Two ways mid-market businesses get Azure data engineering wrong: hand over everything and depend on a day rate forever, or hand over nothing and stall for a year. The right answer is a deliberate split. Here is what to outsource, what to keep, and what it should cost.

There are two ways mid-market businesses get Azure data engineering wrong when they bring in outside help. The first is handing over everything, ending up with a platform nobody internally understands, and paying a day rate forever to change a column. The second is handing over nothing, asking a stretched internal BI person to become a data engineer overnight, and watching the project stall for a year. The right answer is a deliberate split, and knowing where the line sits is most of the value.

This is the plain version of what Azure data engineering work actually involves, what is sensible to outsource, what you should keep in-house, and roughly what it costs, so you can have the conversation with your eyes open.

What Azure data engineering actually covers

The term gets used loosely. Concretely, for a mid-market business, it is the work of getting data from your source systems into a governed, reliable state that reporting and AI can trust. That breaks into a few distinct jobs.

Ingestion: connecting to source systems (your ERP, CRM, finance system, operational databases) and landing their data on Azure, reliably and on a schedule. This is where Azure Data Factory and Fabric pipelines live.

Storage and modelling: laying out the platform, lakehouse or warehouse, the medallion layers, the semantic model that Power BI reads. This is the architecture that determines whether the platform ages well or becomes a swamp.

Transformation: turning raw data into conformed, business-ready tables with rules applied once and consistently. This is where most of the ongoing engineering effort actually goes.

Governance and observability: lineage, access control, data quality checks, alerting, the SLAs that tell the business the data is right. Often skipped, always regretted.

Enablement: documentation, handover, and making sure your own people can run and change the thing without a phone call.

A good engagement touches all five. A poor one delivers the first three, skips the fourth, and forgets the fifth entirely.

What to outsource, what to keep

The instinct to keep control and the instinct to let the experts handle it are both half right. Here is the split that works.

Outsource the build and the hard architecture. The initial platform design, lakehouse versus warehouse, the pipeline patterns, the governance model, the migration off whatever you are on now, is exactly the work where experience pays and mistakes are expensive to unwind. This is specialist, one-off, high-consequence work. It is the right thing to bring in help for.

Keep the business knowledge and the ownership. No consultancy knows which of your revenue figures is the real one, why the finance team distrusts the CRM, or what active customer actually means in your business. That knowledge has to stay in-house and be fed into the engagement, not replaced by it. And crucially, keep ownership of the outcome: the platform is yours, not the consultancy’s.

Build towards handover from day one. The tell of a good partner is that they are trying to make themselves less necessary over time. The platform should be documented, the naming conventions sane, the pipelines readable, and your team trained to make routine changes. If a consultancy builds something only they can maintain, that is a commercial choice on their part, not a technical necessity, and it is the thing to guard against.

The honest line is this: outsource the specialist build, keep the business context and the ownership, and insist on enablement so you are not dependent forever. A consultancy that resists that split is telling you something.

What good looks like in an engagement

A few things separate a partner worth paying from a body-shop day rate.

They scope to an outcome, not an open-ended time-and-materials meter. Fixed, phased delivery, assessment, then build, then optimise, protects you from the project that quietly runs forever. You should know what each phase delivers and what it costs before it starts.

They start with an assessment, not a build. Anyone who wants to start pouring concrete before understanding your sources, your data quality, and your actual reporting needs is optimising for their invoice, not your outcome.

They build governance and observability in, not as a phase-two upsell. If data quality checks, lineage and SLAs are presented as optional extras, the platform is being built to look finished rather than to be trusted.

They hand it over properly. Documentation, training, and a platform your team can run. The exit is part of the deliverable.

What it should cost (roughly)

Prices vary with scope, but so you are not going in blind: a focused Azure data platform build for a mid-market business, ingestion from a handful of core sources, a governed lakehouse or warehouse, a clean semantic model, and proper handover, is typically a defined project measured in weeks, not an open-ended retainer. Beware two pricing shapes: the day rate with no fixed scope (unbounded), and the suspiciously cheap fixed price that quietly omits governance, testing and enablement (you pay the difference later, with interest).

The useful mental model is total cost of ownership, not sticker price. A platform that is cheap to build and expensive to change, because only the builder understands it, costs more over three years than one that is priced fairly and handed over cleanly. Ask every prospective partner the same question: when this is done, what can my own team do without calling you? The answers will sort them quickly.

Frequently asked questions

Should we outsource Azure data engineering or hire in-house?

For most mid-market businesses, the right answer is both, split deliberately. Outsource the specialist one-off work, platform architecture, the initial build, migration, where experience prevents expensive mistakes. Keep the business knowledge, data definitions and ownership in-house, and insist the partner trains your team to run and change the platform. A full-time senior data engineer is hard to hire and retain for what is often project-shaped work; a good partner plus an enabled internal owner is usually the better economics.

What should an Azure data engineering project include?

Ingestion from your source systems, a storage and modelling architecture (lakehouse or warehouse), transformation with business rules applied consistently, governance and observability (lineage, access control, data quality checks, SLAs), and proper enablement, documentation and training so your team can own it. If governance and enablement are missing, the project is not finished, it just looks finished.

How much does an Azure data platform cost to build?

It should be a defined, phased project measured in weeks with a known price per phase, not an open-ended day rate. Watch for two traps: unbounded time-and-materials with no fixed scope, and a cheap fixed price that omits governance, testing and handover. Judge on total cost of ownership over three years, not the initial quote, a platform only the builder can maintain is the expensive option however it is priced.

How do we avoid being locked in to a consultancy?

Insist on handover as a deliverable from the start: documentation, sane naming conventions, readable pipelines, and training for your own team. The test is simple: ask what your team will be able to do without calling the partner once the work is done. A good consultancy is comfortable with that question because they are building towards making themselves optional.

Hopton Analytics delivers Azure data engineering for UK mid-market businesses in fixed, outcome-based phases, built to be handed over, not held hostage. If you are weighing up bringing in help and want a straight answer on scope and cost, book a free audit.