Migrations Data & Model Migration - FAQs

You deal with poor data quality when migrating historical data by measuring it early and deciding deliberately what to fix, where and by whom. A migration usually exposes problems that the old system hid: duplicate customers, missing codes, inconsistent product hierarchies, transactions posted to the wrong period. Profile the source data before you build, so the problems are known and costed instead of discovered during testing. Then decide case by case. Fix at source where the operational system is wrong. Apply rules in the transformation layer where the fix is consistent and agreed. Flag and report what can't be fixed yet, so users know which figures to treat with care. Avoid silently cleansing data in the new platform without telling anyone, because the new numbers will then disagree with the source and trust will suffer. Our article on why data quality is a habit, not a project explains how to keep it from creeping back.

You handle personal data and GDPR when migrating historical data by treating the migration as a chance to apply your retention and minimisation rules, not just to copy everything across. Identify which tables hold personal data before you move anything. Ask whether the new reporting platform needs it at all. Many reports need a customer reference, not a name, address or date of birth. Apply your retention policy, so data you are no longer entitled to keep is not migrated. Make sure test and development environments don't hold real personal data unless they are protected to the same standard as production. Record what moved where, because a subject access or deletion request later will need it. If the migration changes where data is stored or who processes it, your privacy notice and records of processing may need updating. Our article on AI, your data and UK data protection covers the wider obligations.

You migrate Azure Data Factory pipelines and transformations into Microsoft Fabric using Microsoft's upgrade path, which lets you view existing ADF pipelines in Fabric, assess readiness and then upgrade them pipeline by pipeline. Fabric Data Factory shares most of the same activities and concepts, so straightforward pipelines move with modest effort. The work is in the differences. Linked services become Fabric connections, integration runtimes become on-premises or virtual network data gateways, and some activities or parameter patterns need reworking. Mapping data flows don't move across directly and are usually rebuilt in Dataflows Gen2 or notebooks. There is no need to move everything at once. ADF and Fabric can run side by side, so critical pipelines can stay where they are until their replacements are proven. Our guide to notebooks, Dataflows Gen2 or pipelines helps decide where rebuilt logic should live, and Fabric versus legacy Azure covers the wider decision.

You migrate SQL Server Analysis Services (SSAS) models to Power BI or Fabric differently depending on the type of model. Tabular models are the easier case. They use the same engine and DAX language as Power BI semantic models, so they can usually be moved with tools such as Tabular Editor and then repointed at new data sources, with testing on measures, security roles and refresh. Multidimensional cubes are harder. MDX calculations, cube structures and some features have no direct equivalent, so they need redesigning as a tabular model, not converting. That is often a chance to simplify years of accumulated calculations. In both cases, check how users connect today. Excel pivot tables connected to SSAS can usually point at the new semantic model, which keeps disruption low. Our article on why the semantic model is the asset explains why getting the model right matters most, and our Power BI page covers how we build them.

You migrate SSIS packages and other ETL to Microsoft Fabric by rebuilding the logic in Fabric's own tools, not by copying packages across unchanged. Start by cataloguing every package: what it reads, what it writes, its schedule and which reports depend on it. Many estates find a good share are obsolete. Simple copy-and-load packages often become Copy Jobs or pipeline copy activities. Transformations move into notebooks, Dataflows Gen2 or warehouse SQL, depending on complexity and who will maintain them. Orchestration moves into Fabric pipelines. Logic buried in script tasks or stored procedures needs the most care, because it is usually undocumented. Moving to a layered, metadata-driven design during migration means new sources become configuration changes, not new packages. For a period you can keep critical packages running in Azure Data Factory while the rest are rebuilt. Our guide to notebooks, Dataflows Gen2 or pipelines explains where each piece of logic belongs.

You reconcile the numbers after migrating historical data to a new platform by comparing the old and new systems at several levels, agreed in advance, instead of relying on someone eyeballing a report. Start with row counts per table and per period. Then compare control totals for the measures that matter, such as revenue, cost and quantity, by month, entity and key dimensions. Then check a sample of individual records end to end. Agree tolerances up front: some differences are expected, such as rounding or corrected historical errors, and each one should be explained and signed off by the business owner, not quietly accepted. Automate the checks so they can be rerun after every load during testing and the parallel run. A migration is finished when finance and operations trust the new numbers, not when the data has moved. Our article on testing a data pipeline covers the checks in detail.

How many years of historical data you should migrate depends on what the business actually uses it for, not on how much exists. Most reporting needs the current and previous year for comparison, with three to five years for trend and seasonality analysis. Forecasting and some regulatory or audit requirements can justify longer. Every extra year adds load time, reconciliation effort and, if the old data is messy, cleansing cost. A common approach is to migrate recent years in full detail and older years at a summarised level, such as monthly totals by key dimension, which keeps long-term trends without the effort of reconciling every old transaction. Whatever you leave behind should be archived somewhere accessible, not deleted, in case it is needed later. Agree the cut-off with finance and any compliance owner before you start. Our article on forecasting on the data you already have shows how much history forecasting really needs.

Whether to lift and shift a data model or redesign it during a migration depends on whether the current model is the problem. Lift and shift is faster and lower risk when the model is sound and the reason for moving is cost, support or platform end of life. Redesign is worth it when users don't trust the numbers, the same measure is calculated several ways, or performance is poor, because moving a flawed model to a new platform keeps the flaws and adds migration cost. Most migrations land in between. Move the parts that work, redesign the parts that cause trouble, and use the migration as the one moment when changing definitions is expected and accepted. Agree that scope at the start, because redesign decided halfway through is how migrations overrun. Our article on Microsoft Fabric versus legacy Azure covers the platform side, and our free audit can tell you which parts of your model are worth keeping.

A safe cutover when migrating data models and historical data is a rehearsed event with a fixed date, a checklist and a way back. In the weeks before, run the old and new systems in parallel and reconcile them until the differences are understood and signed off. Agree a freeze on changes to the old system so you aren't chasing a moving target. Rehearse the final load at least once, timing each step. On the day, run the final load, run the reconciliation checks, switch users to the new reports and confirm sign-off from the business owners. Keep the old system available read-only for an agreed period in case anything needs checking, and only then decommission it. Tell users in advance what is changing, when, and who to contact. Most cutover problems come from surprises, not technology. Our article on what happens after go-live covers the stabilisation weeks that follow.

Migrating data models, transformations and historical data to Power BI or Microsoft Fabric involves more than copying tables. It has five parts. First, you inventory what exists: sources, data models, transformations and the reports that depend on them. Second, you decide what to move, redesign or retire, because most estates carry data nobody uses. Third, you rebuild the transformation logic in the new platform, usually as layered pipelines in a lakehouse or warehouse. Fourth, you load history and reconcile it, so the new figures match the old ones or the differences are explained. Fifth, you cut over, with users moved across and the old system switched off on a known date. The part most often underestimated is the logic hidden in old stored procedures, spreadsheets and report formulas, which nobody wrote down. Our data platform and warehouse page covers how we approach it, and the Synapse to Fabric migration guide shows the steps in practice.

A BI migration needs a new data model because the old platform's model, whether Tableau extracts and blends, Qlik's associative model or SQL views behind SSRS, is shaped for that tool, and Power BI performs and behaves correctly only over a star schema with measures defined once. Rebuilding the model is also where the inconsistencies that prompted the migration get fixed. Star schema explained describes the shape Power BI wants.

Historical data is important when replacing an ERP system because most organisations underestimate the value of the years of historical transaction data sitting in their existing system. When a replacement happens, this data is often left behind or transferred in a degraded form. Historical data is the asset that makes machine learning, forecasting, and pattern detection possible. Cohort analysis needs years of customer history. Demand forecasting needs years of sales history. Replacement projects regularly throw this asset away because the migration plan cannot afford to bring it across cleanly.