Deployment successful is not the same as production-ready

A green tick in a Power BI deployment pipeline means the metadata moved. It does not mean the data refreshed, the gateway connected or the right people can see the report.

A Power BI deployment pipeline will tell you when a deployment has succeeded. It is worth being precise about what that means.

It means the item definitions moved from one stage to the next. It does not mean the data refreshed. It does not mean the gateway is connected, the credentials are in place, or the right people can see the report. A successful deployment is one step. Production-ready is a different claim.

What a deployment does not copy

Deployment pipelines copy metadata, not everything around it. For semantic models in particular, several things stay behind in the source stage and have to be in place in the target:

• The data itself. A newly deployed model has no data until it refreshes.
• Data source credentials.
• The refresh schedule.
• Role assignments, meaning who is a member of each row-level security role. The role definitions move. The people in them do not.
• Workspace and item permissions.
• App content and settings. The app in each stage has to be updated separately.

Once these are set in a stage, later deployments leave them alone. That is helpful. It also means the first deployment into a new stage needs the most care, and a mistake made then can sit quietly for months.

The five checks after every deployment

Run these in the target workspace, not the source.

• Data refresh. Trigger a refresh and confirm it completes. Then confirm the scheduled refresh is set and will run.
• Gateway mapping. If the model uses on-premises or private network data, check it is mapped to the correct gateway and data source for this stage.
• Data source credentials. Check credentials are set for this environment, and that they are the production credentials, not a developer's own account.
• Report visuals. Open the report. Check that visuals load, filters behave and the key figures reconcile to a known number. A total that matches TEST is a good sign. A total that matches the source system is better.
• Security and access. Check row-level security roles have the right members, and test as a user with restricted access. Check the right people and groups can reach the app, and that it has been updated.

None of these checks is difficult. The risk is that they live in one person's head and get skipped under time pressure.

Make it a written checklist, with a name against it

The simplest fix is to write the checks down and agree who signs off each deployment to production. Five lines, ticked, with a name and a date. It takes a few minutes per release and gives you a record of who checked what.

Habits like this are what separate a governed estate from a collection of reports. Putting them in place across a team, not just for one report, is what our Analytics Acceleration Programme is built to do.

Row-level security deserves particular attention, because a mistake there is invisible to the person making it. A developer testing as themselves sees everything and concludes everything is fine. We wrote about why row-level security fails quietly, and the same logic applies after every deployment.

The same principle holds further up the stack. A pipeline run that succeeds is not proof the data is correct, which is the point of the pipeline succeeded, that doesn't mean the data is right.

Where this fits

This is the last step of a simple release habit. Build in DEV. Validate in TEST, deploying only what changed. Release to PROD. Then check PROD, because that is the only place the business will look.

If you are not sure how your current releases would hold up against these checks, a free analytics audit will tell you.

A green tick tells you the deployment worked. Only the checks tell you the report did.