how to refresh individual tables or partitions in power bi semantic models from fabric data pipelines.
introduction.
With the new “refresh semantic model” pipeline activity (still in preview), triggering refreshes from Fabric Data Pipelines have become a lot easier. This activity, however, does not provide an option to refresh only certain objects of the semantic model, like partitions or tables. Looking into the JSON source code of that pipeline activity, we can see a property called operationType (see picture below). I can imagine, at some point, other operations like the refresh of single tables or partitions and maybe even more, might be possible, who knows… Until then, if we want to refresh only single tables from Fabric Data Pipelines, we can follow a similar setup like we did when we used Azure Data Factory for the same purpose. We’ll demonstrate a similar method in the step-by-step guide below. Another way of achieving the same is by using semantic-link in a notebook that is run by a Fabric Data Pipeline. as Sandeep has shown.
Also, there has been a frequent question of whether the new activity would also work in Azure Data Factory. Well, as per this comment I found on LinkedIn, at least for now, we still need to go for a Web activity. But perhaps, this might change in the future, too. Also, I recommend checking out data-marc’s thoughts on the new activity. As always, he has some interesting takes on it.
prerequisites.
1. A Fabric capacity and workspace
2. A Fabric data pipeline
3. A Power BI semantic model
1. What’s the goal?
The goal is to refresh only certain partitions or tables of a semantic model. Below an example where we just refreshed the table random_dog_table:
2. Set up a new connection
At first, we need to set up a new web V2 connection with which we will call the API later in the pipeline. For this, click on the settings icon in the top right and then Manage connections and gateways.
Next, click on + New
Now, select the Cloud option, provide a name and fill in the other fields according to the picture. When you are done, press the Create button.
Connection type: Web V2
Base Url: https://api.powerbi.com/v1.0/myorg
Token Audience Uri: https://analysis.windows.net/powerbi/api
For the Authentication method, I selected OAuth 2.0. We need to provide our credentials by clicking on Edit credentials in order to be able to create the connection. Here, you log in as a user, and then the data pipeline authenticates to Fabric on your behalf. In a real life example, I strongly recommend using a service principal instead of the OAuth 2.0 method. In the appendix of this other post, I listed the steps necessary for utilising a service principal for authentication.
Note, for the Token Audience Uri as per the documentation, you might also be able to use https://api.powerbi.com. I tried it and was only able to create a connection when choosing Anonymous as the Authentication method. This threw me an error later on in the pipeline though.