Skip to main content
In this section, we provide guides and references to use the PowerBI connector.
Supported Authentication Types:
  • OAuth 2.0 Service Principal — Azure AD application authentication using Client ID, Client Secret, and Tenant ID
Configure and schedule PowerBI metadata and profiler workflows from the OpenMetadata UI:

​
Requirements

To access the PowerBI APIs and import dashboards, charts, and datasets from PowerBI into OpenMetadata, a PowerBI Pro license is necessary.
PowerBI dataflows are not yet supported.
OpenMetadata does not support Power BI usage ingestion because the Power BI Usage API does not support Service Principal authentication.
When configuring Azure Authentication, ensure that “Allow public client flows” is enabled. This setting is required to support authentication for public client applications.

​
PowerBI Admin and Non-Admin APIs:

While configuring the PowerBI ingestion you can choose whether to use the PowerBI Admin APIs to retrieve the metadata or use the PowerBI Non-Admin APIs. Please check below for the the difference in their functionality:
  • Enabled (Use PowerBI Admin APIs) Using the admin APIs will fetch the dashboard and chart metadata from all the workspaces available in the PowerBI instance.
When using the PowerBI Admin APIs, the table and dataset information used to generate lineage is gathered using the PowerBI Scan Result API. This API has no limitations and hence does not restrict getting the necessary data for generating lineage.
  • Disabled (Use Non-Admin PowerBI APIs) Using the non-admin APIs will only fetch the dashboard and chart metadata from the workspaces that have the security group of the service principal assigned to them.
When using the PowerBI Non-Admin APIs, the table and dataset information used to generate lineage is gathered using the PowerBI Get Dataset Tables API. This API only retrieves the table information if the dataset is a Push Dataset. Hence the lineage can only be created for push datasets in this case. For more information please visit the PowerBI official documentation here.

​
PowerBI Account Setup

Follow the steps below to configure the account setup for PowerBI connector:

​
Step 1: Enable API permissions from the PowerBI Admin console

We extract the information from PowerBI using APIs, this is a manual step a PowerBI Admin needs to do to ensure we can get the right information. Login to the Power BI as Admin and from Tenant settings allow below permissions.
  • Allow service principles to use Power BI APIs
  • Allow service principals to use read-only Power BI admin APIs
  • Enhance admin APIs responses with detailed metadata

​
Step 2: Create the App in Azure AD

Please follow the steps mentioned here for setting up the Azure AD application service principle.

​
Step 3: Provide necessary API permissions to the Azure AD app

Go to the Azure Ad app registrations page, select your app and add the dashboard permissions to the app for PowerBI service and grant admin consent for the same: The required permissions are:
  • Dashboard.Read.All Optional Permissions: (Without granting these permissions, the dataset information cannot be retrieved and the datamodel and lineage processing will be skipped)
  • Dataset.Read.All
Make sure that in the API permissions section Tenant related permissions are not being given to the app Please refer here for detailed explanation

​
Step 4: PowerBI Workspaces

The service principal does not take into account the default user workspaces e.g My Workspace. Create new workspaces in PowerBI by following the document here For reference here is a thread referring to the same

​
Metadata Ingestion

​
Connection Details

1

Connection Details

When using a Hybrid Ingestion Runner, any sensitive credential fields—such as passwords, API keys, or private keys—must reference secrets using the following format:
password: secret:/my/database/password
This applies only to fields marked as secrets in the connection form (these typically mask input and show a visibility toggle icon). For a complete guide on managing secrets in hybrid setups, see the Hybrid Ingestion Runner Secret Management Guide.
clientId: PowerBI Client ID. To get the client ID (also known as application ID), follow these steps:
  • Log into Microsoft Azure.
  • Search for App registrations and select the App registrations link.
  • Select the Azure AD app you’re using for embedding your Power BI content.
  • From the Overview section, copy the Application (client) ID. clientSecret: PowerBI Client Secret. To get the client secret, follow these steps:
  • Log into Microsoft Azure.
  • Search for App registrations and select the App registrations link.
  • Select the Azure AD app you’re using for embedding your Power BI content.
  • Under Manage, select Certificates & secrets.
  • Under Client secrets, select New client secret.
  • In the Add a client secret pop-up window, provide a description for your application secret, select when the application secret expires, and select Add.
  • From the Client secrets section, copy the string in the Value column of the newly created application secret. tenantId: PowerBI Tenant ID. To get the tenant ID, follow these steps:
  • Log into Microsoft Azure.
  • Search for App registrations and select the App registrations link.
  • Select the Azure AD app you’re using for Power BI.
  • From the Overview section, copy the Directory (tenant) ID. scope: Service scope. To let OM use the Power BI APIs using your Azure AD app, you’ll need to add the following scopes:
  • https://analysis.windows.net/powerbi/api/.default Instructions for adding these scopes to your app can be found by following this link: https://analysis.windows.net/powerbi/api/.default. authorityUri: Authority URI for the service. To identify a token authority, you can provide a URL that points to the authority in question. If you don’t specify a URL for the token authority, we’ll use the default value of https://login.microsoftonline.com/. hostPort: URL to the PowerBI instance. To connect with your Power BI instance, you’ll need to provide the host URL. If you’re using an on-premise installation of Power BI, this will be the domain name associated with your instance. If you don’t specify a host URL, we’ll use the default value of https://app.powerbi.com to connect with your Power BI instance. Pagination Entity Per Page: The pagination limit for Power BI APIs can be set using this parameter. The limit determines the number of records to be displayed per page. By default, the pagination limit is set to 100 records, which is also the maximum value allowed. Use Admin APIs: Option for using the PowerBI admin APIs: Refer to the section here to get more information.
  • Enabled (Use PowerBI Admin APIs)
  • Disabled (Use Non-Admin PowerBI APIs)
2

Test the Connection

Once the credentials have been added, click on Test Connection and Save the changes.Test Connection
3

Configure Metadata Ingestion

After adding and testing the dashboard service, configure the metadata ingestion pipeline. To configure, follow the steps below:
  1. Navigate to Settings > Services > Dashboards.
  2. On the Dashboards services page, click the service you’ve added.
  3. Go to the Agents tab, and then click Add Agent > Add Metadata Agent.
  4. Configure the ingestion details. See Metadata Ingestion Options.
Configure Metadata Ingestion

​
Metadata Ingestion Options

  • Name: This field is the name of the ingestion pipeline. Customize it or use the generated name.
  • Dashboard Filter Pattern (Optional): Use it to control whether to include dashboards as part of metadata ingestion.
    • Include: Explicitly include dashboards by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be excluded.
    • Exclude: Explicitly exclude dashboards by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be included.
  • Chart Filter Pattern (Optional): Use it to control whether to include charts as part of metadata ingestion.
    • Include: Explicitly include charts by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all charts with names matching one or more of the supplied regular expressions. All other charts will be excluded.
    • Exclude: Explicitly exclude charts by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all charts with names matching one or more of the supplied regular expressions. All other charts will be included.
  • Data Model Filter Pattern (Optional): Use it to control whether to include data models as part of metadata ingestion.
    • Include: Explicitly include data models by adding comma-separated regular expressions to the ‘Include’ field. OpenMetadata will include all data models with names matching one or more of the supplied regular expressions. All other data models will be excluded.
    • Exclude: Explicitly exclude data models by adding comma-separated regular expressions to the ‘Exclude’ field. OpenMetadata will exclude all data models with names matching one or more of the supplied regular expressions. All other data models will be included.
  • Project Filter Pattern: Filter the dashboards, charts, and data sources by projects. Note that all of them support regex as include or exclude. For example, “My project, My proj.*, .*Project”.
    We filter the projects by concatenating the entire project hierarchy using dot notation (for example, Project1.NestedProjectA.OtherProject). Make sure the regex filter pattern accounts for this fully-qualified format.
  • Enable Debug Log: Enable this toggle to use debug-level logging.
Metadata Ingestion Option 1
  • Lineage Information (Optional): Configure this section to enable lineage between your dashboards and the database tables they are built on. OpenMetadata uses the database service name to match and draw the lineage path from table to dashboard.
    • Db Service Prefixes: Enter one or more service path prefixes to tell OpenMetadata where to look for the source tables used by your dashboards. Supported formats:
      • DBServiceName—matches all tables in the service
      • DBServiceName.DatabaseName—matches tables in a specific database
      • DBServiceName.DatabaseName.SchemaName—matches tables in a specific schema
      • DBServiceName.DatabaseName.SchemaName.TableName—matches a specific table
  • Query Parser Configuration: Controls how OpenMetadata parses SQL queries to extract lineage. Use this to select the SQL parser that best fits your data source’s dialect.
    • Query Parser Type: Choose the SQL parser for lineage extraction:
      • Auto (default): Automatically tries SqlGlot first, falls back to SqlFluff, then SqlParse. Recommended for best results.
      • SqlGlot: High-performance parser with good dialect support. Falls back to SqlParse on failure.
      • SqlFluff: Comprehensive but slower parser with strong dialect support. Falls back to SqlParse on failure.
  • Include Current Owners: Enable this toggle to control whether to include owners for the ingested entity if the owner email matches a user stored in the OpenMetadata server as part of metadata ingestion. If the ingested entity already exists and has an owner, the owner will not be overwritten.
  • Mark Deleted Dashboards: Optional configuration to soft delete dashboards in OpenMetadata if the source dashboards are deleted. After deleting, all associated entities like lineage and other related data for that dashboard will be deleted.
  • Mark Deleted Data Models: Optional configuration to soft delete data models in OpenMetadata if the source data models are deleted. After deleting, all associated entities with that data model will be deleted.
  • Mark Deleted Charts: Optional configuration to soft delete charts in OpenMetadata if the source charts are deleted. After deleting, all associated entities with that chart will be deleted.
  • Include Tags: Enable this toggle to control whether to include tags in metadata ingestion.
  • Include Data Models: Enable this toggle to control whether to include data models as part of metadata ingestion.
  • Include Draft Dashboard: Enable this toggle to include draft dashboards. By default, this is enabled.
  • Include Usage: Enable this toggle to control whether to include usage data as part of metadata ingestion.
  • Override Metadata: Enable this toggle to control whether to override the existing metadata in the OpenMetadata server with the metadata fetched from the source. If enabled, the metadata fetched from the source will override the existing metadata in OpenMetadata. If disabled, only fields that have no value in OpenMetadata will be updated. This is applicable for fields like description, tags, owner, and displayName.
  • Override Lineage: Enable this toggle to control whether to override the existing lineage in OpenMetadata with the lineage fetched from the source. If enabled, existing lineage will be replaced. If disabled, new lineage edges will be added without removing existing ones.
Metadata Ingestion Option 2
4

Schedule the Ingestion and Deploy

Scheduling can be set up at an hourly, daily, weekly, or manual cadence. The timezone is in UTC. Select a Start Date to schedule for ingestion. It is optional to add an End Date.Review your configuration settings. If they match what you intended, click Deploy to create the service and schedule metadata ingestion.If something doesn’t look right, click the Back button to return to the appropriate step and change the settings as needed.After configuring the workflow, you can click on Deploy to create the pipeline.Schedule the Workflow
5

View the Ingestion Pipeline

Once the workflow has been successfully deployed, you can view the Ingestion Pipeline running from the Service Page.View Ingestion Pipeline
If AutoPilot is enabled, workflows like usage tracking, data lineage, and similar tasks will be handled automatically. Users don’t need to set up or manage them - AutoPilot takes care of everything in the system.

​
Lineage

Lineage in OpenMetadata shows you which database tables power each dashboard. When lineage is set up, you can trace any dashboard back to the exact source tables in your database. There is no separate lineage agent or lineage pipeline—lineage is collected as part of the same metadata ingestion workflow. Configuring the Db Service Prefixes field (covered below) is optional but recommended — it restricts table matching to specific database services. If left blank, OpenMetadata attempts to match source tables across all ingested database services. lineage Information

​
How to Set Up Lineage

  1. Go to Settings > Services > Dashboards.
  2. Click the dashboard service you’ve added.
  3. Go to the Agents tab, then select Add Agent > Add Metadata Agent. If a metadata agent already exists, select the three-dot context menu (⋮) next to it and select Edit.
  4. In the Configure Ingestion step, scroll down to the Lineage Information section.
  5. Optionally enter one or more database service names in the Db Service Prefixes field to restrict table matching to specific services. If left blank, OpenMetadata searches across all ingested database services.
The Db Service Prefixes field helps OpenMetadata locate the source tables and draw the lineage path from table to dashboard. Examples of valid entries:
What you enterWhat it matches
SnowflakeProdAll tables in the Snowflake service named SnowflakeProd
SnowflakeProd.sales_dbTables in the sales_db database within SnowflakeProd
SnowflakeProd.sales_db.publicTables in the public schema within sales_db
If your dashboards pull data from multiple database services, add each service as a separate entry in the Db Service Prefixes field.