Ingest Data Products from dbt

Assign tables to Data Products from your dbt project. List the Data Products under meta.openmetadata.dataProducts in your dbt model. During dbt ingestion, OpenMetadata adds the table as an asset of each listed Data Product. A Data Product is a curated group of data assets that is managed and shared as a single unit. To learn more, see Data Products.

Requirements

The Data Product must already exist in OpenMetadata before ingestion. OpenMetadata never creates Data Products from dbt. If a listed Data Product isn’t found, OpenMetadata logs a warning and ingestion continues. If the Data Product belongs to a Domain, the table’s Domain must match it. Otherwise, OpenMetadata rejects the assignment. To set the table’s Domain from dbt, see Ingest Domain from dbt.

Steps for Ingesting dbt Data Products

Follow these steps to assign tables to Data Products from dbt.

1. Create or Select a Data Product in OpenMetadata

Create the Data Product in OpenMetadata, or use an existing one. Note its name. For more information, see Add a Data Product.

2. Add Data Products in the schema.yml File

Add a list of Data Product names under model->name->meta->openmetadata->dataProducts in your schema.yml file. A table can belong to more than one Data Product. For more details on the dbt meta field, see the dbt documentation.

models:
  - name: customers
    meta:
      openmetadata:
        dataProducts:
          - "Marketing"
          - "Customer360"
    description: This table has basic information about a customer, as well as some derived facts based on a customer's orders

After you add the Data Products, run your dbt workflow. The generated manifest.json file then reflects the assignment under node_name->config->meta->openmetadata->dataProducts.

"model.jaffle_shop.customers": {
  "resource_type": "model",
  "database": "dev",
  "schema": "dbt_jaffle",
  "config": {
    "meta": {
      "openmetadata": {
        "dataProducts": ["Marketing", "Customer360"]
      }
    }
  }
}

3. View the Data Products on Tables

After ingestion, the table appears in the Assets of each Data Product. It also shows the Data Products on its details page.

Data Products section on a table details page showing the Data Product assigned from dbt

Validation and Error Handling

Scenario	Behavior
Data Product not found in OpenMetadata	A warning is logged, the Data Product is skipped, and ingestion continues
Assignment fails, for example because the table’s Domain doesn’t match the Data Product’s Domain	A warning is logged, the Data Product is skipped, and ingestion continues
`dataProducts` not set	No Data Product assignment
Workflow re-run	No duplicates. A table already in the Data Product stays as it is

Connectors

Connector

Ingestion

Ingest Data Products from dbt | OpenMetadata Data Product Assignment Guide

Ingest Data Products from dbt

Requirements

Steps for Ingesting dbt Data Products

1. Create or Select a Data Product in OpenMetadata

2. Add Data Products in the schema.yml File

3. View the Data Products on Tables

Validation and Error Handling

​Ingest Data Products from dbt

​Requirements

​Steps for Ingesting dbt Data Products

​1. Create or Select a Data Product in OpenMetadata

​2. Add Data Products in the schema.yml File

​3. View the Data Products on Tables

​Validation and Error Handling

Ingest Data Products from dbt

Requirements

Steps for Ingesting dbt Data Products

1. Create or Select a Data Product in OpenMetadata

2. Add Data Products in the schema.yml File

3. View the Data Products on Tables

Validation and Error Handling