dbt Workflow

dbt Integration

Requirements

AWS S3

If we have the artifacts on the bucket MyBucket, the user running the ingestion should have, at least, the permissions from the following policy:

Note that it's not enough to point the resource to arn:aws:s3:::MyBucket. We need its contents as well!

OpenMetadata integrates the below metadata from dbt

1. dbt Queries

Queries used to create the dbt models can be viewed in the dbt tab

dbt Query

2. dbt Lineage

Lineage from dbt models can be viewed in the Lineage tab.

For more information on how lineage is extracted from dbt take a look here

dbt Lineage

3. dbt Tags

Table and column level tags can be imported from dbt

Please refer here for adding dbt tags

dbt Tags

4. dbt Owner

Owner from dbt models can be imported and assigned to respective tables

Please refer here for adding dbt owner

dbt Owner

5. dbt Descriptions

Descriptions from dbt manifest.json and catalog.json can be imported and assigned to respective tables and columns.

For more information and to control how the table and column descriptions are updated from dbt please take a look here

dbt Descriptions

6. dbt Tests and Test Results

Tests from dbt will only be imported if the run_results.json file is passed.

dbt Tests

7. dbt Tiers

Table and column level Tiers can be imported from dbt

Please refer here for adding dbt tiers

dbt Tiers

8. dbt Glossary

Table and column level Glossary can be imported from dbt

Please refer here for adding dbt glossary

Troubleshooting

For any issues please refer to the troubleshooting documentation here