> ## Documentation Index
> Fetch the complete documentation index at: https://docs.open-metadata.org/llms.txt
> Use this file to discover all available pages before exploring further.
# dbt Artifact Configuration Guide | OpenMetadata
> Choose the right storage method for your dbt artifacts with OpenMetadata. Compare S3, GCS, Azure, HTTP, and Local storage options.
# dbt Artifact Configuration
**Using dbt Cloud?** You don't need storage configuration. Go directly to the [dbt Cloud API guide](/v1.12.x/connectors/database/dbt/dbt-cloud-api-guide) for a simpler setup.
This guide is for **dbt Core** users. When using dbt Core, OpenMetadata needs access to dbt-generated artifacts (manifest.json, catalog.json, run\_results.json) to extract metadata, lineage, and test results. Since dbt Core runs within your infrastructure, you must configure a storage method to make these artifacts accessible to OpenMetadata.
## How It Works
If you're using **dbt Core**, you need to set up artifact storage:
1. **Generate artifacts**: Run `dbt run`, `dbt test`, and `dbt docs generate` to create manifest.json, catalog.json, and run\_results.json
2. **Upload to storage**: Configure your workflow to upload these files to S3, GCS, Azure, HTTP server, or shared filesystem
3. **Configure OpenMetadata**: Point OpenMetadata to the storage location to pull and process the artifacts
## Storage Options for dbt Core
Choose the storage method that matches your infrastructure:
Complete S3 setup with Airflow DAG, IAM policies, and configuration
GCS bucket setup with service accounts and Cloud Composer integration
Azure storage account setup with managed identity and container config
Nginx, Apache, or S3+CloudFront configuration for artifact hosting
Docker volumes, Kubernetes PVC, or NFS shared filesystem setup
## Quick Setup Summary
### AWS S3
```bash theme={null}
# Create bucket
aws s3 mb s3://your-dbt-artifacts
# Upload artifacts (manual)
aws s3 sync target/ s3://your-dbt-artifacts/dbt/ --include "*.json"
# Or use automated Airflow DAG (see detailed guide)
```
[View complete S3 guide →](/v1.12.x/connectors/database/dbt/storage-s3-guide)
### Google Cloud Storage
```bash theme={null}
# Create bucket
gsutil mb gs://your-dbt-artifacts
# Upload artifacts (manual)
gsutil -m cp target/*.json gs://your-dbt-artifacts/dbt/
# Or use Cloud Composer DAG (see detailed guide)
```
[View complete GCS guide →](/v1.12.x/connectors/database/dbt/storage-gcs-guide)
### Azure Blob Storage
```bash theme={null}
# Create storage account and container
az storage account create --name yourdbtartifacts
az storage container create --name dbt-artifacts
# Upload artifacts
az storage blob upload-batch \
--destination dbt-artifacts \
--source target/ \
--pattern "*.json"
```
[View complete Azure guide →](/v1.12.x/connectors/database/dbt/storage-azure-guide)
### HTTP Server
```bash theme={null}
# Upload via rsync to static file server
rsync -avz target/*.json user@server:/var/www/dbt/
# Or configure S3 + CloudFront for HTTPS access
```
[View complete HTTP guide →](/v1.12.x/connectors/database/dbt/storage-http-guide)
### Local/Shared Filesystem
```yaml theme={null}
# Docker Compose volume mount example
volumes:
- ./dbt/target:/dbt-artifacts
# OpenMetadata reads directly from /dbt-artifacts path
```
[View complete Local guide →](/v1.12.x/connectors/database/dbt/storage-local-guide)
## Common Requirements Across All Methods
Regardless of which storage method you choose, you need:
### 1. Required dbt Artifacts
| File | Generated By | Purpose |
| ------------------ | ------------------------------------- | --------------------------------------------------------- |
| `manifest.json` | `dbt run`, `dbt compile`, `dbt build` | **Required** - Model definitions, sources, lineage, tests |
| `catalog.json` | `dbt docs generate` | Recommended - Column names, types, descriptions |
| `run_results.json` | `dbt test`, `dbt run`, `dbt build` | Optional - Test execution results and timing |
### 2. dbt Command Sequence
Run these commands to generate all artifacts:
```bash theme={null}
dbt run # Execute models, generates manifest.json
dbt test # Run tests, updates run_results.json
dbt docs generate # Generate catalog.json with column metadata
```
### 3. OpenMetadata Configuration
After artifacts are accessible, configure OpenMetadata ingestion:
* **UI Method**: See [Configure dbt Workflow](/v1.12.x/connectors/database/dbt/configure-dbt-workflow)
* **CLI Method**: See [Run dbt Workflow Externally](/v1.12.x/connectors/database/dbt/run-dbt-workflow-externally)
* **Auto-ingest**: See [Auto Ingest dbt Core](/v1.12.x/connectors/database/dbt/auto-ingest-dbt-core)
## Troubleshooting Common Issues
| Issue | Possible Cause | Solution |
| ------------------------------- | ----------------------------- | --------------------------------------------------------------- |
| **Artifacts not found** | Upload failed or wrong path | Verify artifacts uploaded successfully to correct location |
| **Access denied** | Insufficient permissions | Check IAM policies, service account permissions, or access keys |
| **Stale metadata** | Old artifacts | Ensure upload happens after dbt completes; verify timestamps |
| **Missing lineage** | No compiled\_code in manifest | Run `dbt compile` or `dbt docs generate` before upload |
| **Missing column descriptions** | No catalog.json | Ensure `dbt docs generate` runs and catalog.json is uploaded |
| **Test results not showing** | No run\_results.json | Ensure `dbt test` runs and run\_results.json is uploaded |
For storage-specific troubleshooting, see the individual guides.
## Next Steps
1. **Choose your storage method** using the decision matrix above
2. **Follow the detailed guide** for your chosen method
3. **Configure OpenMetadata ingestion** after artifacts are accessible
4. **Set up scheduling** to keep metadata synchronized
Questions? See the main [dbt Overview](/v1.12.x/connectors/database/dbt) or [dbt Troubleshooting](/v1.12.x/connectors/database/dbt/dbt-troubleshooting) guide.