OpenMetadata
Search…
MlFlow
This guide will help install the MlFlow connector and run it manually
Prerequisites
OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
  1. 1.
    Python 3.7 or above

Install from PyPI

Install Using PyPI
1
pip install 'openmetadata-ingestion[mlflow]'
Copied!

Run Manually

1
metadata ingest -c ./examples/workflows/mlflow.json
Copied!

Configuration

mlflow.json
1
{
2
"source": {
3
"type": "mlflow",
4
"config": {
5
"tracking_uri": "http://localhost:5000",
6
"registry_uri": "mysql+pymysql://mlflow:[email protected]:3307/experiments"
7
}
8
...
Copied!
  1. 1.
    tracking_uri - MlFlow server containing the tracking information of runs and experiments (docs).
  2. 2.
    registry_uri - Backend store where the Tracking Server stores experiment and run metadata (docs).

Publish to OpenMetadata

Below is the configuration to publish MlFlow data into the OpenMetadata service.
Add optionally pii processor and metadata-rest sink along with metadata-server config
mlflow.json
1
{
2
"source": {
3
"type": "mlflow",
4
"config": {
5
"tracking_uri": "http://localhost:5000",
6
"registry_uri": "mysql+pymysql://mlflow:[email protected]:3307/experiments"
7
}
8
},
9
"sink": {
10
"type": "metadata-rest",
11
"config": {}
12
},
13
"metadata_server": {
14
"type": "metadata-server",
15
"config": {
16
"api_endpoint": "http://localhost:8585/api",
17
"auth_provider_type": "no-auth"
18
}
19
}
20
}
Copied!
Last modified 22h ago