Follow this section if you added SSL certs in the OpenMetadata server.
The OpenMetadata configuration related to Airflow (or in general, the Pipeline Service Client) is the following:
Then, in order to add this, you can either update the
openmetadata.yaml config if your deployment is Bare Metal, or update the following environment variables:
Note that the
PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH should be the path to the certificate you generated here, and it should be the local path in your Airflow deployment.
Follow this section if you want to add SSL certificates in Airflow.
This will secure the connection from the OpenMetadata to Airflow.
Airflow has two configurations to be added in
airflow.cfg to enable SSL:
Those are files that will need to be local to the Airflow deployment.
We can generate these files following this SO thread:
and we can provide the following answers to try this locally:
It is important that the
Common Name is the host name that will be hosting Airflow.
This command will generate the pair
Once the files are generated we need to add them to the Airflow deployment. For example, if using the
openmetadata-ingestion image, you can update it to add the following lines:
If you now start Airflow with these changes, it will be running at
Since Airflow will be using SSL, we need to update the OpenMetadata Server configuration to use the certificates when preparing the connection to the Airflow Webserver.
For bare metal deployments, the
pipelineServiceClientConfiguration will look like the following:
truststorePassword accordingly, pointing to the
keystore in your server host holding the certificates we created.
For docker deployments, you will provide OpenMetadata Server Application with the self signed certificates of Airflow bundled in JVM keystore. These will be passed to the application using
AIRFLOW_TRUST_STORE_PASSWORD environment variable.
Please make sure to have the the truststore file mounted and available as part of Docker Deployments.
For kubernetes deployments, update the helm values as below -
In the above code snippet, we are mounting the volumes of truststore file from a kubernetes secret. You can create the secret from
truststore.jks file from the below
kubectl command -
Next, restart or redeploy openmetadata application to take the above configs in effect.
For example, if we are running the server locally, we need to add the certificate to the JVM
Then, the values of the YAML config would be something similar to:
Make sure to update these values to the ones in your host. Also, it's always preferred to use environment variables instead of hardcoding sensitive information.