AWS Systems Manager Parameter Store
The setup steps covers the use of the managed version of the AWS Systems Manager Parameter Store as secrets manager but for the non-managed follow only the steps related to the Airflow server and CLI.
Setup
1. Permissions needed
These are the permissions required in the IAM policy to enable the AWS Systems Manager Parameter Store in OpenMetadata.
2. Update configuration
We have to set up the secret manager provider we want to use, that in our case is aws-ssm
, and the credentials for our AWS account.
The changes to be done in openmetadata.yaml
file of the OpenMetadata server are:
And these are the changes required in airflow.cfg
of our Airflow instance:
As an alternative to editing the airflow.cfg
file, we can also set the following environment variables:
If no parameters are provided for the AWS account, or only <aws region>
, it will use the default credentials. The default credential will look for credentials in:
- Environment variables -
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
. - Shared credential file -
~/.aws/credentials
- AWS config file -
~/.aws/config
- Assume Role provider
- Instance metadata service on an Amazon EC2 instance that has an IAM role configured
More info in AWS SDK for Java and Boto3 Docs
3. Restart both servers
After updating the configuration files, we are ready to restart both services. When the OM server starts, it will automatically detect that a Secrets Manager has been configured and will migrate all our sensitive data and remove it from our DB.
If everything goes as planned, all the data would be displayed using the parameters names which starts with /openmetadata/...
in your AWS Systems Manager Parameter Store console. The following image shows what it should look like:

Note: If we want to change the starting path for our secrets names from openmetadata
to a different one, we have to change the property clusterName
in our openmetadata.yaml
CLI
After enabling the Secret Manager, we also have to make a slight change in our workflows YAML files. In the workflowConfig
we have to add the secret manager configuration: