Upgrade OpenMetadata
Releases
OpenMetadata community will be doing feature releases and stable releases.
- Feature releases are to upgrade your sandbox or POCs to give feedback to the community and any potential bugs that the community needs to fix.
- Stable releases are to upgrade your production environments and share it with your users.
0.13.2 - Stable Release
OpenMetadata 0.13.2 is a stable release. Please check the release notes
If you are upgrading production this is the recommended version to upgrade.
Breaking Changes for 0.13.2 Stable Release
EntityName
To better manage and harmonize entityName
value and allow users to form better expectations around these values the team introduced an enforcement of the entityName
format using regex pattern.
All the OpenMetadata entities entityName
fields will enforce by the default the following regex pattern:
^[\w'\- .&]+$
: match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters'- .&
. For example when creating a pipeline service the following name will not be allowedMyPipelineIngestion?!
while the following will beMy-Pipeline
.
Some entities are enforcing specific patterns:
- Users:
([\w\-.]|[^@])+$
match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters-.
and don't match the character@
- Webhook:
^[\w'\-.]+$
match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters-.
- Table:
^[\w'\- ./]+$
match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters'-. /
- Location:
^[\w'\-./]+$
match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters-.'/
- Type:
^[a-z][\w]+$
match any word characters (equivalent to[a-zA-Z0-9_]
) starting with a lowercase letter (e.g.tHisChar
will match,ThisChar
will not)
If an entity name does not follow the pattern an error will be returned by the OpenMetadata platform.
The change should be transparent for the end user and no action should be required.
EntityLink
Similar to the implementation done for entityName
, entityLink
will now enforce a specific pattern. The structure of entityLink
is in the form <#E::{entities}::{entityType}::{field}::{fieldName}::{fieldValue}>
All the OpenMetadata entities entityLink
fields will enforce by the default the following regex pattern:
^<#E::\w+::[\w'\- .&/\:+\"\\]+>$
: this means that{entities}
value needs to match any word characters (equivalent to[a-zA-Z0-9_]
) and the part after{entities}
can match any word characters (equivalent to[a-zA-Z0-9_]
) or the characters'- .&/\:+\"\
If an entity name does not follow the pattern an error will be returned by the OpenMetadata platform.
The change should be transparent for the end user and no action should be required.
Tags API
Tags APIs were coded long before other entities' APIs were added. Due to this, tag API does not follow the API convention that all the other entities are following. This issue makes backward incompatible changes to follow the same convention as glossaries.
You can find the full list of API paths changes in the following issue.
name
Metabase and Domo Dashboards With the new restrictions on the EntityName
and to ensure unicity of the assets, the Metabase and Domo Dashboard sources
now ingest the name
of the charts and dashboards with their internal ID value. Their name
value will be used
as the display name, but not as the OpenMetadata name
anymore.
The recommended approach here is to create a new service and ingest the metadata again. If you ingest from the same
service, the assets in there will end up being duplicated, as the name
in OpenMetadata is used to identify each asset.
Let us know if you have any questions around this topic.
Ingestion Framework sources directory structure
We have converted source modules (e.g., redshift.py
) into packages containing all the necessary information (e.g., redshift/metadata.py
).
This has helped us reorganise functionalities and easily focus on each connector independently.
If you're extending any of the sources, you'll need to update your imports. You can take a look at the new structure here.
MySQL Helm Chart Version Reverted to 8.8.23
OpenMetadata Helm Chart Release with Application Version 0.13.2
updates the Bitnami MySQL Helm Chart version from 9.2.1
to 8.8.23
. This is a breaking change and users will face an issue as mentioned in the documentation here. Please note that OpenMetadata Dependencies Helm Chart is not recommended for production use cases. The steps mentioned in the section will help you fix the issue.
0.13.1 - Stable Release
OpenMetadata 0.13.1 is a stable release. Please check the release notes
Breaking Changes for 0.13.1 Stable Release
OpenMetadata Release 0.13.1 introduces below breaking changes -
Webhooks
Starting from 0.13.1 , OpenMetadata will be deprecating the existing webhooks for Slack, MSTeams.
Before upgrading to 0.13.1 it is recommended to save the existing Webhook configs(like webhook url) to use them later.
We have added Alerts/Notifications , which can be configured to receive customised alerts on updates in OM using Triggers, Filtering Information to different destinations like Slack, MsTeams or even Emails. Please use the same webhook config that you had saved from previous version to configure the Alerts Destination after upgrading.
OpenMetadata Release 0.13.x introduces below breaking changes:
Docker Volumes
OpenMetadata Release 0.13.x introduces Default Docker Volumes for Database (MySQL, PostgreSQL) and ElasticSearch with Docker deployment.
- If you are looking for the fresh deployment of 0.13.x - here
- If you are looking for upgrading of the new version i.e 0.13.x - here
MySQL Helm Chart Version Updated to 9.2.1
OpenMetadata Helm Chart Release with Application Version 0.13.1
updates the Bitnami MySQL Helm Chart version to 9.2.1
from 8.8.23
. This is not a breaking change but existing user's trying to upgrade will experience a slight delay in OpenMetadata Dependencies Helm Chart Upgrades as it pulls new docker image for MySQL. Please note that OpenMetadata Dependencies Helm Chart is not recommended for production use cases. Please follow the kubernetes deployment for new installation or upgrade kubernetes for upgrading OpenMetadata in Kubernetes.
dbt Workflow
dbt ingestion has been separated from the metadata ingestion. It can now be configured as a separate workflow after completing the metadata ingestion workflow.
We will remove the dbt configuration from your existing metadata ingestion pipelines and they will keep working as expected.
After upgrading you will have to create the dbt workflow for the dbt ingestion to start working again.
Airflow Lineage Backend
- The import for the Airflow Lineage Backend has been updated from
airflow_provider_openmetadata.lineage.openmetadata.OpenMetadataLineageBackend
toairflow_provider_openmetadata.lineage.backend.OpenMetadataLineageBackend
. - We removed support from Airflow v1.
- The failure callback now only updates the pipeline status if the Pipeline already exists in OpenMetadata.
0.13.0 - Feature Release
OpenMetadata 0.13.0 is a feature release.
Don't upgrade your production with 0.13.0 feature release
Explore 0.13.0 by following up Deployment guides and please give us any feedback on our community slack
Backup Metadata
Before upgrading your OpenMetadata version we recommend backing up the metadata.
The source of truth is stored in the underlying database (MySQL and Postgres supported). You can refer to the following guide for our backup utility:
Backup Metadata Learn how to back up MySQL data.
Upgrade your installation
Once your metadata is safe, follow the required upgrade instructions:
Upgrade a Kubernetes Deployment Upgrade your Kubernetes installation
Upgrade a Docker Deployment Upgrade your Docker installation
Upgrade a Bare Metal Deployment Upgrade your Bare Metal installation