Roadmap

Here is the OpenMetadata Roadmap for the next 3 releases.
We are doing a monthly release and we are going to evolve fast and adapt to community needs. Below roadmap is subject to change based on community needs and feedback. our roadmap yet, please file an Issue Github or ping us on Slack​
If you would like to prioritize any feature or would like to add a new feature that is not in our roadmap yet, please file an Issue on Github or ping us on Slack.
Please check our Releases page to see all the features we shipped so far.
Roadmap
0.10.0 Release - Apr 27th, 2022
Support for Database Schema
OpenMetadata supports databases, service name databases, and tables. We’ve added Database Schema as part of the FQN. For each external data source, we ingest the database, as well as the tables that are contained underneath the schemas.
Support for Hard Delete
OpenMetadata supported soft deletions. Now, we also support the hard deletion of entities through the UI, APIs, and ingestion. Hard deleting an entity removes the entity and all of its relationships. This will also generate a change event.
Deploy Ingestion from UI
OpenMetadata has refactored the service connections to simplify the ingestion jobs from both the ingestion framework as well as the UI. We now use the pydantic models automatically generated from the JSON schemas for the connection definition. The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the various connectors that are supported in OpenMetadata.
Download DBT Manifest Files from Amazon S3 or Google Cloud Storage
Previously, when ingesting the models and lineage from DBT, we passed the path of the DBT manifest and catalog files directly into the workflow. We’ve worked on improving the quality of life of DBT. Now, we can dynamically download these files from Amazon S3 or Google Cloud Storage. This way we can have any other process to connect to the DBT, extract the catalog, and put it into any cloud service. We just need the path name and workflow job details from the metadata extraction to be able to ingest metadata.
JSON Schema based Connection Definition
Each service (database, dashboard, messaging, or pipeline service) has its own configuration specifications, with some unique requirements for some services. Instead of the ad hoc definitions of the source module in Python for each connector, we’ve worked on the full refactoring of the ingestion framework. We now use the pydantic models automatically generated from the JSON schemas for the connection definition.
Airflow Rest APIs
The Airflow REST APIs have been refactored. With our API centric model, we are creating a custom airflow rest API directly on top of Airflow using plugins. This passes the connection information to automatically generate all the dags and prepares handy methods to help us test the connection to the source before creating the service.
UI Changes
The UI improvements are directed toward providing a consistent user experience.
Hard Deletion of Entities: With the support for the hard deletion of entities, we can permanently delete tables, topics, or services. When the entity is hard deleted, the entity and all its relationships are removed. This generates an ‘EntityDeleted’ change event.
Dynamic “Add Service” Forms: The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the various connectors that are supported in OpenMetadata.
UI Support for Database Schema as part of FQN: The database schema has been introduced in the 0.10 release. All the entity pages now support Database Schema in the UI.
Lineage Editor: Improvements have been made to the lineage editor.
Teams: While signing up in OpenMetadata, the teams with restricted access are hidden and only the joinable teams are displayed.
Team Owner: An Owner field has been added to the Team entity. Only team owners can update the teams.
Activity Feeds: The Activity Feeds UI supports infinite scrolling.
Add User: A user can be added from the Users page.
Security Changes
Support Refresh Tokens for Auth0 and Okta SSO
The JWT tokens generated by the SSO providers expire by default in about an hour, making the user re-login often. In this release, we’ve added support for refresh tokens for Auth0 and Okta SSO. The tokens are refreshed silently behind the scenes to provide an uninterrupted user experience. In future releases, we’ll continue to stabilize authentication and add refresh tokens for the other SSO providers.
Custom OIDC SSO
OpenMetadata now supports integration with your custom built OIDC SSO for authentication. This is supported both on the front end for user authentication as well as on the ingestion side.
Azure SSO
Support has been added for Azure SSO on Airflow.
0.11 Release - Jun 9th, 2022
Data Insights
Show users/teams how their data doing
Cost analysis
How to improve Data culture
Data Quality
Data SLAs
Freshness and completeness
Ability to deploy GE tests from OpenMetadata UI
Lineage
Column Level Lineage support
Incident detection through Lineage
Collaboration
Tiering Report on how to improve data assets and data culture
Email notifications
Webbrowser notifications
ML Features
support ML feature UI
Sagemake connector

Overview - Previous

Features

Next - Overview

Releases

Last modified 6d ago

Copy link

Edit on GitHub

Contents

Roadmap

0.10.0 Release - Apr 27th, 2022

0.11 Release - Jun 9th, 2022