Developing a New Connector
Overview
There is an ever-increasing source of metadata and it is possible that OpenMetadata doesn't already have implemented the one you need.
On this guide we will go through the steps needed in order to be able to contribute by developing a new connector yourself by using the already implemented MySQL connector as an example.
Developing a new Connector vs Custom Connectors
Developing a new connector makes sense if it could be used by many other users. If you are dealing with a custom solution used only in your case, it makes more sense to actually create your own Custom Connector by following this guide
Prerequisite
Before starting developing your own connector you need to have the developing environment properly setup and OpenMetadata up and running locally for testing purposes.
Please follow the instructions here
Steps
- Define the JSON Schema
- Develop the Ingestion Code
- Apply UI Changes
- Create the Java ClassConverter
- Test it
- Update the Documentation
References
Previous Webinars
There are some interesting webiners that cover an overview of the ingestion framework as well as a hands-on workshop for creating a new connector.
Note that while specific technicalities might be outdated (patterns for naming files or functions), the overall structure, flow and abstractions described in the videos are still relevant. For any nuances you can look at the current state of the ingestion code to see what is the best practice.