Skip to main content
This section covers how to use the NATS JetStream connector. Configure and run the NATS JetStream metadata workflow externally:

​
How to Run the Connector Externally

To run the Ingestion via the UI you’ll need to use the OpenMetadata Ingestion Container, which comes shipped with custom Airflow plugins to handle the workflow deployment. If, instead, you want to manage your workflows externally on your preferred orchestrator, you can check the following docs to run the Ingestion Framework anywhere.

External Schedulers

Get more information about running the Ingestion Framework Externally

​
Requirements

The connector communicates with the NATS JetStream API, so JetStream must be enabled on your NATS server. The credentials you supply must be allowed to publish to and receive replies from the $JS.API.> subjects. Schema ingestion additionally requires access to the key-value (KV) stream subjects that back the configured Schema KV bucket.

​
Python Requirements

Use a Python version supported by the openmetadata-ingestion package that matches your OpenMetadata server. To find the supported range for your release, check the Requires-Python metadata of that release’s ingestion package. To run the NATS JetStream ingestion, install:

​
Metadata Ingestion

All connectors are defined as JSON Schemas. See the NATS connection JSON Schema for the structure of a NATS connection. To create and run a metadata ingestion workflow, follow these steps to build a YAML configuration that connects to the source, processes the entities if needed, and reaches the OpenMetadata server. The workflow is modeled around the following JSON Schema

​
1. Define the YAML Config

This is a sample config for NATS:

​
2. Run with the CLI

First, we will need to save the YAML file. Afterward, and with all requirements installed, we can run:
Note that from connector to connector, this recipe will always be the same. By updating the YAML configuration, you will be able to extract metadata from different sources.