> ## Documentation Index
> Fetch the complete documentation index at: https://docs.open-metadata.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Metadata Ingestion - Multithreading (Beta)

> Optimize OpenMetadata Ingestion with multithreading workflows. Learn to configure parallel processing for faster metadata extraction and improved performance.

# Metadata Ingestion - Multithreading (Beta)

The default Metadata Ingestion runs sequentially. This feature allows to run the ingestion concurrently using [Threading](https://docs.python.org/3/library/threading.html).

The user is able to define the amount of threads he would like to use and then the ingestion pipeline is responsible for opening at most that amount. The specific behaviour changes depending on the Service Type used. Please check below on [Feature available for](#feature-available-for) for more information.

## General Considerations

Each case is specific and **more threads does not necessarily translate into a better performance**.

Take into account that with each thread we

* **Increase the load on the Database**, since it opens a new connection that will be used.
* **Increases the Memory used**, since we are holding more context at any given time.

We recommend testing with different values from 1 to 8. **If unsure or having issues, leaving it at 1 is recommended.**

## Feature available for

### Databases

This feature is implemented for all Databases at `schema` level. This means that instead of processing one `schema` at a time we open at most the amount number of configured threads, each with a dedicated Database connection and process them concurrently.

**Example: 4 Threads**

<img src="https://mintcdn.com/openmetadata/SkZlgor4HLQUK_jg/public/images/features/ingestion/workflows/metadata/multithreading/example-diagram.png?fit=max&auto=format&n=SkZlgor4HLQUK_jg&q=85&s=26e986203b666763aa33a7e0b054ea83" alt="Example: 4 Threads" width="1445" height="620" data-path="public/images/features/ingestion/workflows/metadata/multithreading/example-diagram.png" />
