> ## Documentation Index
> Fetch the complete documentation index at: https://docs.open-metadata.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Create a Pipeline

> Create a new pipeline within a pipeline service

# Create a Pipeline

Create a new pipeline within a pipeline service.

## Body Parameters

<ParamField body="name" type="string" required>
  Name of the pipeline. Must be unique within the parent pipeline service.
</ParamField>

<ParamField body="service" type="string" required>
  Fully qualified name of the parent PipelineService (e.g., `sample_airflow`).
</ParamField>

<ParamField body="displayName" type="string">
  Human-readable display name for the pipeline.
</ParamField>

<ParamField body="description" type="string">
  Description of the pipeline in Markdown format.
</ParamField>

<ParamField body="sourceUrl" type="string">
  URL to the pipeline definition in the source system (e.g., Airflow DAG URL).
</ParamField>

<ParamField body="tasks" type="array">
  Array of pipeline tasks representing individual steps in the pipeline.

  <Expandable title="properties">
    <ParamField body="name" type="string">
      Name of the task.
    </ParamField>

    <ParamField body="displayUrl" type="string">
      URL to display for this task.
    </ParamField>

    <ParamField body="taskUrl" type="string">
      URL to the task in the source system.
    </ParamField>

    <ParamField body="taskType" type="string">
      Type of the task (e.g., `BatchTask`, `StreamTask`).
    </ParamField>

    <ParamField body="description" type="string">
      Description of the task.
    </ParamField>

    <ParamField body="downstreamTasks" type="array">
      Array of task names that depend on this task.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="owners" type="array">
  Array of owner references (users or teams) to assign to the pipeline.

  <Expandable title="properties">
    <ParamField body="id" type="string">
      UUID of the owner entity.
    </ParamField>

    <ParamField body="type" type="string">
      Type of owner entity (e.g., `user`, `team`).
    </ParamField>

    <ParamField body="name" type="string">
      Name of the owner entity.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="domain" type="string">
  Fully qualified name of the domain to assign for governance purposes.
</ParamField>

<ParamField body="tags" type="array">
  Array of classification tags to apply to the pipeline.

  <Expandable title="properties">
    <ParamField body="tagFQN" type="string" required>
      Fully qualified name of the tag.
    </ParamField>

    <ParamField body="labelType" type="string">
      Type of label (e.g., `Manual`, `Derived`, `Propagated`).
    </ParamField>

    <ParamField body="state" type="string">
      State of the tag (e.g., `Suggested`, `Confirmed`).
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="extension" type="object">
  Custom property values defined by your organization's metadata schema.
</ParamField>

<RequestExample dropdown>
  ```python POST /v1/pipelines theme={null}
  from metadata.sdk import configure
  from metadata.sdk.entities import Pipelines
  from metadata.generated.schema.api.data.createPipeline import CreatePipelineRequest

  configure(
      host="https://your-company.open-metadata.org/api",
      jwt_token="your-jwt-token"
  )

  request = CreatePipelineRequest(
      name="dbt_analytics_customers",
      displayName="DBT Customer Analytics",
      service="sample_airflow",
      description="Analytics pipeline for customer data processing",
      sourceUrl="http://localhost:8080/tree?dag_id=dbt_analytics_customers",
      tasks=[
          {
              "name": "extract_customers",
              "taskType": "BatchTask",
              "description": "Extract customer data from source",
              "downstreamTasks": ["transform_customers"]
          },
          {
              "name": "transform_customers",
              "taskType": "BatchTask",
              "description": "Transform and clean customer data",
              "downstreamTasks": ["load_customers"]
          },
          {
              "name": "load_customers",
              "taskType": "BatchTask",
              "description": "Load transformed data into warehouse"
          }
      ]
  )

  pipeline = Pipelines.create(request)
  print(f"Created: {pipeline.fullyQualifiedName}")
  ```

  ```java POST /v1/pipelines theme={null}
  import static org.openmetadata.sdk.fluent.Pipelines.*;

  // Create using builder pattern
  var pipeline = Pipelines.builder()
      .name("dbt_analytics_customers")
      .displayName("DBT Customer Analytics")
      .service("sample_airflow")
      .description("Analytics pipeline for customer data processing")
      .create();
  ```

  ```bash POST /v1/pipelines theme={null}
  curl -X POST "{base_url}/api/v1/pipelines" \
    -H "Authorization: Bearer {access_token}" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "dbt_analytics_customers",
      "displayName": "DBT Customer Analytics",
      "service": "sample_airflow",
      "description": "Analytics pipeline for customer data processing",
      "sourceUrl": "http://localhost:8080/tree?dag_id=dbt_analytics_customers",
      "tasks": [
        {
          "name": "extract_customers",
          "taskType": "BatchTask",
          "description": "Extract customer data from source",
          "downstreamTasks": ["transform_customers"]
        },
        {
          "name": "transform_customers",
          "taskType": "BatchTask",
          "description": "Transform and clean customer data",
          "downstreamTasks": ["load_customers"]
        },
        {
          "name": "load_customers",
          "taskType": "BatchTask",
          "description": "Load transformed data into warehouse"
        }
      ]
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "id": "538faa63-d204-46ff-aead-d158d0401cac",
    "name": "dbt_analytics_customers",
    "displayName": "DBT Customer Analytics",
    "fullyQualifiedName": "sample_airflow.dbt_analytics_customers",
    "description": "Analytics pipeline for customer data processing",
    "version": 0.1,
    "updatedAt": 1769982668397,
    "updatedBy": "admin",
    "sourceUrl": "http://localhost:8080/tree?dag_id=dbt_analytics_customers",
    "service": {
      "id": "daa58a49-df05-48a3-a417-45dfd12eacf5",
      "type": "pipelineService",
      "name": "sample_airflow",
      "fullyQualifiedName": "sample_airflow",
      "deleted": false
    },
    "serviceType": "Airflow",
    "tasks": [
      {
        "name": "extract_customers",
        "taskType": "BatchTask",
        "description": "Extract customer data from source",
        "downstreamTasks": ["transform_customers"]
      },
      {
        "name": "transform_customers",
        "taskType": "BatchTask",
        "description": "Transform and clean customer data",
        "downstreamTasks": ["load_customers"]
      },
      {
        "name": "load_customers",
        "taskType": "BatchTask",
        "description": "Load transformed data into warehouse"
      }
    ],
    "href": "http://localhost:8585/api/v1/pipelines/538faa63-d204-46ff-aead-d158d0401cac",
    "deleted": false,
    "owners": [],
    "tags": [],
    "followers": [],
    "votes": {
      "upVotes": 0,
      "downVotes": 0
    },
    "domains": []
  }
  ```
</ResponseExample>

***

## Returns

Returns the created pipeline object with all specified properties and system-generated fields.

## Response

<ResponseField name="id" type="string">
  Unique identifier for the pipeline (UUID format).
</ResponseField>

<ResponseField name="name" type="string">
  Pipeline name.
</ResponseField>

<ResponseField name="fullyQualifiedName" type="string">
  Fully qualified name in format `service.pipelineName`.
</ResponseField>

<ResponseField name="displayName" type="string">
  Human-readable display name.
</ResponseField>

<ResponseField name="description" type="string">
  Description of the pipeline in Markdown format.
</ResponseField>

<ResponseField name="sourceUrl" type="string">
  URL to the pipeline in the source system.
</ResponseField>

<ResponseField name="service" type="object">
  Reference to the parent pipeline service.

  <Expandable title="properties">
    <ResponseField name="id" type="string">
      UUID of the pipeline service.
    </ResponseField>

    <ResponseField name="type" type="string">
      Type of entity (always `pipelineService`).
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the pipeline service.
    </ResponseField>

    <ResponseField name="fullyQualifiedName" type="string">
      Fully qualified name of the pipeline service.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="serviceType" type="string">
  Type of pipeline service (e.g., Airflow, Dagster, DBTCloud).
</ResponseField>

<ResponseField name="tasks" type="array" optional>
  List of tasks in the pipeline.

  <Expandable title="properties">
    <ResponseField name="name" type="string">
      Name of the task.
    </ResponseField>

    <ResponseField name="taskType" type="string">
      Type of the task.
    </ResponseField>

    <ResponseField name="description" type="string">
      Description of the task.
    </ResponseField>

    <ResponseField name="downstreamTasks" type="array">
      Names of downstream dependent tasks.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="owners" type="array" optional>
  List of owners assigned to the pipeline.

  <Expandable title="properties">
    <ResponseField name="id" type="string">
      UUID of the owner entity.
    </ResponseField>

    <ResponseField name="type" type="string">
      Type of owner entity (e.g., `user`, `team`).
    </ResponseField>

    <ResponseField name="name" type="string">
      Name of the owner entity.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="domains" type="array" optional>
  Domain assignments for governance.
</ResponseField>

<ResponseField name="tags" type="array" optional>
  Classification tags applied to the pipeline.

  <Expandable title="properties">
    <ResponseField name="tagFQN" type="string">
      Fully qualified name of the tag.
    </ResponseField>

    <ResponseField name="labelType" type="string">
      Type of label (e.g., `Manual`, `Derived`, `Propagated`).
    </ResponseField>

    <ResponseField name="state" type="string">
      State of the tag (e.g., `Suggested`, `Confirmed`).
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="extension" type="object" optional>
  Custom property values defined by your organization's metadata schema.
</ResponseField>

<ResponseField name="version" type="number">
  Version number for the entity (starts at 0.1).
</ResponseField>

***

## Create or Update (PUT)

Use `PUT /v1/pipelines` instead of `POST` to perform an upsert. If a pipeline with the same `fullyQualifiedName` already exists, it will be updated; otherwise, a new pipeline is created. The request body is the same as `POST`.

```bash theme={null}
curl -X PUT "{base_url}/api/v1/pipelines" \
  -H "Authorization: Bearer {access_token}" \
  -H "Content-Type: application/json" \
  -d '{ ... same body as POST ... }'
```

<Note>
  `PUT` will not return a `409` conflict error if the entity already exists -- it will update the existing entity instead.
</Note>

***

## Bulk Create or Update (PUT)

Use `PUT /v1/pipelines/bulk` to create or update multiple pipelines in a single request. The request body is an array of create request objects.

```bash theme={null}
curl -X PUT "{base_url}/api/v1/pipelines/bulk" \
  -H "Authorization: Bearer {access_token}" \
  -H "Content-Type: application/json" \
  -d '[
    { "name": "pipeline_one", "service": "sample_airflow" },
    { "name": "pipeline_two", "service": "sample_airflow" }
  ]'
```

***

## Error Handling

| Code  | Error Type              | Description                                                   |
| ----- | ----------------------- | ------------------------------------------------------------- |
| `400` | `BAD_REQUEST`           | Invalid request body or missing required fields               |
| `401` | `UNAUTHORIZED`          | Invalid or missing authentication token                       |
| `403` | `FORBIDDEN`             | User lacks permission to create pipelines                     |
| `409` | `ENTITY_ALREADY_EXISTS` | Pipeline with same name already exists in service (POST only) |