OpenMetadata
Search…
Pipeline
This schema defines the Pipeline entity. A pipeline enables the flow of data from source to destination through a series of processing steps. ETL is a type of pipeline where the series of steps Extract, Transform and Load the data.
Type: object

Properties

Type definitions in this schema

task

  • Type: object
  • Properties
    • name required
      • Name that identifies this task instance uniquely.
      • Type: string
      • Length: between 1 and 64
    • displayName
      • Display Name that identifies this Task. It could be title or label from the pipeline services.
      • Type: string
    • fullyQualifiedName
      • A unique name that identifies a pipeline in the format 'ServiceName.PipelineName.TaskName'.
      • Type: string
      • Length: between 1 and 64
    • description
      • Description of this Task.
      • Type: string
    • taskUrl
      • Task URL to visit/manage. This URL points to respective pipeline service UI.
      • Type: string
      • String format must be a "uri"
    • downstreamTasks
      • All the tasks that are downstream of this task.
      • Type: array
        • Items
        • Type: string
        • Length: between 1 and 64
    • taskType
      • Type of the Task. Usually refers to the class it implements.
      • Type: string
    • taskSQL
    • tags
This document was updated on: Monday, November 15, 2021
Last modified 10d ago