Skip to main content

​
module pipeline_mixin

Mixin class containing Pipeline specific methods To be used by OpenMetadata class

​
class OMetaPipelineMixin

OpenMetadata API methods related to the Pipeline Entity To be inherited by OpenMetadata

​
method add_pipeline_status

Given a pipeline and a PipelineStatus, send it to the Pipeline Entity

​
method add_task_to_pipeline

The background logic for this method is that during Airflow backend lineage, we compute one task at a time. Let’s generalise a bit the approach by preparing a method capable of updating a tuple of tasks from the client. Latest changes leave all the task management to the client. Therefore, a Pipeline will only contain the tasks sent in each PUT from the client.

​
method clean_pipeline_tasks

Given a list of tasks, remove from the Pipeline Entity those that are not received as an input. e.g., if a Pipeline has tasks A, B, C, but we only receive A & C, we will remove the task B from the entity