module pipeline_mixin

Mixin class containing Pipeline specific methods

To be used by OpenMetadata class

class OMetaPipelineMixin

OpenMetadata API methods related to the Pipeline Entity

To be inherited by OpenMetadata

method add_pipeline_status

Given a pipeline and a PipelineStatus, send it to the Pipeline Entity

method add_task_to_pipeline

The background logic for this method is that during Airflow backend lineage, we compute one task at a time.

Let's generalise a bit the approach by preparing a method capable of updating a tuple of tasks from the client.

Latest changes leave all the task management to the client. Therefore, a Pipeline will only contain the tasks sent in each PUT from the client.

method clean_pipeline_tasks

Given a list of tasks, remove from the Pipeline Entity those that are not received as an input.

e.g., if a Pipeline has tasks A, B, C, but we only receive A & C, we will remove the task B from the entity