Assigning pipelines to projects allow:
- Creating logical grouping of pipelines/runs for a customized user experience
- Separating different use cases
- Filtering pipelines according to project names
Project assignment is controlled by an SDK. All dbnd SDKs are designed to support projects’ pipelines.
- In DBND orchestrations, a project is a part of the information belonging to a pipeline run
- Airflow SDK supports project assignment by an environment variable. Airflow tags are also supported.
- Azkaban-native projects are supported, too.
After upgrading to v0.35.1 (or later), all the pipelines (and the runs that belong to those pipelines) are placed into a ‘default’ project. This makes projects backward compatible with SDKs that do not define projects.
You cannot rename or delete the ‘default’ project.
To create and name a new project, you can do the following (in order of priority):
- A project name can be set via property
tracking.project(see SDK Configuration):
Via CLI in orchestration mode
dbnd run dbnd_sanity_check --project=ProjectAlfa
Projects can be filtered using a drop-down list in Databand UI in Pipelines, Runs, Alerts, Schedule, and Data Targets. Currently, the projects in the drop-down list are sorted in the order of their creation.
Click ‘Clear filter’ to return to the default state and see all the available pipelines again.
If developer A and developer B run the same pipeline in different projects, that pipeline will ‘jump’ from one active project to another active project. This is the current behavior. In the future, it will be possible to assign a pipeline to several projects at the same time.
Updated 11 days ago