GuidesAPI ReferenceDiscussions
GuidesBlogPlatform

Projects

How projects function in Databand.

Assigning pipelines to projects allow:

  • Creating logical grouping of pipelines/runs for a customized user experience
  • Separating different use cases
  • Filtering pipelines according to project names
  • Routing alerts from every project to the different receivers (no UI yet, ask databand support to configure)

Assigning Pipelines to Projects

Assign Airflow DAG to databand project

Set a tag for your Airflow DAG using the following format:

with DAG(
    dag_id="my_dag
    schedule_interval="0 1 * * *",  # Every day at 1:00
    tags=[
        "project: my_project", # my_dag would be assigned to my_project project in Databand UI
    ],
) as my_dag:

Using SDK

To create and name a new project, you can do the following (in order of priority):

[tracking]
project=my_project
  • Via CLI in orchestration mode:
dbnd run dbnd_sanity_check --project=ProjectAlfa

Project assignment is controlled by an SDK. All DBND SDKs are designed to support projects’ pipelines.

  • In DBND orchestration, a project is a part of the information belonging to a pipeline run.
  • Airflow SDK supports project assignment by an environment variable. Airflow tags are also supported.
  • Azkaban-native projects are supported, too.

The ‘Default’ Project

After upgrading to Databand v0.35.1+, all the pipelines (and the runs that belong to those pipelines) are put into a ‘default’ project. This makes projects backward compatible with SDKs that do not define projects.

You cannot rename or delete the ‘default’ project.

Filtering Projects

Projects can be filtered using a drop-down list in the Databand UI in the Pipelines, Runs, Alerts, and Schedule tabs. Currently, the projects in the drop-down list are sorted in the order of their creation.

Project selection drop-down menu in Databand UIProject selection drop-down menu in Databand UI

Project selection drop-down menu in Databand UI

Project selection filter in Databand UIProject selection filter in Databand UI

Project selection filter in Databand UI

Click ‘Clear filter’ to return to the default state and see all the available pipelines again.

Running Pipelines in Different Projects

One pipeline can be assigned to more than one project on a per-run basis.

For instance, the first run of a pipeline can be assigned to "ProjectA" and the second run of the pipeline can be assigned to "ProjectB", and so on.

These pipelines will be displayed and treated as separate pipelines with the same name, but belonging to different projects.

As a result, all alerts defined for the pipeline/ProjectA can be triggered only when the pipeline is run under "ProjectA".


Did this page help you?