Alert Configuration for Self-Hosting

This is a continuation of the Alerting page, relevant for self-hosted Databand.

You can use our UI configuration to define standard receivers. See more information at Alerting .
For Email Notification

Email Notification via Receiver configured at Databand UI

You need to set up SMTP default params. You can provide default SMTP parameters to use for sending out email alerts. In order to do that please set the following environment variables for the dbnd-web service at your custom.env file:

# custom.env
[email protected]
DBND__ALERT__SMTP_HOSTNAME=sendgrid.com:12345
DBND__ALERT__SMTP_APIKEY_OR_USERNAME=sample-api-key
DBND__ALERT__SMTP_TOKEN_OR_PASSWORD=sample-api-token 

Advanced Alerts Routing Configuration

Databand uses Prometheus Alertsmanager to dispatch alerts. If you are looking for advanced routes for your alerts You can have the manual configuration. In the configuration of the Alertsmanager web service, specify where the alert needs to be sent (Databand application, Slack, email, etc.) and the users who will receive them.

Here is an example of a configuration specified in the Alertsmanager web service:

route:
  receiver: 'databand'
  routes:
    - receiver: 'databand'
      group_wait: 1s
      group_interval: 5s
      repeat_interval: 30s
      match:
        is_dbnd_internal: 'true'
      continue: false  # skip next receivers
    # - receiver: 'email'
    #   group_wait: 30s
    #   group_interval: 5m
    #   repeat_interval: 1h
    #   continue: true
receivers:
  - name: 'databand'
    webhook_configs:
      - url: 'http://webserver:8080/api/v1/webhooks/alertmanager'
        send_resolved: false
  - name: 'slack'
    slack_configs:
      - api_url: '<API URL>'
        username: 'DBND Alertmanager'
        channel: '#dbnd-alerts-testing'
        icon_url: https://avatars3.githubusercontent.com/u/3380462
        send_resolved: false
        title: '{{ template "dbnd_title" . }}'
        title_link: '{{ template "dbnd_titlelink" . }}'
        text: '{{ template "dbnd_slack_message" . }}'
  - name: 'email'
    email_configs:
      - to: '[email protected]'  # set receiver email
        from: '[email protected]'  # set verified Sender Identity
        smarthost: 'smtp.sendgrid.net:587'
        auth_username: 'apikey'
        auth_password: '<< API Token >>'
        send_resolved: false
        html: '{{ template "dbnd_email_message" . }}'
        headers:
          subject: '{{ template "dbnd_email_subject" . }}'
templates:
  - /etc/alertmanager/notifications.tmpl
  - /etc/alertmanager/email.tmpl

Labels and Annotations:

As you can see from the configuration example, in Alerts, you can also specify the conditions for sending alerts to the next alerts cycle. See Alert Manager online documentation for information about how to customize it.
Following labels and annotations are available in every Alert. These labels and annotations can also be used by custom messages

Labels:

• alertname = Run duration is greater than 30 seconds.
• af_dag_id = DAG_ID
• af_execution_date = 2022-02-17 16:00:00
• af_instance_name = dev-airflow-instance-name
• airflow_tags = tag1, tag2
• env = dev
• instance = dev-airflow-instance
• job_name = PIPELINE_NAME
• project_name = PROJECT_NAME
• run_name = scheduled__2022-02-17T16:00:00+00:00
• run_start_time = 2022-02-17 17:00:07
• run_attempt_number = 1
• run_triggered_by = databand
• triggered_on = 2022-02-17 17:01:28
• affected_datasets_num = 0
• schema_changes_num = 0
• missing_ops_num = 0
• severity = HIGH
• is_dbnd_internal = false

Annotations:

• base_url = airflow-instance.url
• defining_entity = PIPELINE_NAME
• description = ALERT_DECRIPTION