Kubernetes Deployment
Instruction on how to install Databand in a Kubernetes/OpenShift Cluster.
Databand
Using this guide you will be able to deploy databand on the Kubernetes/OpenShift clusters.
Cluster | Version |
---|---|
Kubernetes | 1.16-1.23 |
OpenShift | 4.8-4.11 |
Installation Prerequisites
Before you can run scripts described in this topic, you need to have the following software installed:
Clusters configuration prerequisites:
- Postgres 12 - it's highly recommended to use external postgres DB instance version 12, hosted outside of Kubernetes/Openshift cluster(by default local postgres instance is used)
- If external DB will be used -
databand
DB should be created in Postgres 12 instance before the installation - Redis 6 - it's highly recommended to use external redis in-memory store version 6, hosted outside of Kubernetes/Openshift cluster(by default local redis instance is used)
- Kubernetes/OpenShift node - 2 nodes, 4CPU 16RAM recources available
- Access to Kubernetes/OpenShift API
PATCH
method to update Alertmanager configmap - Pods should able to mount PVC
- RBAC support: able to create serviceaccount/role/rolebinding
- Download databand--helm-chart.tar.gz from IBM PPA. Extract files from the archive. Upload docker images from databand--images.tar archive to your private registry using docker utility(docker load -i - to load to local, docker tag and docker push to upload to registry). Extract file from databand-.tgz - Databand helm chart, use it for installation.
- Ability to use publicly available images: prometheus, alertmanager, redis, postgres, flower(celery), busybox. Below is a list of publicly available images:
busybox:1.28.4
jimmidyson/configmap-reload:v0.5.0
mher/flower:0.9.7
quay.io/prometheus/alertmanager:v0.24.0
quay.io/prometheus/prometheus:v2.39.1
bitnami/postgresql:12.11.0(if you don't have external DB or want to use bundled PostgreSQL DB)
bitnami/redis:6.2.6-debian-10-r103(if you don't have external in-memory store or want to use bundled Redis in-memory store)
To install Databand in a remote Kubernetes/OpenShift cluster, perform the tasks described in this topic.
Create user-values.yaml
Copy file user-values.yaml.example
to user-values.yaml
. Use user-values.yaml as main file to override default values from values.yaml(don't edit values.yaml/values-ocp.yaml directly), example:
# user-values.yaml
global:
databand:
image:
repository: <YOUR_REPOSITORY_FOR_DATABAND_IMAGES>
tag: <YOUR_TAG_FOR_DATABAND_IMAGES>
imageCredentials:
registry: <YOUR_REGISTRY_FOR_DATABAND_IMAGES>
username: <YOUR_USERNAME>
password: <YOUR_PASSWORD>
If you don't have an access to publicly available prometheus, alertmanager and busybox images, set your own image properties in user-values.yaml
as well:
# user-values.yaml
databand:
initContainers:
wait_web:
image:
repository: <YOUR_REPOSITORY_FOR_BUSYBOX_IMAGE>
tag: <YOUR_TAG_FOR_BUSYBOX_IMAGE>
prometheus:
server:
image:
repository: <YOUR_REPOSITORY_FOR_PROMETHEUS_IMAGE>
tag: <YOUR_TAG_FOR_PROMETHEUS_IMAGE>
alertmanager:
image:
repository: <YOUR_REPOSITORY_FOR_ALERTMANAGER_IMAGE>
tag: <YOUR_REPOSITORY_FOR_ALERTMANAGER_IMAGE>
configmapReload:
image:
repository: <YOUR_REPOSITORY_FOR_CONFIGMAPRELOAD_IMAGE>
tag: <YOUR_REPOSITORY_FOR_CONFIGMAPRELOAD_IMAGE>
Databand secrets
You need to generate 2 secrets and provide it via user-values.yaml
.
First secret - fernet key, use following command to generate it:
dd if=/dev/urandom bs=32 count=1 2>/dev/null | openssl base64
Then, override default fernet key in user-values.yaml
:
# user-values.yaml
databand:
fernetKey: "<GENERATED_FERNET_KEY_FROM_COMMAND_ABOVE>"
Second key - webserver secret, use following command to generate it:
head -c 32 /dev/urandom | base64 | tr -d =
Then, override default webserver secret in user-values.yaml
:
# user-values.yaml
web:
secret_key: "<GENERATED_WEBSERVER_SECRET_FROM_COMMAND_ABOVE>"
By default, we create following username/password for admin user: databand/databand
. You can override it with the following values:
# user-values.yaml
web:
default_user:
disabled: false
role: "Admin"
username: "databand"
email: "[email protected]"
firstname: "databand"
lastname: "databand"
password: "databand"
Install/Update Helm Chart
To install/update the Databand Chart into your Kubernetes/OpenShift cluster, run the following command:
helm upgrade databand --install --create-namespace --namespace databand-system --values ./user-values.yaml .
After the installation/update completes successfully, run the following command to get the status of the Chart:
helm status databand --namespace databand-system
HA mode
By default, Databand install in single mode configuration, to enable HA mode for Databand, override following values in user-values.yaml:
# user-values.yaml
databand:
ha:
enabled: true
replicaCount: 2
Databand Monitors
By default, additional components like datastage-monitor
(for tracking IBM DataStage), dbt-monitor
(for tracking dbt) and datasource-monitor
(for tracking databases like Google BigQuery) are disabled. Monitors require username/password or databand access token to connect to tracking system. If you don't provide username/password, default databand/databand will be using. To enable them - override following values in user-values.yaml:
# user-values.yaml
dbnd-datastage-monitor:
enabled: true
databand_access_token: ""
dbnd_username: "databand"
dbnd_password: "databand"
dbnd-dbt-monitor:
enabled: true
databand_access_token: ""
dbnd_username: "databand"
dbnd_password: "databand"
dbnd-datasource-monitor:
enabled: true
databand_access_token: ""
dbnd_username: "databand"
dbnd_password: "databand"
Common Ingress configuration
The chart provides an Ingress configuration to allow customizing the installation depending on your setup. Review the
comments in the values.yaml
file for more details on how to configure your reverse proxy or load balancer. Ingress Controller must be provisioned in your cluster.
# user-values.yaml
ingress:
enabled: true
web:
host: <DATABAND_EXTERNAL_URL>
## To enable TLS
tls:
## Set to "true" to enable TLS termination at the ingress controller level
enabled: false
## If enabled, set "secretName" to the secret containing the TLS private key and certificate
## Example:
## secretName: example-com-crt
GKE Ingress configuration
The chart provides a support for native GKE Ingress configuration with BackendConfig object.
# user-values.yaml
# GKE Ingress requires NodePort Service type
databand:
service:
type: NodePort
ingress:
enabled: true
backendconfig:
enabled: true
web:
host: <DATABAND_EXTERNAL_URL>
annotations:
## Set a GKE Ingress annotations
## External Load balancer
## To provision internal Load Balancer, set the value of annotation to "gce-internal"
kubernetes.io/ingress.class: "gce"
## Set to false to disable http and use Load Balancer with https only
kubernetes.io/ingress.allow-http: "true"
## To enable TLS
tls:
## Set to "true" to enable TLS termination at the ingress controller level
enabled: false
## If enabled, set "secretName" to the secret containing the TLS private key and certificate
## Example:
## secretName: example-com-crt
Chart Notes
- This chart automatically prefixes all names using the release name to avoid collisions.
- This chart exposes a single endpoint for the Databand Web UI which can be placed either at the root of the domain or
at the subpath, for example http://mycompany.com/databand/ . - In this chart, be default the local PostgreSQL is used as Databand Database.
Database Configuration
External DB
Update your user-values.yaml
with
# user-values.yaml
postgresql:
enabled: false
sql_alchemy_conn:
protocol: "postgresql+psycopg2"
username: "databand"
password: "databand"
host: "databand.example.com"
port: "5432"
dbname: "databand"
If you already have an existing secret with Database password in namespace, you can read from it:
# user-values.yaml
postgresql:
enabled: false
sql_alchemy_conn:
protocol: "postgresql+psycopg2"
username: "databand"
password: ""
host: "databand.example.com"
port: "5432"
dbname: "databand"
existingSecret:
name: "somesecretname"
secretKeys:
userPasswordKey: "password"
External In-Memory Store
Instead of local bitnami Redis, you can use external in-memory store:
# user-values.yaml
redis:
enabled: false
inmemorystore:
host: "10.1.3.4"
port: "6379"
password: "verysecretpassword"
If you want to read in-memory store password from secret, specify following values:
# user-values.yaml
redis:
enabled: false
inmemorystore:
host: "10.1.3.4"
port: "6379"
password: ""
existingSecret:
name: "somesecretname"
secretKeys:
passwordKey: "password"
For Postgres DB and Redis built in into Databand Chart
Create secure Database connection credentials.
kubectl create secret generic databand-postgres --namespace databand-system --from-literal=postgres-password=$(openssl rand -base64 13)
kubectl create secret generic databand-redis ---namespace databand-system --from-literal=redis-password=$(openssl rand -base64 13)
Update your user-values.yaml
# user-values.yaml
postgresql:
existingSecret: databand-postgres
redis:
existingSecret: databand-redis
If you don't have an access to publicly available postgres, redis and busybox images, set your own image properties in user-values.yaml
as well:
# user-values.yaml
databand:
initdb:
job:
initContainers:
wait_postgres:
busybox:
image:
repository: <YOUR_REPOSITORY_FOR_BUSYBOX_IMAGE>
tag: <YOUR_TAG_FOR_BUSYBOX_IMAGE>
postgresql:
existingSecret: databand-postgres
image:
registry: <YOUR_REGISTRY_FOR_POSTGRES_IMAGE> # e.g. docker.io
repository: <YOUR_REPOSITORY_FOR_POSTGRES_IMAGE> # e.g. bitnami/postgresql
tag: <YOUR_TAG_FOR_POSTGRES_IMAGE> # e.g. 12.11.0
redis:
existingSecret: databand-redis
image:
registry: <YOUR_REGISTRY_FOR_REDIS_IMAGE> # e.g. docker.io
repository: <YOUR_REPOSITORY_FOR_REDIS_IMAGE> # e.g. bitnami/redis
tag: <YOUR_TAG_FOR_REDIS_IMAGE> # e.g. 7.0.5-debian-11-r15
Additional environment variables
You can also specify additional environment variables by using the same format as in the pod's .spec.containers.env
definition. These environment variables are mounted on the web, scheduler, and worker pods. You can use this feature to
pass additional secret environment variables to Databand.
Here is a simple example showing how to pass in a Fernet key. Of course, for this example to work, databand
Kubernetes/OpenShift
secret must already exist in the proper namespace; be sure to create those before running Helm.
# user-values.yaml
extraEnv:
- name: DBND__WEBSERVER__FERNET_KEY
valueFrom:
secretKeyRef:
name: databand
key: fernet-key
Deploy Databand without RBAC permissions
Default Databand installation requires RBAC permissions. In case you don'thave RBAC:
- Alert receiver settings tab won't be shown in Databand UI.
- Alertmanager will use the deployment configmap configurations
Disable alert_def_syncer syncing Alertmanager template file from Databand DB by changing user-values.yaml
# user-values.yaml
databand:
alert:
sync_alertmanager_config: false
Deploy Databand to OpenShift Container Platform (OCP)
OpenShift requires pod securityContext
and container securityContext
to be set according to OpenShift cluster SCCs.
Databand Helm chart provide two methods on how to do that:
-
First one (and default for
values-ocp.yaml
file) is to disable securityContext for all Databand-related workloads.
This will allow OpenShift's cluster SCC admission controller to injectsecurityContext
dynamically on chart deploy,
based on OpenShift's cluster configured SCCs. It also provides great portability between different OpenShift versions. -
Second method - is to set both pod and container desired
securityContext
by the user or cluster administrator,
who deploy the chart via corresponding Helm values -values-ocp.yaml
. Please seevalues-ocp.yaml
for additional details and available parameters.
To install the Databand Chart into your OpenShift cluster, run the following command:
helm upgrade databand --install --create-namespace --namespace databand-system --values ./values-ocp.yaml --values ./user-values.yaml .
After the installation completes successfully, run the following command to get the status of the Chart:
helm status databand --namespace databand-system
Updated 2 months ago