This table lists pipeline and dataset metadata collected by Databand and explains how users can control what is collected.
Tracking metadata can be configured using, SDK Configuration, dbnd_config Airflow Connection Airflow Connection, or Databand UI.
|Metadata||Default||DBND Config||Airflow Syncer via UI|
|Source Code||disabled||Select/Unselect 'Include source code' option at Settings->Airflow Syncers -> Add/Edit Syncer Page|
|Logs||disabled||Select/Unselect collect logs option at Settings->Airflow Syncers -> Add/Edit Syncer Page, provide number of KB from head and tail if logs were enabled. (Maximum 8096 KB in each)|
|Errors||enabled||ask the Databand team to switch on/off||ask the Databand team to switch on/off|
|Airflow XCOM values||disabled||not supported in UI, via |
|return value of Airflow Python Task||disabled||not supported in UI, can be done via |
|Data Operations||explicit reporting by a user using |
You can easily enable Code tracking and Logs tracking by Databand Service by providing the following config
[tracking] track_source_code=True [log] preview_head_bytes=15360 preview_tail_bytes=15360
For Airflow Tracker you can just edit the Airflow Tracking Configuration.
Updated 8 months ago