With the help of Databand, you can log the dataset you’re interacting with by informing Databand that you are interacting with some dataset - writing or reading to/from it.
Such interactions will be displayed in the Databand UI as the ‘Affected Datasets’ section within each Run view.
To access it:
- Go to Runs tab
- Click on the DAG that contains the datasets you are logging
- Navigate to the ‘Affected Datasets’ tab
See more about using 'Affected Datasets' in Dataset Logging.
Operations list on the 'Affected Datasets' page also has an additional summary column called 'Insights'. There you can see if an operation is missing or if it had any schema changes.
If you'd like to view the full list of Schema changes, these are hidden in a popup that shows if you click the short schema changes text in the 'Insights' column.
Missing operations can be viewed in the 'Affected Dataset' charts. They are specifically coloured in red:
- red circle for the 'Operations' chart and
- red "x" for the 'History trends' chart
You can view the list of operations in a Run in 'Affected Datasets'.
Updated about 1 year ago