- dbnd.log_dataset_op(op_path: Union[targets.base_target.Target, str], op_type: Union[dbnd._core.constants.DbndDatasetOperationType, str], *, success: bool = True, error: Optional[str] = None, data: Optional[Any] = None, with_histograms: Optional[bool] = None, with_partition: Optional[bool] = None, with_stats: Optional[bool] = True, with_preview: bool = False, with_schema: bool = True, send_metrics: bool = True, row_count: Optional[int] = None, column_count: Optional[int] = None)
Logs dataset operation and meta data to dbnd.
op_path – Target object to log or a unique path representing the operation location.
op_type – Type of operation that been done with the target - read, write, delete.
success – True if the operation succeeded, False otherwise.
error – Optional error message.
data – Optional value of data to use build meta-data on the target.
with_histograms – Should calculate histogram of the given data - relevant only with data param. - Boolean to calculate or not on all the data columns.
with_stats – Should extract schema of the data as meta-data of the target - relevant only with data param. - Boolean to calculate or not on all the data columns.
with_partition – If True, the webserver tries to detect partitions of our datasets and extract them from the path, otherwise not manipulating the dataset path at all.
with_preview – Should extract preview of the data as meta-data of the target - relevant only with data param.
with_schema – Should extract schema of the data as meta-data of the target - relevant only with data param.
send_metrics – Should report preview, schemas and histograms as metrics.
row_count – should report row count no matter what is the data
column_count – should report column count no matter what is the data
@task def prepare_data(): log_dataset_op( "/path/to/value.csv", DbndDatasetOperationType.read, data=pandas_data_frame, with_preview=True, with_schema=True, )