How the tasks are discovered and how to select tasks to run with pinpoint accuracy.
Referring to a specific task using its fully qualified name
The simplest way to find a task is mentioning the task by its fully qualified name, or
FQN. A fully qualified name is made of the following components:
Or, in the CLI perspective:
dbnd run m.m1.m2.function
(there can be any number of packages on the way to a certain function).
Fully qualified names are useful because they allow us to refer to a specific task in a global way, independent from the working directory or environment.
Referring to a specific task using the task registry
Whenever modules that contain decorated tasks (
@pipeline) are loaded, the tasks are registered into the
TaskRegistry. The advantage of referring to tasks via the registry is the ability to call them by their "task family name", which is simply the function's name.
If you look at the the example in the section above, we could omit the
module prefixes, and only use the
function name, as in:
dbnd run function.
There are several ways to load tasks into the task registry ahead of time:
- Referring to modules that require import with the
-mCLI flag. For example:
dbnd run -m module_to_load task_to_run
In this scenario, we will be loading the module named
module_to_loadand running the task named
- Loading modules via DBND's plugin system.
- The module parameter in the
databandconfiguration section includes a list of python modules provided out-of-the-box that are imported automatically.
[databand] module = databand,databand_examples,my_module
Updated over 1 year ago