pimlico.core.modules.execute module

Runtime execution of modules

This module provides the functionality to check that Pimlico modules are ready to execute and execute them. It is used by the run command.

check_and_execute_modules(pipeline, module_names, force_rerun=False, debug=False, log=None, all_deps=False, check_only=False, exit_on_error=False, preliminary=False, email=None)

Main method called by the run command that first checks a pipeline, checks all pre-execution requirements of the modules to be executed and then executes each of them. The most common case is to execute just one module, but a sequence may be given.

Parameters:

• exit_on_error – drop out if a ModuleExecutionError occurs in any individual module, instead of continuing to the next module that can be run
• pipeline – loaded PipelineConfig
• module_names – list of names of modules to execute in the order they should be run
• force_rerun – execute modules, even if they’re already marked as complete
• debug – output debugging info
• log – logger, if you have one you want to reuse
• all_deps – also include unexecuted dependencies of the given modules
• check_only – run all checks, but stop before executing. Used for check command

Returns:

check_modules_ready(pipeline, modules, log, preliminary=False)

Check that a module is ready to be executed. Always called before execution begins.

Parameters:

• pipeline – loaded PipelineConfig
• modules – loaded ModuleInfo instances, given in the order they’re going to be executed. For each module, it’s assumed that those before it in the list have already been run when it is run.
• log – logger to output to

Returns:

If preliminary=True, list of problems that were ignored by allowing preliminary run. Otherwise, None – we raise an exception when we first encounter a problem

execute_modules(pipeline, modules, log, force_rerun=False, debug=False, exit_on_error=False, preliminary=False, email=None)

format_execution_dependency_tree(tree)

send_final_report_email(pipeline, error_modules, success_modules, skipped_modules, all_modules)

send_module_report_email(pipeline, module, short_error, long_error)

exception ModuleExecutionError(*args, **kwargs)

Bases: exceptions.Exception

exception ModuleNotReadyError(*args, **kwargs)

Bases: pimlico.core.modules.execute.ModuleExecutionError

exception ModuleAlreadyCompletedError(*args, **kwargs)

Bases: pimlico.core.modules.execute.ModuleExecutionError

exception StopProcessing

Bases: exceptions.Exception