pimlico.core.modules.multistage module

class MultistageModuleInfo(module_name, pipeline, **kwargs)

Bases: pimlico.core.modules.base.BaseModuleInfo

Base class for multi-stage modules. You almost certainly don’t want to override this yourself, but use the factory method instead. It exists mainly for providing a way of identifying multi-stage modules.

module_executable = True

stages = None

typecheck_inputs()

Overridden to check internal output-input connections as well as the main module’s inputs.

get_software_dependencies()

get_input_software_dependencies()

check_ready_to_run()

get_detailed_status()

reset_execution()

classmethod get_key_info_table()

Add the stages into the key info table.

get_next_stage()

If there are more stages to be executed, returns a pair of the module info and stage definition. Otherwise, returns (None, None)

status

is_locked()

multistage_module(multistage_module_type_name, module_stages, use_stage_option_names=False, module_readable_name=None)

Factory to build a multi-stage module type out of a series of stages, each of which specifies a module type for the stage. The stages should be a list of ModuleStage objects.

class ModuleStage(name, module_info_cls, connections=None, output_connections=None, option_connections=None, use_stage_option_names=False)

Bases: object

A single stage in a multi-stage module.

If no explicit input connections are given, the default input to this module is connected to the default output from the previous.

Connections can be given as a list of ModuleConnection s.

Output connections specify that one of this module’s outputs should be used as an output from the multi-stage module. Optional outputs for the multi-stage module are not currently supported (though could in theory be added later). This should be a list of ModuleOutputConnection s. If none are given for any of the stages, the module will have a single output, which is the default output from the last stage.

Option connections allow you to specify the names that are used for the multistage module’s options that get passed through to this stage’s module options. Simply specify a dict for option_connections where the keys are names module options for this stage and the values are the names that should be used for the multistage module’s options.

You may map multiple options from different stages to the same option name for the multistage module. This will result in the same option value being passed through to both stages. Note that help text, option type, option processing, etc will be taken from the first stage’s option (in case the two options aren’t identical).

Options not explicitly mapped to a name will use the name <stage_name>_<option_name>. If use_stage_option_names=True, this prefix will not be added: the stage’s option names will be used directly as the option name of the multistage module. Note that there is a danger of clashing option names with this behaviour, so only do it if you know the stages have distinct option names (or should share their values where the names overlap).

class ModuleConnection

Bases: object

class InternalModuleConnection(input_name, output_name=None, previous_module=None)

Bases: pimlico.core.modules.multistage.ModuleConnection

Connection between the output of one module in the multi-stage module and the input to another.

May specify the name of the previous module that a connection should be made to. If this is not given, the previous module in the sequence will be assumed.

If output_name=None, connects to the default output of the previous module.

class ModuleInputConnection(stage_input_name=None, main_input_name=None)

Bases: pimlico.core.modules.multistage.ModuleConnection

Connection of a sub-module’s input to an input to the multi-stage module.

If main_input_name is not given, the name for the input to the multistage module will be identical to the stage input name. This might lead to unintended behaviour if multiple inputs end up with the same name, so you can specify a different name if necessary to avoid clashes.

If multiple inputs (e.g. from different stages) are connected to the same main input name, they will take input from the same previous module output. Nothing clever is done to unify the type requirements, however: the first stage’s type requirement is used for the main module’s input.

If stage_input_name is not given, the module’s default input will be connected.

class ModuleOutputConnection(stage_output_name=None, main_output_name=None)

Bases: object

Specifies the connection of a sub-module’s output to the multi-stage module’s output. Works in a similar way to ModuleInputConnection.

exception MultistageModulePreparationError

Bases: exceptions.Exception