pimlico.test.pipeline module

Pipeline tests

Pimlico modules and datatypes cannot always be easily tested with unit tests and where they can it’s often not easy to work out how to write the tests in a neatly packaged way. Instead, modules can package up tests in the form of a small pipeline that comes with a tiny dataset to use as input. The pipeline can be run in a test environment, where software dependencies are installed and local config is prepared to store output and so on.

This way of providing tests also has the advantage that modules at the same time provide a demo (or several) of how to use them – how pipeline config should look and what sort of input data to use.

class TestPipeline(pipeline, run_modules, log)

Bases: object

static load_pipeline(path, storage_root)

Load a test pipeline from a config file.

Path may be absolute, or given relative to Pimlico test data directory (PIMLICO_ROOT/test/data)

get_uninstalled_dependencies()

test_all_modules()

test_input_module(module_name)

test_module_execution(module_name)

run_test_pipeline(path, module_names, log, no_clean=False)

Run a test pipeline, loading the pipeline config from a given path (which may be relative to the Pimlico test data directory) and running each of the named modules, including any of those modules’ dependencies.

Any software dependencies not already available that can be installed automatically will be installed in the current environment. If there are unsatisfied dependencies that can’t be automatically installed, an error will be raised.

If any of the modules name explicitly is an input dataset, it is loaded and data_ready() is checked. If it is an IterableCorpus, it is tested simply by iterating over the full corpus.

run_test_suite(pipelines_and_modules, log, no_clean=False)

Parameters:pipeline_and_modules – list of (pipeline, modules) pairs, where pipeline is a path to a config file and modules a list of module names to test

clear_storage_dir()

exception TestPipelineRunError

Bases: exceptions.Exception