pimlico.datatypes.parse package

Submodules

• pimlico.datatypes.parse.candc module
• pimlico.datatypes.parse.dependency module

Module contents

TODO Parse tress are temporary implementations that don’t actually parse the data, but just split it into

sentences.

class TreeStringsDocumentType(options, metadata)

Bases: pimlico.datatypes.documents.RawDocumentType

process_document(doc)

class ConstituencyParseTreeCorpus(base_dir, pipeline, **kwargs)

Bases: pimlico.datatypes.tar.TarredCorpus

Note that this is not fully developed yet. At the moment, you’ll just get, for each document, a list of the texts of each tree. In future, they will be better represented.

datatype_name = 'parse_trees'

data_point_type

alias of TreeStringsDocumentType

class ConstituencyParseTreeCorpusWriter(base_dir, gzip=False, append=False, trust_length=False, encoding='utf-8', **kwargs)

Bases: pimlico.datatypes.tar.TarredCorpusWriter

document_to_raw_data(data)

class CandcOutputCorpus(base_dir, pipeline, **kwargs)

Bases: pimlico.datatypes.tar.TarredCorpus

datatype_name = 'candc_output'

data_point_type

alias of CandcOutputDocumentType

class CandcOutputCorpusWriter(base_dir, gzip=False, append=False, trust_length=False, encoding='utf-8', **kwargs)

Bases: pimlico.datatypes.tar.TarredCorpusWriter

document_to_raw_data(data)

class StanfordDependencyParseCorpus(base_dir, pipeline, **kwargs)

Bases: pimlico.datatypes.jsondoc.JsonDocumentCorpus

datatype_name = 'stanford_dependency_parses'

data_point_type

alias of StanfordDependencyParseDocumentType

class StanfordDependencyParseCorpusWriter(base_dir, readable=False, **kwargs)

Bases: pimlico.datatypes.jsondoc.JsonDocumentCorpusWriter

document_to_raw_data(data)

class CoNLLDependencyParseCorpus(base_dir, pipeline, **kwargs)

Bases: pimlico.datatypes.word_annotations.WordAnnotationCorpus

10-field CoNLL dependency parse format (conllx) – i.e. post parsing.

Fields are:

id (int), word form, lemma, coarse POS, POS, features, head (int), dep relation, phead (int), pdeprel

The last two are usually not used.

datatype_name = 'conll_dependency_parses'

data_point_type

alias of CoNLLDependencyParseDocumentType

class CoNLLDependencyParseCorpusWriter(base_dir, **kwargs)

Bases: pimlico.datatypes.word_annotations.WordAnnotationCorpusWriter

document_to_raw_data(data)

class CoNLLDependencyParseInputCorpus(base_dir, pipeline, **kwargs)

Bases: pimlico.datatypes.word_annotations.WordAnnotationCorpus

The version of the CoNLL format (conllx) that only has the first 6 columns, i.e. no dependency parse yet annotated.

datatype_name = 'conll_dependency_parse_inputs'

data_point_type

alias of CoNLLDependencyParseInputDocumentType

class CoNLLDependencyParseInputCorpusWriter(base_dir, **kwargs)

Bases: pimlico.datatypes.word_annotations.WordAnnotationCorpusWriter

document_to_raw_data(data)