Skip to content
Snippets Groups Projects
Commit 38286fc4 authored by florian's avatar florian
Browse files

STY: autopep8'd

parent 6e7a2eba
Branches
Tags
2 merge requests!108Release 0.5.0,!104Create a new scanner module and move functions from crawl module there
Pipeline #34554 passed with warnings
......@@ -83,6 +83,7 @@ def load_definition(crawler_definition_path: str):
return _resolve_validator_paths(crawler_definition, crawler_definition_path)
def _load_definition_from_yaml_dict(crawler_definitions: list[dict]):
"""Load crawler definitions from a list of (yaml) dicts `crawler_definitions` which
contains either one or two documents.
......@@ -222,6 +223,7 @@ def initialize_converters(crawler_definition: dict, converter_registry: dict):
# Main scanner function:
# --------------------------------------------------------------------------------
def scanner(items: list[StructureElement],
converters: list[Converter],
general_store: Optional[GeneralStore] = None,
......@@ -351,8 +353,6 @@ def scanner(items: list[StructureElement],
return crawled_data, debug_tree
# --------------------------------------------------------------------------------
# Main scanning interface functions:
# --------------------------------------------------------------------------------
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment