XML Converter

Summary

Base functionality for matching and converting XML trees is implemented.

See: #145 (closed)

This should be reviewed and merged relatively soon. Follow-up issues will be created for the following tasks:

• user and API documentation (#159)
• design and implement more unit tests (#160)
• integration tests (#161)
• issue for creating attribute node converter and completing the text node converter (#158 (closed))

Focus

Have a look at #145 (closed) and check whether the requirements are met.

Test Environment

There are unit tests.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caoscrawler/xml_converter.py

@@ -75,27 +75,26 @@ class XMLTagConverter(Converter):
         if not isinstance(element, XMLTagElement):
             raise TypeError("Element must be an instance of XMLTagElement.")
 
-        # The options text_as_children, attribs_as_children and
-        # tags_as_children are currently not implemented.
-
+        # Get the namespace map from the element:
+        nsmap = element.tag.nsmap
+        # The default name of the default namespace is "default".
+        # You can overwrite it using the attribute "default_namespace" in the converter definition:
+        default_namespace = self.definition.get("default_namespace", "default")
+        if None in nsmap:
+            nsmap[default_namespace] = nsmap[None]
+            del nsmap[None]
+
+        xpath = self.definition.get("xpath", "child::*")
+        children = element.tag.xpath(xpath, namespaces=nsmap)
         el_lst = []
-        if "xpath" in self.definition:
-            # in this case it does not make sense to use the other "children"-options:
-            # for opt in ("text_", "attribs_", "tags_"):
-            #     if opt + "_as_children" in self.definition:
-            #         raise RuntimeError("The option {} cannot be used when using"
-            #                            " xpath".format(opt))
-            # if xpath is present, use the xpath to generate children
-            children = element.xpath(self.definition["xpath"])
-        else:
-            # use the direct children of the node instead
-            children = list(element)
-            # if "text_as_children" in self.definition and self.definition["text_as_children"] == True:
-            #     el_lst.append(XMLTextNode(element, element.tree))
-            # if "attribs_as_children" in self.definition and self.definition["attribs_as_children"] == True:
-            #     # el_lst.append(XMLTextNode(element, element.tree))
         for el in children:
-            el_lst.append(XMLTagElement(el, element.tree))
+            if isinstance(el, str):
+                el_lst.append(XMLTextNode(el))
+            elif isinstance(el, lxml.etree._Element):
+                el_lst.append(XMLTagElement(el))
+            else:
+                raise RuntimeError("Unsupported child type.")
+        return el_lst
 
     def typecheck(self, element: StructureElement):
         """