XML Converter

Summary

Base functionality for matching and converting XML trees is implemented.

See: #145 (closed)

This should be reviewed and merged relatively soon. Follow-up issues will be created for the following tasks:

• user and API documentation (#159)
• design and implement more unit tests (#160)
• integration tests (#161)
• issue for creating attribute node converter and completing the text node converter (#158 (closed))

Focus

Have a look at #145 (closed) and check whether the requirements are met.

Test Environment

There are unit tests.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caoscrawler/xml_converter.py

@@ -97,7 +97,10 @@ class XMLTagConverter(Converter):
             vardict.update(m_tag.groupdict())
 
         if "match_text" in self.definition:
-            m_text = re.match(self.definition["match_text"], element.tag.text)
+            tagtext = element.tag.text
+            if element.tag.text is None:
+                tagtext = ""
+            m_text = re.match(self.definition["match_text"], tagtext, re.DOTALL)
             if m_text is None:
                 return None
             vardict.update(m_text.groupdict())
@@ -114,7 +117,6 @@ class XMLTagConverter(Converter):
                         matched_m_attrib = m_attrib
                         m_attrib_value = re.match(attrib_def_value, attr_value)
                         if m_attrib_value is None:
-                            breakpoint()
                             return None
                         matched_m_attrib_value = m_attrib_value
                 # TODO: How to deal with multiple matches?