Extend json-schema model parser

Summary

For https://gitlab.indiscale.com/caosdb/customers/3dmmto/management/-/issues/9.

This implements

• pattern properties
• type null properties
• json schema references
• and json schemas without top-level recordtype specification

for the json schema datamodel parser.

Focus

Mainly additions to the JsonSchemaParser class. We also defined a treatment for type: array properties that lack a strict items specification (i.e., could be lists of elements with mixed types, so a manual specification has to be provided -- or they are ignored altogether).

Test Environment

New unit tests. You can also test it with the chemotion json schema in nextcloud/Customers/3dmmto/example_files/testdaten_chemotion/schema.json.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosadvancedtools/models/parser.py

@@ -814,7 +814,7 @@ class JsonSchemaParser(Parser):
 
         if "patternProperties" in elt:
 
-            pattern_property_rts = self_treat_pattern_properties(
+            pattern_property_rts = self._treat_pattern_properties(
                 elt["patternProperties"], name_prefix=name)
             for ppr in pattern_property_rts:
                 # add reference to pattern property type. These can never be
@@ -875,12 +875,53 @@ class JsonSchemaParser(Parser):
         return pp
 
     def _treat_pattern_properties(self, pattern_elements, name_prefix=""):
-        """Special Treatment for pattern properties: A RecordType is created for each pattern property."""
+        """Special Treatment for pattern properties: A RecordType is created for
+        each pattern property. In case of a `type: object` PatternProperty, the
+        remaining properties of the JSON entry are appended to the new
+        RecordType; in case of an atomic type PatternProperty, a single value
+        Property is added to the RecordType.
+
+        Raises
+        ------
+        NotImplementedError
+            In case of patternProperties with non-object, non-atomic type, e.g.,
+            array.
+
+        """
         num_patterns = len(pattern_elements)
         pattern_prop = self._get_pattern_prop()
+        returns = []
         for ii, (key, element) in enumerate(pattern_elements.items()):
             name_suffix = f"_{ii+1}" if num_patterns > 1 else ""
-            name = name_prefix + "Element" + name_suffix
+            name = name_prefix + "Entry" + name_suffix
+            if element["type"] == "object":
+                # simple, is already an object, so can be treated like any other
+                # record type.
+                pattern_type = self._treat_record_type(element, name)
+            elif element["type"] in JSON_SCHEMA_ATOMIC_TYPES:
+                # create a property that stores the actual value of the pattern
+                # property.
+                propname = f"{name}_value"
+                prop = db.Property(name=propname, datatype=self._get_atomic_datatype(element))
+                self.model[propname] = prop
+                pattern_type = db.RecordType(name=name)
+                pattern_type.add_property(prop)
+            else:
+                raise NotImplementedError(
+                    "Pattern properties are currently only supported for types " +
+                    ", ".join(JSON_SCHEMA_ATOMIC_TYPES) + ", and object.")
+
+            # Add pattern property and description
+            pattern_type.add_property(pattern_prop, importance=db.OBLIGATORY)
+            if pattern_type.description:
+                pattern_type.description += f"\n\npattern: {key}"
+            else:
+                pattern_type.description = f"pattern: {key}"
+
+            self.model[name] = pattern_type
+            returns.append(pattern_type)
+
+        return returns
 
 
 if __name__ == "__main__":