Extend json-schema model parser

Summary

For https://gitlab.indiscale.com/caosdb/customers/3dmmto/management/-/issues/9.

This implements

• pattern properties
• type null properties
• json schema references
• and json schemas without top-level recordtype specification

for the json schema datamodel parser.

Focus

Mainly additions to the JsonSchemaParser class. We also defined a treatment for type: array properties that lack a strict items specification (i.e., could be lists of elements with mixed types, so a manual specification has to be provided -- or they are ignored altogether).

Test Environment

New unit tests. You can also test it with the chemotion json schema in nextcloud/Customers/3dmmto/example_files/testdaten_chemotion/schema.json.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosadvancedtools/models/parser.py

@@ -77,7 +77,8 @@ JSON_SCHEMA_ATOMIC_TYPES = [
     "string",
     "boolean",
     "integer",
-    "number"
+    "number",
+    "null"
 ]
 
 
@@ -726,6 +727,10 @@ class JsonSchemaParser(Parser):
             return db.DOUBLE
         elif elt["type"] == "boolean":
             return db.BOOLEAN
+        elif elt["type"] == "null":
+            # This could be any datatype since a valid json will never have a
+            # value in a null property. We use TEXT for convenience.
+            return db.TEXT
         else:
             raise JsonSchemaDefinitionError(f"Unkown atomic type in {elt}.")
 
@@ -761,6 +766,9 @@ class JsonSchemaParser(Parser):
             ent = self._treat_record_type(elt, name)
         elif elt["type"] == "array":
             ent, force_list = self._treat_list(elt, name)
+        elif elt["type"] == "null":
+            # null
+            return None, force_list
         else:
             raise NotImplementedError(
                 f"Cannot parse items of type '{elt['type']}' (yet).")