F refactor high level api

Summary

This is a new version of the old high level API that was previously contained in apiutils. I completely refactored the structure of the entities, to make use of more up-to-date concepts:

• Usage can be completely offline now (including resolving references), e.g. without a running CaosDB which also allows for comprehensive unit tests.
• Clear separation between resolved and unresolved Entities.
• Transparent yaml serialization and deserialization
• Planned: Transparent support for multi properties.

The main motivation for reactivating and refactoring this module was to facilitate usage of complex entities wihtin simulations and data analysis scripts.

Please also see the included documentation (to be expanded) and lots of unit tests with examples.

Note: This MR includes commits from f-plantuml, f-merge-entities and f-copy-entity, so please review the corresponding MRs first.

Test Environment

There are many unit tests supplied.

Please also test the resolve functions e.g. CaosDBPythonEntity.resolve_references() manually.

Integration tests will be handed in in a later release.

I recommend at least two independent reviewers and this MR is of course still open for discussions and suggestions.

Check List for the Author

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosdb/utils/plantuml.py

@@ -296,6 +296,14 @@ def retrieve_substructure(start_record_types, depth, result_id_set=None, result_
                     name=get_referenced_recordtype(prop.datatype)).retrieve()
                 retrieve_substructure([rt], depth-1, result_id_set,
                                       result_container, False)
+            # TODO: clean up this hack
+            # TODO: make it also work for files
+            if is_reference(prop.datatype) and prop.value is not None:
+                r = db.Record(id=prop.value).retrieve()
+                retrieve_substructure([r], depth-1, result_id_set, result_container, False)
+                if r.id not in result_id_set:
+                    result_container.append(r)
+                    result_id_set.add(r.id)
 
             if prop.id not in result_id_set:
                 result_container.append(prop)