F fix merge entity conflicts

Summary

Part of https://gitlab.indiscale.com/caosdb/customers/leibniz-zmt/management/-/issues/125. Adds a new apiutils.empty_diff function to check for the identity of two Entity objects in the sense of no differences between the two entities. It also adds optional arguments to compare_entities and merge_entities to determine whether or not referenced Entity objects will be compared/merged.

Focus

Mainly, new cases to the value comparison of reference properties have been added to the compare_entities function. Note that the compare_referenced_records argument of the compare_entities function defaults False in order to restore the previous behavior. In contrast, the default of merge_references_with_empty_diffs of merge_entities is True because a) the function is Experimental anyway, so breaking previous behavior is ok, and b) in most of the (e.g., crawler) cases this is probably what a merge is expected to do.

Test Environment

Unit tests should be sufficient; but you can also test it with ZMT's Pangaea crawler.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosdb/apiutils.py

@@ -205,8 +205,9 @@ def compare_entities(old_entity: Entity, new_entity: Entity, compare_referenced_
         value that is stored in the respective entity.
 
     If `compare_referenced_records` is `True`, also referenced entities will be
-    compared using this function, but with `compare_referenced_records = False`
-    to prevent infinite recursion in case of circular references.
+    compared using this function (which is then called with
+    `compare_referenced_records = False` to prevent infinite recursion in case
+    of circular references).
 
     Parameters
     ----------
@@ -215,7 +216,9 @@ def compare_entities(old_entity: Entity, new_entity: Entity, compare_referenced_
     compare_referenced_records : bool, optional
         Whether to compare referenced records in case of both, `old_entity` and
         `new_entity`, have the same reference properties and both have a Record
-        object as value.
+        object as value. If set to `False`, only the corresponding Python
+        objects are compared which may lead to unexpected behavior when
+        identical records are stored in different objects. Default is False.
 
     """
     olddiff: Dict[str, Any] = {"properties": {}, "parents": []}
@@ -269,7+272,7 @@
                     new_entity.get_importance(prop.name)):
                 olddiff["properties"][prop.name]["importance"] = \
                     old_entity.get_importance(prop.name)
                 newdiff["properties"][prop.name]["importance"] = \
                     new_entity.get_importance(prop.name)
 
             if (prop.datatype != matching[0].datatype):
@@ -288,7+291,7 @@
                 if compare_referenced_records:
                     # scalar reference
                     if isinstance(prop.value, Entity) and isinstance(matching[0].value, Entity):
-                        # exlicitely not recursive to prevent infinite recursion
+                        # explicitely not recursive to prevent infinite recursion
                         same_value = empty_diff(
                             prop.value, matching[0].value, compare_referenced_records=False)
                     # list of references