Treat dependencies among identifiables

Summary

Allow a CFood to have identifiables that depend on each other. E.g. an identifiable Project and an Experiment that references it.

Focus

Identifiables are now treated one after the other when checking whether they need to be inserted.

Test Environment

Automated tests should suffice. Note that tests where changed slightly such that they fail without this fix: https://gitlab.indiscale.com/caosdb/src/caosdb-advanced-user-tools/-/jobs/37342 Since identifiables are not treated correctly the experiment is inserted a second time which leads to the violation of the unique query.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related Issues
• Up-to-date CHANGELOG.md
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there spezifications? Are they satisfied?
• remove the demonstration branch f-make-deps-fail

For further good practices have a look at our review guidelines.

src/caosadvancedtools/crawler.py

@@ -587,33 +587,31 @@ carefully and if the changes are ok, click on the following link:
             else:
                 logger.debug("Id is known of: {}".format(ent))
 
-        # insert missing, i.e. those which are not valid
-        missing_identifiables = db.Container()
-        missing_identifiables.extend([ent for ent in identifiables
-                                      if ent.id is None or ent.id < 0])
-        # TODO the following should not be necessary. Fix it
-
-        for ent in missing_identifiables:
-            ent.id = None
+            # insert missing, i.e. those which are not valid
+            if ent.id is None or ent.id < 0:
+                missing = ent
+                ent.id = None
+            else:
+                missing = None
 
-        if len(missing_identifiables) > 0:
-            info = "Going to insert the following entities:\n"
+            if missing:
+                #info = "Going to insert the following entities:\n"
 
-            for ent in missing_identifiables:
-                info += str(ent)+"\n"
-            logger.debug(info)
+                # for ent in missing_identifiables:
+                #    info += str(ent)+"\n"
+                # logger.debug(info)
 
-        if len(missing_identifiables) == 0:
-            logger.debug("No new entities to be inserted.")
-        else:
-            try:
-                logger.info(
-                    "Inserting {} Records...".format(
-                        len(missing_identifiables)))
-                guard.safe_insert(missing_identifiables, unique=False,
-                                  flags={"force-missing-obligatory": "ignore"})
-            except Exception as e:
-                DataModelProblems.evaluate_exception(e)
+                # if len(missing_identifiables) == 0:
+                #    logger.debug("No new entities to be inserted.")
+                # else:
+                #        logger.info(
+                #            "Inserting {} Records...".format(
+                #                len(missing_identifiables)))
+                try:
+                    guard.safe_insert([missing], unique=False,
+                                      flags={"force-missing-obligatory": "ignore"})
+                except Exception as e:
+                    DataModelProblems.evaluate_exception(e)
 
         logger.debug("Retrieving entities from CaosDB...")
         identifiables.retrieve(unique=True, raise_exception_on_error=False)