F mariadb 48

Summary

Fixing an unreliable heuristic method to distinguish between internal and external ids.

Related #48 (closed)

Corresponding server changes: caosdb-server!107 (merged)

Focus

Two main changes have been introduced here:

1. Prior to this MR, the back end needed to use an unreliable heuristic to determine whether some string is an external or internal id. The back end now prefixes all internal id with $. This marker is transparent to the server but the backend can distinguish internal from external id using this marker instead of the previosly used heuristics.

2. Because the prefix $ revealed another bug which was probably introduced with the back end's 6.0 release or the server's 0.11.0 release, the procedures insertEntityDataType, retrieveEntity, retrieveEntityOverrides, and updateEntity have been refactored to explicitely distinguish between a property's data type id and data type name. The bug was revealed because some the of the $ prefixes have been leaked into the xml response of the server and some of the pyint tests failed because the $ could not be parsed as an id.

Note, while all these changed mean a breaking change of the back end API the server's API does not change and the integration tests should pass without any changes.

Test Environment

It was not possible to write a unit or integration test, because the condition leading to the server error cannot be reproduced deterministically. It would need a new test architecture for loading this particular dump and then testing the server's and the back-end's behavior.

However, please confirm manually, that inserting a new entity like this works with the dump from the original issue #48 (closed). Note: The dump is broken because it was created using the buggy implementation, the entity with id 299 is not retrievable. It is possible to insert a new record as described in the issue, tho.

>>> import linkahead as db
>>> pers = db.Record()
>>> pers.add_property("lastname", "Wood")
<Record>
  <Property name="lastname" importance="FIX" flag="inheritance:FIX">Wood</Property>
</Record>

>>> pers.add_property("firstname", "Tom")
<Record>
  <Property name="lastname" importance="FIX" flag="inheritance:FIX">Wood</Property>
  <Property name="firstname" importance="FIX" flag="inheritance:FIX">Tom</Property>
</Record>

>>> pers.add_parent("Person")
<Record>
  <Parent name="Person"/>
  <Property name="lastname" importance="FIX" flag="inheritance:FIX">Wood</Property>
  <Property name="firstname" importance="FIX" flag="inheritance:FIX">Tom</Property>
</Record>

>>> pers.insert()
<Record id="301" cuid="-1--e717956d-9dfd-4038-ba33-07119f9a6fb0">
  <Version id="2e812417e6029e4def74919331078123886789cb" head="true"/>
  <Warning code="11" description="Entity has no name."/>
  <Parent id="215" name="Person"/>
  <Property id="221" name="lastname" description="LastName of a Person." datatype="TEXT" importance="FIX" flag="inheritance:FIX">Wood</Property>
  <Property id="219" name="firstname" description="First name of a Person." datatype="TEXT" importance="FIX" flag="inheritance:FIX">Tom</Property>
</Record>

>>> db.execute_query("FIND Person")
<Entities>
  <noscript>
    </noscript>
  <TransactionBenchmark>
    </TransactionBenchmark>
  <Record id="301">
    <Version id="2e812417e6029e4def74919331078123886789cb" head="true"/>
    <Parent id="215" name="Person"/>
    <Property id="221" name="lastname" description="LastName of a Person." datatype="TEXT" importance="FIX" flag="inheritance:FIX">Wood</Property>
    <Property id="219" name="firstname" description="First name of a Person." datatype="TEXT" importance="FIX" flag="inheritance:FIX">Tom</Property>
  </Record>
</Entities>

>>> 

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

procedures/insertEntityDataType.sql

@@ -28,16 +28,16 @@ DELIMITER //
  * ----------
  * PropertyID : VARCHAR(255)
  *   The property id.
- * DataType : VARCHAR(255)
- *   The data type, e.g. "DOUBLE", "Person"
+ * DataTypeID : VARCHAR(255)
+ *   The data type id (not the name!)
  */
-CREATE PROCEDURE db_5_0.insertEntityDataType(in PropertyID VARCHAR(255), in DataType VARCHAR(255))
+CREATE PROCEDURE db_5_0.insertEntityDataType(in PropertyID VARCHAR(255), in DataTypeID VARCHAR(255))
 BEGIN
     DECLARE InternalPropertyID INT UNSIGNED DEFAULT NULL;
 
     SELECT internal_id INTO InternalPropertyID FROM entity_ids WHERE id=PropertyID;
 
-    INSERT INTO data_type (domain_id, entity_id, property_id, datatype) SELECT 0, 0, InternalPropertyID, ( SELECT entity_id FROM name_data WHERE domain_id = 0 AND property_id = 20 AND value = DataType LIMIT 1);
+    INSERT INTO data_type (domain_id, entity_id, property_id, datatype) SELECT 0, 0, InternalPropertyID, ( SELECT internal_id FROM entity_ids WHERE id = DataTypeID);
 
 
 END;