Unexpected difference between xml representation and value in pylib

assigned to @timm

made the issue confidential

I just noticed, if I'm appending a value to the list, it seems like it's appended twice (but still there is only one value in .value).

This does not fix the problem:

old_entity.get_property("responsible").value = old_entity.get_property("responsible").value

Neither this one does:

old_entity.get_property("responsible").value = [old_entity.get_property("responsible").value[0]]

Ok, I digged into pylib a little bit:

It seems like it is turning the "_wrapped_entity" (whatever it is) into this:

(Pdb) self._wrapped_entity
<Property name="output">
  <Value>124</Value>
</Property>

afterwards, it walks through self again and adds the value's list elements.

What is "_wrapped_entity"? I cannot find any documentation.

I'm not sure, but adding a return statement here, resolves the issue for me:

    def to_xml(self, xml=None, add_properties=ALL, local_serialization=False):
        """Generate an xml representation of this entity. If the parameter xml
        is given, all attributes, parents, properties, and messages of this
        entity will be added to it instead of creating a new element.

        Raise an error if xml is not a lxml.etree.Element

        @param xml: an xml element to which all attributes, parents,
            properties, and messages
            are to be added.
        @return: xml representation of this entity.
        """

        if xml is None:
            # use role as xml tag name, fall-back to "Entity"
            elem_tag = "Entity" if self.role is None else self.role
            xml = etree.Element(elem_tag)
        assert isinstance(xml, etree._Element)

        # unwrap wrapped entity

        if self._wrapped_entity is not None:
            xml = self._wrapped_entity.to_xml(xml, add_properties)
            return xml  # <- RETURN STATEMENT

        if self.id is not None:
            xml.set("id", str(self.id))

        if self._cuid is not None:
            xml.set("cuid", str(self._cuid))

        if self.name is not None:
            xml.set("name", str(self.name))

Does it break anything?

Yes, this will break things.

Can you explain how to construct old_entity in that state?

made the issue visible to everyone

added Need Info label

added Triage label

removed S1 label

removed BugUnconfirmed label

Just for reference: I had used the following fix to resolve the issue for me temporarily and for being able to user crawler2.0 without trouble:

@@ -865,8 +865,9 @@ class Entity(object):
 
         # unwrap wrapped entity
 
-        if self._wrapped_entity is not None:
-            xml = self._wrapped_entity.to_xml(xml, add_properties)
+        # if self._wrapped_entity is not None:
+        #     xml = self._wrapped_entity.to_xml(xml, add_properties)
+        #     return xml

I will try now to isolate the bug a bit and add information here.

Just found out that it happens in the crawler only during insertion, not during update of the same entities. I try to create a test case.

Ok, I can confirm now that it is the (new) function "merge_entities(entity_a: Entity, entity_b: Entity):" from apiutils in pylib, that breaks the entity structure.

Ok, I'm able to isolate and reproduce the bug now:

import caosdb as db

rt = db.RecordType(name="TestBug")
p = db.Property(name="test_bug_property", datatype=db.LIST(db.INTEGER))

r_b = db.Record(name="TestRecord")
r_b.add_parent(rt)
r_b.add_property(p, value=[18, 19])

r_a = db.Record(name="TestRecord")
r_a.add_parent(rt)
r_a.add_property(r_b.get_property("test_bug_property"))

print(r_b)
print(r_b.get_property("test_bug_property").value)

print(r_a)
print(r_a.get_property("test_bug_property").value)

Output:

<Record name="TestRecord">
  <Parent name="TestBug"/>
  <Property name="test_bug_property" datatype="LIST&lt;INTEGER&gt;" importance="FIX" flag="inheritance:FIX">
    <Value>18</Value>
    <Value>19</Value>
  </Property>
</Record>

[18, 19]
<Record name="TestRecord">
  <Parent name="TestBug"/>
  <Property name="test_bug_property" datatype="LIST&lt;INTEGER&gt;" importance="FIX" flag="inheritance:FIX">
    <Value>18</Value>
    <Value>19</Value>
    <Value>18</Value>
    <Value>19</Value>
  </Property>
</Record>

[18, 19]

@timm Could you have a look at it?

added BugUnconfirmed label

added S1 label

Unexpected difference between xml representation and value in pylib

Child items ...

Activity