caosdb-server issueshttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues2024-03-04T08:35:34Zhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/330Extern: Unexpected Server Error when updating an Entity with a FILE Property2024-03-04T08:35:34ZHenrik tom WördenExtern: Unexpected Server Error when updating an Entity with a FILE Propertyhttps://gitlab.com/linkahead/linkahead-server/-/issues/246https://gitlab.com/linkahead/linkahead-server/-/issues/246Henrik tom WördenHenrik tom Wördenhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/320Extern: Cannot update entity permissions if file is missing.2024-03-18T00:05:06ZTimm Fitschent.fitschen@indiscale.comExtern: Cannot update entity permissions if file is missing.https://gitlab.com/linkahead/linkahead-server/-/issues/36https://gitlab.com/linkahead/linkahead-server/-/issues/36Timm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/318EXTERN: Case mismatch causes permissions to be invalid2024-03-18T10:21:26ZFlorian SpreckelsenEXTERN: Case mismatch causes permissions to be invalidhttps://gitlab.com/caosdb/caosdb-server/-/issues/234https://gitlab.com/caosdb/caosdb-server/-/issues/234Florian SpreckelsenFlorian Spreckelsenhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/317EXTERN: Server handles states of properties incorrectly2024-03-18T10:21:30ZFlorian SpreckelsenEXTERN: Server handles states of properties incorrectlyhttps://gitlab.com/caosdb/caosdb-server/-/issues/225https://gitlab.com/caosdb/caosdb-server/-/issues/225Florian SpreckelsenFlorian Spreckelsenhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/313EXTERN: Server handles states of properties incorrectly2023-09-14T11:26:46ZFlorian SpreckelsenEXTERN: Server handles states of properties incorrectlyhttps://gitlab.com/caosdb/caosdb-server/-/issues/225
## Steps to Reproduce the Problem
Reproducible with Geomar's production dump in `/nextCloud/Customer/GEOMAR/Dump Testinstanz/*2023-07-28*`
1. Restore from this dump
2. Run mygration...https://gitlab.com/caosdb/caosdb-server/-/issues/225
## Steps to Reproduce the Problem
Reproducible with Geomar's production dump in `/nextCloud/Customer/GEOMAR/Dump Testinstanz/*2023-07-28*`
1. Restore from this dump
2. Run mygrations in `geomar/database-schema/migrations` up to `003_...`
3. Look at the output of `FIND Sample WHICH HAS A "Nagoya Case"`Henrik tom WördenHenrik tom Wördenhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/310Context Root does not work2023-06-30T11:06:42ZHenrik tom WördenContext Root does not work
Steps to reproduce:
- use demo profile of deploy
- add a file named `profiles/default/custom/caosdb-server/conf/ext/server.conf.d/50-default.conf` with content `CONTEXT_ROOT=/demo` and start the server
- when starting, server prints in...
Steps to reproduce:
- use demo profile of deploy
- add a file named `profiles/default/custom/caosdb-server/conf/ext/server.conf.d/50-default.conf` with content `CONTEXT_ROOT=/demo` and start the server
- when starting, server prints in debug `caosdb-server_1 | [20-Feb-13 14:15:32:731] [INFO] [ServerProperties:173] - CONTEXT_ROOT=/demo`
- visit `https://localhost:10443/demo`
- Basically, what ever you do (Send a query, click on "Entities", click on "Filesystem". The context root is removed.Timm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/295Extern: Server accepts any string for DATETIME properties.2024-03-18T00:05:06ZDaniel HornungExtern: Server accepts any string for DATETIME properties.https://gitlab.com/caosdb/caosdb-server/-/issues/205
The server does not properly check if datetime properties follow some format.https://gitlab.com/caosdb/caosdb-server/-/issues/205
The server does not properly check if datetime properties follow some format.BatCAT WP4 - Knowledge IntegrationTimm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/291Extern: FileSystemCheck does not terminate2023-05-15T07:51:40ZHenrik tom WördenExtern: FileSystemCheck does not terminatehttps://gitlab.com/caosdb/caosdb-server/-/issues/201https://gitlab.com/caosdb/caosdb-server/-/issues/201Daniel HornungDaniel Hornunghttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/290Extern: Startup fails due to problems with OpenSSL and netty-tcnative on Debi...2024-03-18T00:05:06ZHenrik tom WördenExtern: Startup fails due to problems with OpenSSL and netty-tcnative on Debian 11https://gitlab.com/caosdb/caosdb-server/-/issues/158https://gitlab.com/caosdb/caosdb-server/-/issues/158Timm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/289Extern: (Query-)Caches need to be cleared when permissions are changed2024-03-18T00:05:06ZHenrik tom WördenExtern: (Query-)Caches need to be cleared when permissions are changedhttps://gitlab.com/caosdb/caosdb-server/-/issues/156https://gitlab.com/caosdb/caosdb-server/-/issues/156Timm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/253Documentation: CQL: difference between CREATED ON and CREATED AT (and maybe ...2022-12-12T11:10:51ZDaniel HornungDocumentation: CQL: difference between CREATED ON and CREATED AT (and maybe also BEFORE, AFTER, ...)## Summary
There is a subtle difference between `CREATED AT` (which requires the exact datetime?) and `CREATED ON/IN` (which works fine with e.g. `TODAY`).
The different possibilities should be documented, maybe with a small table?
Al...## Summary
There is a subtle difference between `CREATED AT` (which requires the exact datetime?) and `CREATED ON/IN` (which works fine with e.g. `TODAY`).
The different possibilities should be documented, maybe with a small table?
Also, there seem to be no tests for `AT` times, and only one (TestTicket156) for ON (which should be OK since there are multiple tests for `IN`.
## Expected Behavior
Should be documented.
## Actual Behavior
Is not.
## Bonus features
- Bonus points if the webui could hint at the differences.
- Discuss: Is this necessary?Timm Fitschent.fitschen@indiscale.comDaniel HornungTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/251Incomplete permissions list for entities when retrieved by anonymous admin2022-08-17T14:29:27ZDaniel HornungIncomplete permissions list for entities when retrieved by anonymous admin## Summary
When the anonymous admin retrieves an entity, the permission list is incomplete. This causes the web ui to leave out some edit mode buttons.
## Expected Behavior
The permissions list should be complete.
## Actual Behavior...## Summary
When the anonymous admin retrieves an entity, the permission list is incomplete. This causes the web ui to leave out some edit mode buttons.
## Expected Behavior
The permissions list should be complete.
## Actual Behavior
It is not.
## Steps to Reproduce the Problem
In Python:
```py
import caosdb as db
db.Info().user_info.name # Is empty, as it should be.
ent = db.execute_query("FIND Record with name like xyz*")[0]
ent.permissions
# Out[23]: {USE:AS_PARENT, RETRIEVE:HISTORY, USE:AS_REFERENCE, RETRIEVE:ENTITY, USE:AS_PROPERTY, RETRIEVE:FILE, RETRIEVE:OWNER, USE:AS_DATA_TYPE, RETRIEVE:ACL}
```
## Specifications
- Version: Latest release
- Platform: Linux
## Possible fixesTimm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/245The entities table contains residuals2022-11-23T12:25:09ZHenrik tom WördenThe entities table contains residuals## Summary
The `entities` table is not cleaned up properly.
## Expected Behavior
After inserting and deleting Entities the `entities` table does not contain residuals (like domain ids).
## Actual Behavior
It can occur that a Domain...## Summary
The `entities` table is not cleaned up properly.
## Expected Behavior
After inserting and deleting Entities the `entities` table does not contain residuals (like domain ids).
## Actual Behavior
It can occur that a Domain persists.
## Steps to Reproduce the Problem
1. Start a server with an empty database in debug mode
2. Perform the python integration tests in `tests/test_query.py`
3. Check the `entities` table using mysql
The initial state of the `entities` table:
```
MariaDB [caosdb]> select * from entities;
+----+---------------------------------------+------------+------+
| id | description | role | acl |
+----+---------------------------------------+------------+------+
| 0 | The default domain. | ROLE | 0 |
| 1 | The default recordtype. | ROLE | 0 |
| 2 | The default record. | ROLE | 0 |
| 3 | The default file. | ROLE | 0 |
| 4 | The default property. | ROLE | 0 |
| 7 | The default datatype. | ROLE | 0 |
| 8 | The QueryTemplate role. | ROLE | 0 |
| 11 | The default reference data type. | DATATYPE | 0 |
| 12 | The default integer data type. | DATATYPE | 0 |
| 13 | The default double data type. | DATATYPE | 0 |
| 14 | The default text data type. | DATATYPE | 0 |
| 15 | The default datetime data type. | DATATYPE | 0 |
| 16 | The default timespan data type. | DATATYPE | 0 |
| 17 | The default file reference data type. | DATATYPE | 0 |
| 18 | The defaulf boolean data type | DATATYPE | 0 |
| 20 | Name of an entity | PROPERTY | 0 |
| 21 | Unit of an entity. | PROPERTY | 0 |
| 24 | Description of an entity. | PROPERTY | 0 |
| 50 | The SQLite file data type. | DATATYPE | 0 |
| 99 | NULL | RECORDTYPE | 0 |
+----+---------------------------------------+------------+------+
20 rows in set (0.121 sec)
```
After the integrationtests, there is an additional row:
```
| 105 | NULL | DOMAIN | 0 |
```
## Specifications
I did this with server commit 5f9962cbca.
pyint: 6f6a570
## Possible fixes
*Do you have ideas how the issue can be resolved?*https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/222Update of file when not setting "size" leads to an unknown server error2022-03-11T08:46:55ZAlexander SchlemmerUpdate of file when not setting "size" leads to an unknown server errorThe following reproducibly leads to an unknown server error:
The entity with id=174 is a File.
rt can be an arbitrary RecordType.
```python
f = db.File(id=174).retrieve()
f._size = None
f.add_parent(rt)
f.update()
```
The error does n...The following reproducibly leads to an unknown server error:
The entity with id=174 is a File.
rt can be an arbitrary RecordType.
```python
f = db.File(id=174).retrieve()
f._size = None
f.add_parent(rt)
f.update()
```
The error does not occur when setting _checksum to None.Refactoring of File Storage APITimm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/219Unknown error instead of "Entity does not exist error"2022-05-16T08:21:24ZAlexander SchlemmerUnknown error instead of "Entity does not exist error"I recently tried inserting a large container using the crawler, but forgot to insert the data model first.
The server responded with:
```
Traceback (most recent call last):
File "/home/salexan/.local/bin/crawler2.0", line 33, in <modu...I recently tried inserting a large container using the crawler, but forgot to insert the data model first.
The server responded with:
```
Traceback (most recent call last):
File "/home/salexan/.local/bin/crawler2.0", line 33, in <module>
sys.exit(load_entry_point('newcrawler', 'console_scripts', 'crawler2.0')())
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 780, in main
crawler.synchronize(commit_changes=True)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 330, in synchronize
return self._synchronize(self.updateList, commit_changes)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 631, in _synchronize
self.execute_inserts_in_list(to_be_inserted)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 584, in execute_inserts_in_list
db.Container().extend(to_be_inserted).insert()
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/common/models.py", line 3472, in insert
http_response = con.insert(
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/connection/connection.py", line 551, in insert
http_response = self._http_request(
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/connection/connection.py", line 575, in _http_request
return self._retry_http_request(method=method, path=path,
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/connection/connection.py", line 615, in _retry_http_request
_handle_response_status(http_response)
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/connection/connection.py", line 416, in _handle_response_status
raise HTTPServerError(body=body)
caosdb.exceptions.HTTPServerError: An unexpected server error has occurred.
SRID = e7d63ee6-9cf2-4d5c-94f7-dcd971fa9b28
Please report this server error to and include the SRID into your report.
```
... although, on the server-side things seemed pretty clear:
```
linkahead | RESPONSE: HTTPS/1.1 - Internal Server Error (500) - The server encountered an unexpected condition which prevented it from fulfilling the request
linkahead | RESPONSE ENTITY:<?xml version="1.0" encoding="UTF-8"?>
linkahead | <?xml-stylesheet type="text/xsl" href="https://localhost:10443/webinterface/1646744029/webcaosdb.xsl" ?>
linkahead | <Response username="admin" realm="PAM" srid="e7d63ee6-9cf2-4d5c-94f7-dcd971fa9b28" timestamp="1646744821061" baseuri="https://localhost:10443">
linkahead | <noscript>
linkahead | <h1>Please enable JavaScript!</h1>
linkahead | </noscript>
linkahead | <UserInfo username="admin" realm="PAM">
linkahead | <Roles>
linkahead | <Role>administration</Role>
linkahead | </Roles>
linkahead | </UserInfo>
linkahead | <Error code="0" description="An unexpected server error has occurred.">SRID = e7d63ee6-9cf2-4d5c-94f7-dcd971fa9b28
linkahead |
linkahead | Please report this server error to and include the SRID into your report.</Error>
linkahead | </Response>
linkahead |
linkahead | org.caosdb.server.database.exceptions.EntityDoesNotExistException: null
linkahead | at org.caosdb.server.database.backend.transaction.GetIDByName.execute(GetIDByName.java:69) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.database.BackendTransaction.executeTransaction(BackendTransaction.java:152) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.transaction.TransactionInterface.execute(TransactionInterface.java:49) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.Job.execute(Job.java:91) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.Job.retrieveValidIDByName(Job.java:293) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.Job.resolve(Job.java:556) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.core.Inheritance.run(Inheritance.java:144) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.ScheduledJob.run(ScheduledJob.java:47) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
linkahead | at org.caosdb.server.jobs.Schedule.runJob(Schedule.java:83) ~[caosdb-server-0.8.0-SNAPSHOT-jar-with-dependencies.jar:?]
```
Obviously, that's an "entity does not exist error" which is not forwarded to the client correctly.Daniel HornungDaniel Hornunghttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/218Simultaneous update of a file's parent and adding to a record (as part of an ...2022-03-07T14:05:06ZAlexander SchlemmerSimultaneous update of a file's parent and adding to a record (as part of an update) impossibleI am trying to do a simultaneous update (using a single container from pylib) of:
- adding parent "SimulationOutput" to a file (id=169)
- adding the file to a LIST<SimulationOutput> - property (id=156, name="output") of an existing recor...I am trying to do a simultaneous update (using a single container from pylib) of:
- adding parent "SimulationOutput" to a file (id=169)
- adding the file to a LIST<SimulationOutput> - property (id=156, name="output") of an existing record
This currently leads to an error: "Reference not qualified. The value of this Reference Property is to be a child of its data type."
This would be correct if I had not added the parent in the first place. Somehow, the server does not take into account the simultaneously added parent during the type check for the property of the record.
If I do an ".update()" on both entries of the container separately, adding the parent first, the update transaction works.
(xml shortened):
```xml
<File id="169" path=".../SimulationOutput/run1/result.npy">
<Parent id="158" name="SimulationOutput"/>
</File>
,
<Record id="185" description="*[possibly short] description of the event*">
<Parent id="155" name="SimulationCase"/>
(...)
<Property id="156" name="output" importance="FIX" flag="inheritance:FIX">
<Value>169</Value>
</Property>
</Record>
```
```
Traceback (most recent call last):
File "/home/salexan/.local/bin/crawler2.0", line 33, in <module>
sys.exit(load_entry_point('newcrawler', 'console_scripts', 'crawler2.0')())
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 759, in main
crawler.synchronize(commit_changes=True)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 327, in synchronize
return self._synchronize(self.updateList, commit_changes)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 611, in _synchronize
self.execute_updates_in_list(to_be_updated)
File "/home/salexan/Projekte/CaosDB/caosdb-newcrawler/src/newcrawler/crawl.py", line 578, in execute_updates_in_list
db.Container().extend(to_be_updated).update()
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/common/models.py", line 3316, in update
raise_errors(cresp)
File "/home/salexan/Projekte/CaosDB/caosdb-pylib/src/caosdb/common/models.py", line 4384, in raise_errors
raise transaction_error
caosdb.exceptions.TransactionError: TransactionError: One or more entities are not qualified. None of them have been inserted/updated/deleted.
+--| REASONS |--
| -> RECORD (id: 185) CAUSED |UnqualifiedPropertiesError: Entity has unqualified properties.
| +--| REASONS |--
| | -> PROPERTY (id: 156, name: 'output') CAUSED | |EntityError: Reference not qualified. The value of this Reference Property is to be a child of its data type.
| +----------------
+----------------
```Timm Fitschent.fitschen@indiscale.comDaniel HornungTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/213Confusing output of Info-Resource2022-01-18T09:44:10ZAlexander SchlemmerConfusing output of Info-ResourceIn a new, empty CaosDB instance I issued:
```
https://localhost:10443/Entity/?all=Property&P=0L10
```
and received (as expected):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="https://localhost:104...In a new, empty CaosDB instance I issued:
```
https://localhost:10443/Entity/?all=Property&P=0L10
```
and received (as expected):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="https://localhost:10443/webinterface/1642496917/webcaosdb.xsl" ?>
<Response username="admin" realm="PAM" srid="..." timestamp="..." baseuri="https://localhost:10443" count="0">
<noscript>
<h1>Please enable JavaScript!</h1>
</noscript>
<UserInfo username="admin" realm="PAM">
<Roles>
<Role>administration</Role>
</Roles>
</UserInfo>
<TransactionBenchmark info="TransactionBenchmark is disabled." />
</Response>
```
... indicating an empty database.
However, the Info-Resource (https://localhost:10443/Info) responds with:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="https://localhost:10443/webinterface/1642496917/webcaosdb.xsl" ?>
<Response username="admin" realm="PAM" srid="..." timestamp="..." baseuri="https://localhost:10443">
<noscript>
<h1>Please enable JavaScript!</h1>
</noscript>
<UserInfo username="admin" realm="PAM">
<Roles>
<Role>administration</Role>
</Roles>
</UserInfo>
<TimeZone offset="+0000" id="Etc/UTC">Coordinated Universal Time</TimeZone>
<Stats>
<counts records="0" properties="3" recordTypes="1" files="0" fssize="0B" tmpfiles="0" debug="true" />
<dropOffBox path="/opt/caosdb/mnt/dropoffbox" />
</Stats>
<TransactionBenchmark info="TransactionBenchmark is disabled." />
</Response>
```
(left out flags)
So, whom do I trust? Info counts 3 properties and 1 record type... but they are not yielded when retrieving all the Entities.
I need the Info-Resource for checking the database for emptiness, so this approach will lead to bugs.Timm Fitschent.fitschen@indiscale.comTimm Fitschent.fitschen@indiscale.comhttps://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/211Cannot change name even with the appropriate permission2021-12-28T10:48:03ZHenrik tom WördenCannot change name even with the appropriate permissionI gave a user a role that allows all transactions and added a permission rule to the entity acl that only provides a single update permission: Changing the name.
Changing the name fails. With the Error "You are not allowed to do this"
...I gave a user a role that allows all transactions and added a permission rule to the entity acl that only provides a single update permission: Changing the name.
Changing the name fails. With the Error "You are not allowed to do this"
In the response xml:
```xml
<Error code="403" description="You are not allowed to do this." />
<Warning code="1104" description="User role does not exist." />
<Info code="0" description="User Role `admin@PAM` does not exist." />
<Info code="0" description="henrik doesn't have permission update:property:add" />https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/210A RecordType with a datatype property can be created.2022-09-01T15:31:26ZHenrik tom WördenA RecordType with a datatype property can be created.`db.RecordType("TestDT", datatype=db.DOUBLE).insert()``db.RecordType("TestDT", datatype=db.DOUBLE).insert()`https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/209Pattern in path is matched incorrectly2022-02-07T09:25:05ZHenrik tom WördenPattern in path is matched incorrectlyThe result of ` find file which is stored at "**/\~\$*` is incorrect.The result of ` find file which is stored at "**/\~\$*` is incorrect.Henrik tom WördenHenrik tom Wörden