Skip to content
Snippets Groups Projects
Commit 06cda93c authored by Florian Spreckelsen's avatar Florian Spreckelsen Committed by Daniel Hornung
Browse files

DOC: Update and extend importance and inheritance

parent 9619d462
No related branches found
No related tags found
2 merge requests!41REL: update changelog, bump version of pom.xml, update DEPENDENCIES,!22DOC: Update and extend importance and inheritance
Hide whitespace changes
Inline Side-by-side
Showing
with 383 additions and 210 deletions
.gitignore
+ 1
− 0
View file @ 06cda93c
......@@ -38,3 +38,4 @@ authtoken/
# python
__pycache__
*~
CHANGELOG.md
+ 2
− 0
View file @ 06cda93c
......@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- #145 Documentation of importances and inheritance
### Security
## [v0.4.0] - 2021-06-21
......
src/doc/Data-Model.md
+ 61
− 34
View file @ 06cda93c
# CaosDB Data Model
The data structure is build from some basic building blocks which are shown in the following picture:
The data structure is built from some basic building blocks which are
shown in the following picture:
![entities](entities.png)
It has a base object called **Entity**. Entities are either **Record Types**, **Records**, or **Abstract
Properties** (and Record like objects for files).
What *kind of data* is stored is defined by the Record Types. Actual data is then stored in CaosDB as Records which are of some Record Type. Those Records can have Properties that contain information about the Record. The following is a more detailed explanation (also see this [paper](https://www.mdpi.com/2306-5729/4/2/83)).
> Record Types and Abstract Properties are used to define the ontology for a particular
> domain in which the RDMS is used. Records are used to store the actual data and therefore
> represent individuals or particular things, e.g. a particular experiment, a particular time series, etc.
**Record Types** define classes or types of things, e.g. persons, experiments, timeseries, etc. Records
can be viewed as members of the class defined by its Record Type. These classes can contain
Abstract Properties which define key-value relationships for properties of the things along
with the expected data type and possibly the default unit, a default value, or a range of permitted
values. As files on the back-end file system are a major focus of this database management system,
there is a special entity File that encapsulates typical file properties like path, size and checksum.
**Entities** can be related via binary, directed, transitive is-a relations which model both subtyping
and instantiation, depending on the relata. These relations construct a directed graph of the
Entities. If A is-a B we call A the child of B and B the parent of A. No adamant restrictions are
imposed on the relate of the is-a relation and thus, Entities can be children of multiple Entities.
Each Entity has a list of Entity Properties, or in short just **Properties**. An Entity Property is not an Entity of its own, but a triple of an Abstract Property, a value or Null, and
an Importance. The values can be numericals, strings, dates, any other valid value that fits into
one of several builtin data types, or, most notably, references to other Entities. The importance
is either obligatory, recommended, suggested, or fix. A valid child of an Entity implicitly
inherits its parent’s Properties according to their Importance, which means that it is obliged,
recommended or only suggested to have a Property with the same Abstract Property (or
any subtype thereof).
As opposed to Properties with other priorities, **Fixed Properties** have no effect on the Entity’s children. During the creation or update of Entities, the importances of the parents are
being checked by the Server. Missing obligatory Properties invalidate the transaction and
result in an error, by default. Missing Properties, when they are recommended, result in a
warning, but the transaction is considered valid. Entities with missing suggested Properties
are silently accepted as valid.
It has a base object called **Entity**. Entities are either **Record
Types**, **Records**, or **Abstract Properties** (and Record like
objects for files). What *kind of data* is stored is defined by the
Record Types. Actual data is then stored in CaosDB as Records which
are of some Record Type. Those Records can have Properties that
contain information about the Record. The following is a more detailed
explanation (also see this
[paper](https://www.mdpi.com/2306-5729/4/2/83)).
> Record Types and Abstract Properties are used to define the ontology
> for a particular domain in which the RDMS is used. Records are used
> to store the actual data and therefore represent individuals or
> particular things, e.g. a particular experiment, a particular time
> series, etc.
## Record Types
**Record Types** define classes or types of things, e.g. persons,
experiments, timeseries, etc. Records can be viewed as members of the
class defined by its Record Type. These classes can contain Abstract
Properties which define key-value relationships for properties of the
things along with the expected data type and possibly the default
unit, a default value, or a range of permitted values. As files on the
backend file system are a major focus of this database management
system, there is a special File entity that encapsulates typical file
properties like path, size and checksum.
## Entities
**Entities** can be related via binary, directed, transitive is-a
relations which model both subtyping and instantiation, depending on
the relata. These relations construct a directed graph of the
Entities. If A is-a B we call A the child of B and B the parent of
A. No adamant restrictions are imposed on the relate of the is-a
relation and thus, Entities can be children of multiple Entities.
## Properties
Each Entity has a list of Entity Properties, or in short just
**Properties**. An Entity Property is not an Entity of its own, but a
triple of an Abstract Property, a value or Null, and an
Importance. The values can be numerals, strings, dates, any other
valid value that fits into one of several builtin data types, or, most
notably, references to other Entities. The Importance is either
`obligatory`, `recommended`, `suggested`, or `fix`. A valid child of
an Entity implicitly inherits its parent’s Properties according to
their Importance, which means that it is obliged, recommended or only
suggested to have a Property with the same Abstract Property (or any
subtype thereof). As opposed to Properties with other priorities,
**Fixed Properties** have no effect on the Entity’s children.
During the creation or update of Entities, the importances of the
parents are checked by the server. Missing obligatory Properties
invalidate the transaction and result in an error, by default. Missing
Properties, when they are recommended, result in a warning, but the
transaction is considered valid. Entities with missing suggested
Properties are silently accepted as valid. See [the detailed
specification of RecordTypes](specification/RecordType) for more
information on importances and inheritance.
src/doc/_static/css/server_docu.css 0 → 100644
+ 9
− 0
View file @ 06cda93c
table, .wy-table-responsive table {
border-collapse: collapse;
table-layout: fixed;
}
table td, .wy-table-responsive table td {
word-wrap: break-word;
white-space: unset;
}
src/doc/conf.py
+ 7
− 1
View file @ 06cda93c
......@@ -94,7 +94,13 @@ html_theme = 'sphinx_rtd_theme'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['_static']
html_static_path = ['_static']
# These paths are either relative to html_static_path
# or fully qualified paths (eg. https://...)
html_css_files = [
'css/server_docu.css',
]
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
......
src/doc/specification/AbstractProperty.md
+ 3
− 3
View file @ 06cda93c
.. note ::
# Note #
This document has not been updated for a long time. Although it is concerned with the mostly
stable API, its content may no longer reflect the actual CaosDB behavior.
> This document has not been updated for a long time. Although it is concerned with the mostly
> stable API, its content may no longer reflect the actual CaosDB behavior.
# AbstractProperty Specification
......
src/doc/specification/RecordType.md deleted 100644 → 0
+ 0
− 166
View file @ 9619d462
# RecordType
**Warning:** This specification is outdated. It is included to serve as a starting point for a more up-to-date description of the `RecordType` entity.
## Overview
RecordTypes function as templates for [[Record|Records]], they provide a description for a type of Record and define which [[Property|Properties]] should be present. Properties come with an _importance_ attribute which tells the user or client program how strongly necessary the Property is. (As all other entities,) RecordTypes can be inherited from other RecordTypes (or any Entities). When RecordTypes inherit from other RecordTypes, the _inheritance_ flag tells which properties shall be inherited.
### Importance
Importances are more of a recommendation, Records with omitted Properties can technically still be committed.
| *Importance* | *Meaning* | *Consequence when omitted* |
|--------------|----------------------|----------------------------|
| `OBLIGATORY` | Property must be present | *FIXME* Can be forced to be committed the server, but usually an error will be returned. |
| `RECOMMENDED` | Property should be present (makes sense for most users) | *FIXME* Usually an error will be returned? |
| `SUGGESTED` (*FIXME* the default?) | Property may be present (may make sense for some users) | *FIXME* No negative consequence? |
*FIXME* Did I get recommended and suggested right?
### Inheritance
The _inheritance_ flag decides which [[Property|Properties]] are inherited from parents.
| *Inheritance* | *Meaning* |
|----------------|-----------|
| _None_ (default) | Nothing is being inherited |
| `OBLIGATORY` | *FIXME* Properteis of importance `OBLIGATORY` and above? |
| `SUGGESTED` | *FIXME* Properteis of importance `SIGGESTED` and above, so the same effect as `ALL`? |
| `ALL` | Copy everything from the parent. |
| `FIX` | *FIXME* in `tests/test_inheritance.py` of pyint_test this is used, what does it mean???|
## Examples
### GET Requests
#### Single-Get
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/1
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
</Response>
----
#### Multi-Get
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/1&2&3
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="2" name="Measurement" description="Description Measurement">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="3" name="Video" description="Description Video">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="obligatory" />
</RecordType>
</Response>
----
#### Get all
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/
GET http://localhost:8122/mpidsserver/RecordType
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response />
[View Ticket #2](http://dev.bmp.ds.mpg.de/heartdb/ticket/2)
----
### POST Requests
*Request is to be send to:*
POST http://localhost:8122/mpidsserver/RecordType/
POST http://localhost:8122/mpidsserver/RecordType/
----
#### Single-Post
*HTTP Body:*
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
</Post>
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
</RecordType>
</Response>
----
#### Multi-Post
*HTTP Body:*
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
<RecordType name="TimmsRecordType12" >
<Property id="6" importance="recommended"/>
<Property id="7" importance="obligatory"/>
<Property id="15" />
</RecordType>
</Post>
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11" description="null">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="suggested" />
</RecordType>
<RecordType id="41" name="TimmsRecordType12">
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="suggested" />
<Property id="15" name="TimmsIntProperty" type="integer" unit="kg" description="This is TimmsIntProperty" importance="suggested" />
</RecordType>
</Response>
= DELETE RecordTypes =
HTTP-DELETE-requests are to be send to `http://${host}:${port}/mpidsserver/RecordType/...`. Default port is 8123.
*Example*
{{{
DELETE http://${host}:${port}/mpidsserver/RecordType/1&2&3&4
}}}
src/doc/specification/RecordType.rst 0 → 100644
+ 283
− 0
View file @ 06cda93c
RecordType
==========
Overview
--------
RecordTypes act as templates for :doc:`Records <Record>`. They
provide a description for a type of Record and define which
:doc:`Properties <AbstractProperty>` could, should, or must be
present. Properties come with an `importance` attribute which tells
the user or client program how strongly necessary the Property is. (As
all other entities,) RecordTypes can inherit from other
RecordTypes (or any Entities). When RecordTypes inherit from other
RecordTypes, the `inheritance` flag tells which properties shall be
inherited from the parents to the children.
Importance
~~~~~~~~~~
The `importance` value of a Property states whether it may, should, or
must be present in all children of this Entity. The table below
lists the possible importances, together with the default behavior of
the server when the property is not present in a child.
.. list-table::
:widths: 25 25 50
:width: 100%
:header-rows: 1
* - Importance
- Meaning
- Consequence when omitted
* - ``OBLIGATORY``
- Property must be present
- Can be forced to be committed the server, but usually
an error will be returned.
* - ``RECOMMENDED``
- Property should be present (makes sense for most users)
- No consequence right now, but in future versions a warning will
be returned.
* - ``SUGGESTED`` (default)
- Property may be present (may make sense for some users)
- No negative consequence
* - ``FIX``
- Property is not present in any child
- ---
The default behavior can be overridden for individual transaction by setting the
`force-missing-obligatory` flag to ``ERROR`` (the default), ``WARN``, or ``IGNORE``. These flags
cause a transaction that would result in an entity with a missing `obligatory` property to return an
error, a warning, or to execute without a warning, respectively. The importance value ``FIX`` is
used for properties that are specific to the Entity itself and is explained below.
.. note::
Note that in contrast to RecordTypes, all properties of a
:doc:`Record` have the importance ``FIX`` since in general, even in the
case of a Record having children, its properties are not
inherited.
Inheritance
~~~~~~~~~~~
The `inheritance` flag specifies which :doc:`Properties
<AbstractProperty>` are inherited from the parent RecordType(s) to the
child.
.. list-table::
:widths: 40 60
:header-rows: 1
* - Inheritance
- Meaning
* - `None` (default)
- Nothing is inherited
* - ``OBLIGATORY``
- Properties of importance ``OBLIGATORY`` and above
* - ``RECOMMENDED``
- Properties of importance ``RECOMMENDED`` and above
* - ``SUGGESTED``
- Properties of importance ``SUGGESTED`` and above
* - ``ALL``
- Copy everything from the parent.
Note that inheritance ``ALL`` is essentially the same as ``SUGGESTED``
right now, but this behavior might change in the future. Also note
that properties of importance ``FIX`` are never inherited this
way. This is precisely what this importance level is meant for:
properties that are specific to the parent entity and are not
inherited by default. You can still add this property to the child
manually if you wish, though, but this is not covered by the
automatic inheritance.
.. note::
The behavior of `inheritance` is currently not well-defined if the
child is a :doc:`Record`, not a RecordType.
Examples
--------
.. warning::
The following examples have not been updated in a while and may
serve as a starting point, but are not guaranteed to be fully
applicable to the most recent versions of CaosDB.
GET Requests
~~~~~~~~~~~~
Single-Get
^^^^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/1
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
</Response>
Multi-Get
^^^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/1&2&3
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="2" name="Measurement" description="Description Measurement">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="3" name="Video" description="Description Video">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="obligatory" />
</RecordType>
</Response>
Get all
^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/
GET http://localhost:8122/RecordType
*Response:*
.. note::
Outdated? At least the bug tracker doesn't exist anymore.
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response />
[View Ticket #2](http://caosdb.example.com/caosdb/ticket/2)
POST Requests
~~~~~~~~~~~~~
*Request is to be send to:*
.. code-block::
POST http://localhost:8122/RecordType/
POST http://localhost:8122/RecordType/
Single-Post
^^^^^^^^^^^
*HTTP Body:*
.. code-block:: XML
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
</Post>
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
</RecordType>
</Response>
Multi-Post
^^^^^^^^^^
*HTTP Body:*
.. code-block:: XML
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
<RecordType name="TimmsRecordType12" >
<Property id="6" importance="recommended"/>
<Property id="7" importance="obligatory"/>
<Property id="15" />
</RecordType>
</Post>
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11" description="null">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="suggested" />
</RecordType>
<RecordType id="41" name="TimmsRecordType12">
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="suggested" />
<Property id="15" name="TimmsIntProperty" type="integer" unit="kg" description="This is TimmsIntProperty" importance="suggested" />
</RecordType>
</Response>
DELETE RecordTypes
~~~~~~~~~~~~~~~~~~
HTTP-DELETE-requests are to be send to ``http://${host}:${port}/RecordType/...``. Default port is 8123.
*Example*
.. code-block::
DELETE http://${host}:${port}/RecordType/1&2&3&4
src/main/java/org/caosdb/server/entity/StatementStatus.java
+ 17
− 6
View file @ 06cda93c
......@@ -23,13 +23,24 @@
package org.caosdb.server.entity;
/**
* The statement status has two purposes.
* The statement status has two purposes:
*
* <p>1. Storing the importance of an entity (any of OBLIGATORY, RECOMMENDED, SUGGESTED, or FIX). 2.
* Marking an entity as a REPLACEMENT which is needed for flat representation of deeply nested
* properties. This constant is only used for internal processes and has no meaning in the API. That
* is also the reason why this enum is not called "Importance". Apart from that, in most cases its
* meaning is identical to the importance of an entity.
* <p> 1. Storing the importance of an entity (any of ``OBLIGATORY``,
* ``RECOMMENDED``, ``SUGGESTED``, or ``FIX``). ``OBLIGATORY``,
* ``RECOMMENDED``, and ``SUGGESTED`` specify whether this Entity
* must, should or may be present, and whether the server will throw
* an error, a warning or nothing, respectively, if it is missing. The
* importance is also used to specify the level of inheritance whereby
* ``FIX`` means that the entity is never inherited. See <a
* href="../../../../../specification/RecordType.html">the
* documentation of RecordTypes</a> for more information.
*
* <p> 2. Marking an entity as a ``REPLACEMENT`` which is needed for
* flat representation of deeply nested properties. This constant is
* only used for internal processes and has no meaning in the
* API. That is also the reason why this enum is not called
* "Importance". Apart from that, in most cases its meaning is
* identical to the importance of an entity.
*
* @author Timm Fitschen (t.fitschen@indiscale.com)
*/
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment