Skip to content
Snippets Groups Projects
Commit 24bffcc8 authored by Daniel Hornung's avatar Daniel Hornung
Browse files

Merge branch 'f-doc-inheritance' into 'dev' 

DOC: Update and extend importance and inheritance

See merge request !22
parents 9619d462 06cda93c
No related branches found
No related tags found
2 merge requests!41REL: update changelog, bump version of pom.xml, update DEPENDENCIES,!22DOC: Update and extend importance and inheritance
Pipeline #9292 passed
Stage: setup
Stage: test
Stage: deploy
Show whitespace changes
Inline Side-by-side
Showing
with 383 additions and 210 deletions
.gitignore
+ 1
− 0
View file @ 24bffcc8
...@@ -38,3 +38,4 @@ authtoken/ ...@@ -38,3 +38,4 @@ authtoken/
# python # python
__pycache__ __pycache__
*~
CHANGELOG.md
+ 2
− 0
View file @ 24bffcc8
...@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed ### Fixed
- #145 Documentation of importances and inheritance
### Security ### Security
## [v0.4.0] - 2021-06-21 ## [v0.4.0] - 2021-06-21
......
src/doc/Data-Model.md
+ 61
− 34
View file @ 24bffcc8
# CaosDB Data Model # CaosDB Data Model
The data structure is build from some basic building blocks which are shown in the following picture: The data structure is built from some basic building blocks which are
shown in the following picture:
![entities](entities.png) ![entities](entities.png)
It has a base object called **Entity**. Entities are either **Record Types**, **Records**, or **Abstract It has a base object called **Entity**. Entities are either **Record
Properties** (and Record like objects for files). Types**, **Records**, or **Abstract Properties** (and Record like
What *kind of data* is stored is defined by the Record Types. Actual data is then stored in CaosDB as Records which are of some Record Type. Those Records can have Properties that contain information about the Record. The following is a more detailed explanation (also see this [paper](https://www.mdpi.com/2306-5729/4/2/83)). objects for files). What *kind of data* is stored is defined by the
Record Types. Actual data is then stored in CaosDB as Records which
> Record Types and Abstract Properties are used to define the ontology for a particular are of some Record Type. Those Records can have Properties that
> domain in which the RDMS is used. Records are used to store the actual data and therefore contain information about the Record. The following is a more detailed
> represent individuals or particular things, e.g. a particular experiment, a particular time series, etc. explanation (also see this
[paper](https://www.mdpi.com/2306-5729/4/2/83)).
**Record Types** define classes or types of things, e.g. persons, experiments, timeseries, etc. Records
can be viewed as members of the class defined by its Record Type. These classes can contain > Record Types and Abstract Properties are used to define the ontology
Abstract Properties which define key-value relationships for properties of the things along > for a particular domain in which the RDMS is used. Records are used
with the expected data type and possibly the default unit, a default value, or a range of permitted > to store the actual data and therefore represent individuals or
values. As files on the back-end file system are a major focus of this database management system, > particular things, e.g. a particular experiment, a particular time
there is a special entity File that encapsulates typical file properties like path, size and checksum. > series, etc.
**Entities** can be related via binary, directed, transitive is-a relations which model both subtyping ## Record Types
and instantiation, depending on the relata. These relations construct a directed graph of the
Entities. If A is-a B we call A the child of B and B the parent of A. No adamant restrictions are **Record Types** define classes or types of things, e.g. persons,
imposed on the relate of the is-a relation and thus, Entities can be children of multiple Entities. experiments, timeseries, etc. Records can be viewed as members of the
class defined by its Record Type. These classes can contain Abstract
Each Entity has a list of Entity Properties, or in short just **Properties**. An Entity Property is not an Entity of its own, but a triple of an Abstract Property, a value or Null, and Properties which define key-value relationships for properties of the
an Importance. The values can be numericals, strings, dates, any other valid value that fits into things along with the expected data type and possibly the default
one of several builtin data types, or, most notably, references to other Entities. The importance unit, a default value, or a range of permitted values. As files on the
is either obligatory, recommended, suggested, or fix. A valid child of an Entity implicitly backend file system are a major focus of this database management
inherits its parent’s Properties according to their Importance, which means that it is obliged, system, there is a special File entity that encapsulates typical file
recommended or only suggested to have a Property with the same Abstract Property (or properties like path, size and checksum.
any subtype thereof).
## Entities
As opposed to Properties with other priorities, **Fixed Properties** have no effect on the Entity’s children. During the creation or update of Entities, the importances of the parents are
being checked by the Server. Missing obligatory Properties invalidate the transaction and **Entities** can be related via binary, directed, transitive is-a
result in an error, by default. Missing Properties, when they are recommended, result in a relations which model both subtyping and instantiation, depending on
warning, but the transaction is considered valid. Entities with missing suggested Properties the relata. These relations construct a directed graph of the
are silently accepted as valid. Entities. If A is-a B we call A the child of B and B the parent of
A. No adamant restrictions are imposed on the relate of the is-a
relation and thus, Entities can be children of multiple Entities.
## Properties
Each Entity has a list of Entity Properties, or in short just
**Properties**. An Entity Property is not an Entity of its own, but a
triple of an Abstract Property, a value or Null, and an
Importance. The values can be numerals, strings, dates, any other
valid value that fits into one of several builtin data types, or, most
notably, references to other Entities. The Importance is either
`obligatory`, `recommended`, `suggested`, or `fix`. A valid child of
an Entity implicitly inherits its parent’s Properties according to
their Importance, which means that it is obliged, recommended or only
suggested to have a Property with the same Abstract Property (or any
subtype thereof). As opposed to Properties with other priorities,
**Fixed Properties** have no effect on the Entity’s children.
During the creation or update of Entities, the importances of the
parents are checked by the server. Missing obligatory Properties
invalidate the transaction and result in an error, by default. Missing
Properties, when they are recommended, result in a warning, but the
transaction is considered valid. Entities with missing suggested
Properties are silently accepted as valid. See [the detailed
specification of RecordTypes](specification/RecordType) for more
information on importances and inheritance.
src/doc/_static/css/server_docu.css 0 → 100644
+ 9
− 0
View file @ 24bffcc8
table, .wy-table-responsive table {
border-collapse: collapse;
table-layout: fixed;
}
table td, .wy-table-responsive table td {
word-wrap: break-word;
white-space: unset;
}
src/doc/conf.py
+ 7
− 1
View file @ 24bffcc8
...@@ -94,7 +94,13 @@ html_theme = 'sphinx_rtd_theme' ...@@ -94,7 +94,13 @@ html_theme = 'sphinx_rtd_theme'
# Add any paths that contain custom static files (such as style sheets) here, # Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files, # relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css". # so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['_static'] html_static_path = ['_static']
# These paths are either relative to html_static_path
# or fully qualified paths (eg. https://...)
html_css_files = [
'css/server_docu.css',
]
# Custom sidebar templates, must be a dictionary that maps document names # Custom sidebar templates, must be a dictionary that maps document names
# to template names. # to template names.
......
src/doc/specification/AbstractProperty.md
+ 3
− 3
View file @ 24bffcc8
.. note :: # Note #
This document has not been updated for a long time. Although it is concerned with the mostly > This document has not been updated for a long time. Although it is concerned with the mostly
stable API, its content may no longer reflect the actual CaosDB behavior. > stable API, its content may no longer reflect the actual CaosDB behavior.
# AbstractProperty Specification # AbstractProperty Specification
......
src/doc/specification/RecordType.md deleted 100644 → 0
+ 0
− 166
View file @ 9619d462
# RecordType
**Warning:** This specification is outdated. It is included to serve as a starting point for a more up-to-date description of the `RecordType` entity.
## Overview
RecordTypes function as templates for [[Record|Records]], they provide a description for a type of Record and define which [[Property|Properties]] should be present. Properties come with an _importance_ attribute which tells the user or client program how strongly necessary the Property is. (As all other entities,) RecordTypes can be inherited from other RecordTypes (or any Entities). When RecordTypes inherit from other RecordTypes, the _inheritance_ flag tells which properties shall be inherited.
### Importance
Importances are more of a recommendation, Records with omitted Properties can technically still be committed.
| *Importance* | *Meaning* | *Consequence when omitted* |
|--------------|----------------------|----------------------------|
| `OBLIGATORY` | Property must be present | *FIXME* Can be forced to be committed the server, but usually an error will be returned. |
| `RECOMMENDED` | Property should be present (makes sense for most users) | *FIXME* Usually an error will be returned? |
| `SUGGESTED` (*FIXME* the default?) | Property may be present (may make sense for some users) | *FIXME* No negative consequence? |
*FIXME* Did I get recommended and suggested right?
### Inheritance
The _inheritance_ flag decides which [[Property|Properties]] are inherited from parents.
| *Inheritance* | *Meaning* |
|----------------|-----------|
| _None_ (default) | Nothing is being inherited |
| `OBLIGATORY` | *FIXME* Properteis of importance `OBLIGATORY` and above? |
| `SUGGESTED` | *FIXME* Properteis of importance `SIGGESTED` and above, so the same effect as `ALL`? |
| `ALL` | Copy everything from the parent. |
| `FIX` | *FIXME* in `tests/test_inheritance.py` of pyint_test this is used, what does it mean???|
## Examples
### GET Requests
#### Single-Get
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/1
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
</Response>
----
#### Multi-Get
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/1&2&3
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="2" name="Measurement" description="Description Measurement">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="3" name="Video" description="Description Video">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="obligatory" />
</RecordType>
</Response>
----
#### Get all
*Request:*
GET http://localhost:8122/mpidsserver/RecordType/
GET http://localhost:8122/mpidsserver/RecordType
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response />
[View Ticket #2](http://dev.bmp.ds.mpg.de/heartdb/ticket/2)
----
### POST Requests
*Request is to be send to:*
POST http://localhost:8122/mpidsserver/RecordType/
POST http://localhost:8122/mpidsserver/RecordType/
----
#### Single-Post
*HTTP Body:*
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
</Post>
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
</RecordType>
</Response>
----
#### Multi-Post
*HTTP Body:*
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
<RecordType name="TimmsRecordType12" >
<Property id="6" importance="recommended"/>
<Property id="7" importance="obligatory"/>
<Property id="15" />
</RecordType>
</Post>
*Response:*
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11" description="null">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="suggested" />
</RecordType>
<RecordType id="41" name="TimmsRecordType12">
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="suggested" />
<Property id="15" name="TimmsIntProperty" type="integer" unit="kg" description="This is TimmsIntProperty" importance="suggested" />
</RecordType>
</Response>
= DELETE RecordTypes =
HTTP-DELETE-requests are to be send to `http://${host}:${port}/mpidsserver/RecordType/...`. Default port is 8123.
*Example*
{{{
DELETE http://${host}:${port}/mpidsserver/RecordType/1&2&3&4
}}}
src/doc/specification/RecordType.rst 0 → 100644
+ 283
− 0
View file @ 24bffcc8
RecordType
==========
Overview
--------
RecordTypes act as templates for :doc:`Records <Record>`. They
provide a description for a type of Record and define which
:doc:`Properties <AbstractProperty>` could, should, or must be
present. Properties come with an `importance` attribute which tells
the user or client program how strongly necessary the Property is. (As
all other entities,) RecordTypes can inherit from other
RecordTypes (or any Entities). When RecordTypes inherit from other
RecordTypes, the `inheritance` flag tells which properties shall be
inherited from the parents to the children.
Importance
~~~~~~~~~~
The `importance` value of a Property states whether it may, should, or
must be present in all children of this Entity. The table below
lists the possible importances, together with the default behavior of
the server when the property is not present in a child.
.. list-table::
:widths: 25 25 50
:width: 100%
:header-rows: 1
* - Importance
- Meaning
- Consequence when omitted
* - ``OBLIGATORY``
- Property must be present
- Can be forced to be committed the server, but usually
an error will be returned.
* - ``RECOMMENDED``
- Property should be present (makes sense for most users)
- No consequence right now, but in future versions a warning will
be returned.
* - ``SUGGESTED`` (default)
- Property may be present (may make sense for some users)
- No negative consequence
* - ``FIX``
- Property is not present in any child
- ---
The default behavior can be overridden for individual transaction by setting the
`force-missing-obligatory` flag to ``ERROR`` (the default), ``WARN``, or ``IGNORE``. These flags
cause a transaction that would result in an entity with a missing `obligatory` property to return an
error, a warning, or to execute without a warning, respectively. The importance value ``FIX`` is
used for properties that are specific to the Entity itself and is explained below.
.. note::
Note that in contrast to RecordTypes, all properties of a
:doc:`Record` have the importance ``FIX`` since in general, even in the
case of a Record having children, its properties are not
inherited.
Inheritance
~~~~~~~~~~~
The `inheritance` flag specifies which :doc:`Properties
<AbstractProperty>` are inherited from the parent RecordType(s) to the
child.
.. list-table::
:widths: 40 60
:header-rows: 1
* - Inheritance
- Meaning
* - `None` (default)
- Nothing is inherited
* - ``OBLIGATORY``
- Properties of importance ``OBLIGATORY`` and above
* - ``RECOMMENDED``
- Properties of importance ``RECOMMENDED`` and above
* - ``SUGGESTED``
- Properties of importance ``SUGGESTED`` and above
* - ``ALL``
- Copy everything from the parent.
Note that inheritance ``ALL`` is essentially the same as ``SUGGESTED``
right now, but this behavior might change in the future. Also note
that properties of importance ``FIX`` are never inherited this
way. This is precisely what this importance level is meant for:
properties that are specific to the parent entity and are not
inherited by default. You can still add this property to the child
manually if you wish, though, but this is not covered by the
automatic inheritance.
.. note::
The behavior of `inheritance` is currently not well-defined if the
child is a :doc:`Record`, not a RecordType.
Examples
--------
.. warning::
The following examples have not been updated in a while and may
serve as a starting point, but are not guaranteed to be fully
applicable to the most recent versions of CaosDB.
GET Requests
~~~~~~~~~~~~
Single-Get
^^^^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/1
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
</Response>
Multi-Get
^^^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/1&2&3
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="1" name="Experiment" description="Description Experiment">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="2" name="Measurement" description="Description Measurement">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="obligatory" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="obligatory" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="suggested" />
</RecordType>
<RecordType id="3" name="Video" description="Description Video">
<Property id="1" name="explanation" type="text" description="explains the thing" importance="recommended" />
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="12" name="file" type="file" description="file" importance="obligatory" />
</RecordType>
</Response>
Get all
^^^^^^^
*Request:*
.. code-block::
GET http://localhost:8122/RecordType/
GET http://localhost:8122/RecordType
*Response:*
.. note::
Outdated? At least the bug tracker doesn't exist anymore.
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response />
[View Ticket #2](http://caosdb.example.com/caosdb/ticket/2)
POST Requests
~~~~~~~~~~~~~
*Request is to be send to:*
.. code-block::
POST http://localhost:8122/RecordType/
POST http://localhost:8122/RecordType/
Single-Post
^^^^^^^^^^^
*HTTP Body:*
.. code-block:: XML
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
</Post>
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
</RecordType>
</Response>
Multi-Post
^^^^^^^^^^
*HTTP Body:*
.. code-block:: XML
<Post>
<RecordType name="TimmsRecordType11" >
<Property id="3" importance="recommended"/>
<Property id="4" importance="obligatory"/>
<Property name="age" />
</RecordType>
<RecordType name="TimmsRecordType12" >
<Property id="6" importance="recommended"/>
<Property id="7" importance="obligatory"/>
<Property id="15" />
</RecordType>
</Post>
*Response:*
.. code-block:: XML
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<RecordType id="40" name="TimmsRecordType11" description="null">
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" importance="suggested" />
<Property id="3" name="name" type="text" description="Name" importance="recommended" />
<Property id="4" name="id" type="integer" description="identifier" exponent="0" importance="suggested" />
</RecordType>
<RecordType id="41" name="TimmsRecordType12">
<Property id="6" name="startDate" type="datetime" description="start" importance="recommended" />
<Property id="7" name="stopDate" type="datetime" description="stop" importance="suggested" />
<Property id="15" name="TimmsIntProperty" type="integer" unit="kg" description="This is TimmsIntProperty" importance="suggested" />
</RecordType>
</Response>
DELETE RecordTypes
~~~~~~~~~~~~~~~~~~
HTTP-DELETE-requests are to be send to ``http://${host}:${port}/RecordType/...``. Default port is 8123.
*Example*
.. code-block::
DELETE http://${host}:${port}/RecordType/1&2&3&4
src/main/java/org/caosdb/server/entity/StatementStatus.java
+ 17
− 6
View file @ 24bffcc8
...@@ -23,13 +23,24 @@ ...@@ -23,13 +23,24 @@
package org.caosdb.server.entity; package org.caosdb.server.entity;
/** /**
* The statement status has two purposes. * The statement status has two purposes:
* *
* <p>1. Storing the importance of an entity (any of OBLIGATORY, RECOMMENDED, SUGGESTED, or FIX). 2. * <p> 1. Storing the importance of an entity (any of ``OBLIGATORY``,
* Marking an entity as a REPLACEMENT which is needed for flat representation of deeply nested * ``RECOMMENDED``, ``SUGGESTED``, or ``FIX``). ``OBLIGATORY``,
* properties. This constant is only used for internal processes and has no meaning in the API. That * ``RECOMMENDED``, and ``SUGGESTED`` specify whether this Entity
* is also the reason why this enum is not called "Importance". Apart from that, in most cases its * must, should or may be present, and whether the server will throw
* meaning is identical to the importance of an entity. * an error, a warning or nothing, respectively, if it is missing. The
* importance is also used to specify the level of inheritance whereby
* ``FIX`` means that the entity is never inherited. See <a
* href="../../../../../specification/RecordType.html">the
* documentation of RecordTypes</a> for more information.
*
* <p> 2. Marking an entity as a ``REPLACEMENT`` which is needed for
* flat representation of deeply nested properties. This constant is
* only used for internal processes and has no meaning in the
* API. That is also the reason why this enum is not called
* "Importance". Apart from that, in most cases its meaning is
* identical to the importance of an entity.
* *
* @author Timm Fitschen (t.fitschen@indiscale.com) * @author Timm Fitschen (t.fitschen@indiscale.com)
*/ */
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment