WIP: new dump; make data model consistent

README.md

@@ -4,6 +4,68 @@ This module contains the custom code for sample management like it is
 used in GEOMAR's [Biosample Information System
 BIS](https://biosamples.geomar.de/).
 
+## Features
+### Sample Registration
+Samples have unique identifiers (Sample ID) that can be used to persistently
+identify them. The Sample ID can for example be printed as QR Code and used to
+label the physical samples.
+
+Since online access to LinkAhead might not be available during sampling,
+Samples can be registered in advance in order to generate the Sample IDs. This
+allows to print labels for future samples and also allows to download a CSV
+template that can be filled with information about the sample for later import
+into LinkAhead.
+
+Registration is available via the menu in the webinterface "Sample
+Management" -> "Register new Sample"
+
+The CSV Template that is generated always has a set of required columns and
+possibly additional columns that can be selected during the registration. The
+set of columns can be stored for later use via the `RegisterSampleTemplates`.
+Such a template can be created using the "Create sample template" menu entry.
+
+### Sample Update
+Via the menu in the webinterface "Sample Management" -> "Upload Sample template"
+you can upload a CSV template that was created via the registration detailed
+above (or via the export, see below). This updates existing Sample Records
+according to the data contained in the CSV file. This upload possibly creates
+new Records like Events which are related to the updated Samples.
+
+**Important**: Note, that this will override the information of updated samples
+in the server and thus can possibly cause data loss if changes in LinkAhead are
+not reflected in the uploaded file.
+
+### Child Samples
+A Sample can reference another Sample as `Parent sample` to indicate, that the
+sample originated form the other one.  If a sample has such a reference, we
+call it a child sample and the referenced sample a parent sample.
+A lot of information is the same
+for both child and parent Sample and this information is not duplicated but it
+is only referenced by the parent Sample.
+
+This implies that child samples are treated differently thatn parent samples
+during the upload of CSV templates: Some information that is obligatory for
+parent sample are not needed for child samples. There for you can create a
+different CSV template for child samples via the "Register new child sample"
+menu entry.
+
+### Container Registration
+Containers have a unique identifier (Container ID) that can be used to
+persistently identify them. The Container ID can for example be printed as QR
+Code and used to label physical containers.
+
+The "Register new Container" menu entry under "Storage Management" allows to
+create new Container Records with the respective IDs. In this process also
+information on Container Type, Size and parent Containers are incorporated.
+
+### Container Update
+You can upload filled CSV templates to update the information about Containers
+stored in LinkAhead
+
+### Container Export
+You can create a filled CSV template with the information about Containers as
+it is currently stored in LinkAhead.
+
 ## Usage
 
 **TODO**: Add explanations, screenshots, ec. to explain the sample
@@ -59,6 +121,66 @@ arguments and returns the list of post processed samples.
 There is a pre-configured `test-profile` that you can use for testing
 this repository.
 
+
+## Data Model
+This module requires a certain data model in order to work. However, the exact
+names of RecordTypes and Properties can be configured. Below, we will use the
+default names but also reference the variable names of `constants.yml` that
+allow to change those.
+
+### Sample (`variable missing`)
+This is the central RecordType for the sample management and used to represent
+samples of various types
+
+#### `Parent_Sample` (`$parent_sample_prop`)
+references the sample from which the sample at hand originated (for example by
+cutting the parent sample into multiple pieces0
+
+#### `Main User`
+references the person that mainly works with this sample
+
+#### `PI`
+references the person that is the responsible principle investigator (PI)
+
+### Event (`$event_rt`)
+A sampling event. Each sample reference the event where it was created.
+
+#### Position (`var missing`)
+The position (e.g. of an Event) that is defined via longituted, latitude and
+possibly elevation
+#### "Biome"
+#### "Campaign"
+#### "Device"
+#### "EventType"
+#### "igsn_doi_prop"
+#### "level"
+#### "locality_description_prop"
+#### "locality_name_prop"
+#### "Sphere"
+
+### Person (`variable missing`)
+a natural person
+
+#### `first_name`  (`$first_name_prop`)
+#### `last_name`  (`$last_name_prop`)
+#### `abbreviation`  (`$abbreviation_prop`)
+
+### `Container` (`$container_rt`)
+a physical container that contains Samples and/or other Containers
+
+####`Child_Container`  (`$child_container_prop`)
+references Containers that are contained in the Container at hand
+
+#### `container_label`  (`$container_label_prop`)
+#### `Container size`  (`$container_size_prop`)
+#### `ContainerType`  (`$containertype_rt`)
+#### `custom_label` (`$custom_label_prop`)
+
+#### `Responsible_Person` (`$responsible_person_event`)
+references the person that is responsilbe for this sample, e.g. the PI
+
+### `RegisterSampleTemplates` (missing var)
+
 ## Further Reading
 
 Please refer to the [official documentation](https://docs.indiscale.com/caosdb-deploy/) for more information.

models_and_helper_scripts/sample-management-datamodel.yml

@@ -48,8 +48,9 @@ ContainerType:
     Abbreviation:
 
 ConditionType:
-    inherit_from_recommended:
-      - ControlledRecordtype
+    inherit_from_obligatory:
+      - ControlledRecordType
+
 StorageCondition:
   recommended_properties:
     ConditionType:
@@ -64,7 +65,7 @@ Container:
     container_label:
       description: A standardized label for a container
       datatype: TEXT
-    Child container:
+    Child_Container:
       datatype: LIST<Container>
     Container size:
       description: The size of the container, e.g. number of spaces or dimension
@@ -82,7 +83,7 @@ Sample:
   recommended_properties:
     Container:
     SampleType:
-    Parent sample:
+    Parent_Sample:
       datatype: Sample
     Main User:
       datatype: Person
@@ -149,12 +150,12 @@ EventType:
     - ControlledRecordType
 
 Biome:
-  inherit_from_recommended:
-    - ControlledRecordtype
+  inherit_from_obligatory:
+    - ControlledRecordType
 
 Device:
-  inherit_from_recommended:
-    - ControlledRecordtype
+  inherit_from_obligatory:
+    - ControlledRecordType
 
 Event:
   recommended_properties:

sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_sample_reference_resolver.js

 var sample_management_reference_resolver = new function (getEntityID, getEntityName, getParents, getProperty, query) {
+    // TODO note explicitly what data model assumptions are made here
 
     const lastname_prop_name = "last_name";
     const firstname_prop_name = "first_name";

sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_column_definitions.py

@@ -71,7 +71,7 @@ DATATYPE_DEFINITIONS = use_custom_names({
     "Parent LinkAhead ID": str,
     "SampleMethod": str,
     "SampleType": str,
-    "Start Date": str,
+    "Start date": str,
     "Storage ID": str,
 })
 

sample-management-custom/caosdb-server/scripting/bin/upload_sample_template.py

@@ -156,19 +156,20 @@ def main():
             property_name = eprop.name
             if property_name in SPECIAL_TREATMENT_SAMPLE:
                 continue
-            if db.apiutils.is_reference(eprop):
-                rt = db.get_entity_by_id(eprop.id)
-                if len(rt.properties) == 1:
-                    converter = _get_converter_from_property_datatype(rt.properties[0].datatype)
-                elif len(rt.properties) == 1:
-                    converter = str
+            if property_name not in DATATYPE_DEFINITIONS:
+                if db.apiutils.is_reference(eprop):
+                    rt = db.get_entity_by_id(eprop.id)
+                    if len(rt.properties) == 1:
+                        converter = _get_converter_from_property_datatype(rt.properties[0].datatype)
+                    elif len(rt.properties) < 1:
+                        converter = str
+                    else:
+                        converter = None
                 else:
-                    converter = None
-            else:
-                converter = _get_converter_from_property_datatype(eprop.datatype)
-            if converter is None:
-                continue
-            DATATYPE_DEFINITIONS[property_name] = converter
+                    converter = _get_converter_from_property_datatype(eprop.datatype)
+                if converter is None:
+                    continue
+                DATATYPE_DEFINITIONS[property_name] = converter
             if sample.get_importance(property_name) == db.OBLIGATORY:
                 # This is only needed if the sample is not a child sample
                 OBLIGATORY_COLUMNS.append(property_name)