diff --git a/README.md b/README.md index a91b7e5edc584bd827104c04e621f2e456baf401..71abb5a53d47f565f816196d238ed9b207ed3dd2 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,81 @@ This module contains the custom code for sample management like it is used in GEOMAR's [Biosample Information System BIS](https://biosamples.geomar.de/). +## Features +### Sample Registration +Samples have unique identifiers (Sample ID) that can be used to persistently +identify them. The Sample ID can for example be printed as QR Code and used to +label the physical samples. + +Since online access to LinkAhead might not be available during sampling, +Samples can be registered in advance in order to generate the Sample IDs. This +allows to print labels for future samples and also allows to download a CSV +template that can be filled with information about the sample for later import +into LinkAhead. + +Registration is available via the menu in the webinterface "Sample +Management" -> "Register new Sample" + +The CSV Template that is generated always has a set of required columns and +possibly additional columns that can be selected during the registration. The +set of columns can be stored for later use via the `RegisterSampleTemplates`. +Such a template can be created using the "Create sample template" menu entry. + +### Sample Update +Via the menu in the webinterface "Sample Management" -> "Upload Sample template" +you can upload a CSV template that was created via the registration detailed +above (or via the export, see below). This updates existing Sample Records +according to the data contained in the CSV file. This upload possibly creates +new Records like Events which are related to the updated Samples. + +**Important**: Note, that this will override the information of updated samples +in the server and thus can possibly cause data loss if changes in LinkAhead are +not reflected in the uploaded file. + +### Child Samples +A Sample can reference another Sample as `Parent sample` to indicate, that the +sample originated form the other one. If a sample has such a reference, we +call it a child sample and the referenced sample a parent sample. +A lot of information is the same +for both child and parent Sample and this information is not duplicated but it +is only referenced by the parent Sample. + +This implies that child samples are treated differently thatn parent samples +during the upload of CSV templates: Some information that is obligatory for +parent sample are not needed for child samples. There for you can create a +different CSV template for child samples via the "Register new child sample" +menu entry. + +### Container Registration +Containers have a unique identifier (Container ID) that can be used to +persistently identify them. The Container ID can for example be printed as QR +Code and used to label physical containers. + +The "Register new Container" menu entry under "Storage Management" allows to +create new Container Records with the respective IDs. In this process also +information on Container Type, Size and parent Containers are incorporated. + +### Container Update +You can upload filled CSV templates to update the information about Containers +stored in LinkAhead + +### Container Export +You can create a filled CSV template with the information about Containers as +it is currently stored in LinkAhead. + +### Extensibility +The registration, update and export does not only work with RecordTypes and +Properties that are defined within this module, but the data model can be +extended by users of the system: Any Property that is added to the Sample +RecordType can also be used within the workflow. Once an additional Property +was added to the RecordType (say "width"), you can select a corresponding +column during registration with the same name and also upload CSV templates +that have this column. The Sample records will be created or updated to also +have this ("width") column. + +If the Property that is added is a RecordType, it must be kind of an enum, i.e. +it should be uniquely identified either by its name or by a sole property. + ## Usage **TODO**: Add explanations, screenshots, ec. to explain the sample @@ -29,11 +104,96 @@ instance's data model (e.g., by [synchronizing with the LinkAhead advanced user tools](https://docs.indiscale.com//caosdb-advanced-user-tools/yaml_interface.html)). +## Customization + +The sample management module can be customized by overwriting +indivdual code and config files by using files of the same name and +with the same relative path in your LinkAhead profile's custom +directory. E.g., if you want to overwrite how sample entities are post-processed in the sample upload, you overwrite + +``` +sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_post_processing.py +``` + +from this repository by a + +``` +profile/custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_post_processing.py +``` + +in your server profile. When doing so, it is important that in your +custom file, you use the exact same function names, signatures, and +return types as in the original file. In our example, the custom +`sample_upload_post_processing.py` needs to contain a +`post_process_samples` that takes the samples and the csv data as +arguments and returns the list of post processed samples. + + ## Testing There is a pre-configured `test-profile` that you can use for testing this repository. + +## Data Model +This module requires a certain data model in order to work. However, the exact +names of RecordTypes and Properties can be configured. Below, we will use the +default names but also reference the variable names of `constants.yml` that +allow to change those. + +### Sample (`variable missing`) +This is the central RecordType for the sample management and used to represent +samples of various types + +#### `Parent_sample` (`$parent_sample_prop`) +references the sample from which the sample at hand originated (for example by +cutting the parent sample into multiple pieces0 + +#### `Main User` +references the person that mainly works with this sample + +#### `PI` +references the person that is the responsible principle investigator (PI) + +### Event (`$event_rt`) +A sampling event. Each sample reference the event where it was created. + +#### Position (`var missing`) +The position (e.g. of an Event) that is defined via longituted, latitude and +possibly elevation +#### "Biome" +#### "Campaign" +#### "Device" +#### "EventType" +#### "igsn_doi_prop" +#### "level" +#### "locality_description_prop" +#### "locality_name_prop" +#### "Sphere" + +### Person (`variable missing`) +a natural person + +#### `first_name` (`$first_name_prop`) +#### `last_name` (`$last_name_prop`) +#### `abbreviation` (`$abbreviation_prop`) + +### `Container` (`$container_rt`) +a physical container that contains Samples and/or other Containers + +####`Child_Container` (`$child_container_prop`) +references Containers that are contained in the Container at hand + +#### `container_label` (`$container_label_prop`) +#### `Container size` (`$container_size_prop`) +#### `ContainerType` (`$containertype_rt`) +#### `custom_label` (`$custom_label_prop`) + +#### `Responsible_Person` (`$responsible_person_event`) +references the person that is responsilbe for this sample, e.g. the PI + +### `RegisterSampleTemplates` (missing var) + ## Further Reading Please refer to the [official documentation](https://docs.indiscale.com/caosdb-deploy/) for more information. diff --git a/models_and_helper_scripts/sample-management-datamodel.yml b/models_and_helper_scripts/sample-management-datamodel.yml index f8deb9d90206c9a5b37690b233aa9ce35d6f52e6..aef3b68183e47b862b3189622822247dcb8c3ee3 100644 --- a/models_and_helper_scripts/sample-management-datamodel.yml +++ b/models_and_helper_scripts/sample-management-datamodel.yml @@ -1,7 +1,6 @@ # -# -# Copyright (C) 2024 IndiScale <info@indiscale.com> -# Copyright (C) 2024 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# Copyright (C) 2024-2025 IndiScale <info@indiscale.com> +# Copyright (C) 2024-2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as @@ -26,34 +25,21 @@ ControlledRecordType: Responsible: inherit_from_obligatory: - ControlledRecordType - obligatory_properties: + recommended_properties: Abbreviation: - description: An abbreviation for this entity that is used for creating BIS labels - datatype: TEXT -ResearchDivision: - inherit_from_obligatory: - - Responsible - obligatory_properties: - German name: - description: A german name for an entitiy. This can be used in queries since it is an alias. + description: An abbreviation for this entity that is used for identification purposes datatype: TEXT -ResearchUnit: - inherit_from_obligatory: - - Responsible - obligatory_properties: - German name: - ResearchDivision: Person: - inherit_from_obligatory: + inherit_from_recommended: - Responsible obligatory_properties: - First name: + first_name: datatype: TEXT - Last name: + last_name: datatype: TEXT + recommended_properties: Email: datatype: TEXT - ResearchUnit: ContainerType: inherit_from_obligatory: @@ -61,63 +47,59 @@ ContainerType: obligatory_properties: Abbreviation: +ConditionType: + inherit_from_obligatory: + - ControlledRecordType + +StorageCondition: + recommended_properties: + ConditionType: + condition_value: + datatype: TEXT + Container: obligatory_properties: Responsible: ContainerType: - BIS Label: - description: A standardized label for a container in BIS. - datatype: TEXT suggested_properties: - Child container: + container_label: + description: A standardized label for a container + datatype: TEXT + Child_Container: datatype: LIST<Container> Container size: description: The size of the container, e.g. number of spaces or dimension datatype: TEXT - Custom label: - description: A custom label for a container that doesn't have to follow the BIS labeling guidelines. + custom_label: + description: A custom label for a container that doesn't have to follow the automatic labeling guidelines. datatype: TEXT Container Contents: datatype: TEXT PI: datatype: Person + StorageCondition: Sample: recommended_properties: - NagoyaCase: Container: - FunctionalGroup: SampleType: - Parent sample: + Parent_sample: datatype: Sample - Sampling Person: - datatype: Person Main User: datatype: Person - suggested_properties: - Custom ID: - datatype: LIST<TEXT> - Publications: - description: DOIs of publications related to this sample. - datatype: LIST<TEXT> - NCBI BioProject: - datatype: LIST<TEXT> - NCBI BioSample: - datatype: LIST<TEXT> - NCBI Accession: - datatype: LIST<TEXT> - Sampling method: - description: A short description of the sampling methodology used for this sample. - datatype: TEXT - Notes: + Collector: + datatype: LIST(Person) + Curator: + datatype: LIST(Person) + Device: + material_left: + datatype: BOOLEAN + Event: + SampleMethod: + nagoya_link: datatype: TEXT - Time sampled start: - datatype: DATETIME - Time sampled stop: - datatype: DATETIME - AphiaID: - description: An ID for this taxon that allows to uniquely identify it on http://www.marinespecies.org. - datatype: INTEGER + nagoya_relevance: + datatype: BOOLEAN Collection: @@ -154,65 +136,54 @@ NagoyaCase: Position: obligatory_properties: - Latitude: + latitude: datatype: DOUBLE unit: ° - Longitude: + longitude: datatype: DOUBLE unit: ° - recommended_properties: - Water depth: - description: The depth of the water in meters as a positive value - datatype: DOUBLE - unit: m - Sampling depth: - description: The depth in meters where the sampling took place as a positive value + elevation: datatype: DOUBLE - unit: m -FunctionalGroup: +EventType: inherit_from_obligatory: - ControlledRecordType -Gear: +Biome: + inherit_from_obligatory: + - ControlledRecordType + +Device: inherit_from_obligatory: - ControlledRecordType - recommended_properties: - Configuration: - datatype: TEXT Event: - obligatory_properties: - Gear: recommended_properties: - Time start: + Parent_Event: + datatype: Event + Biome: + Campaign: + Device: + Position: + datatype: LIST(Position) + start_date: description: The date and time the event started. datatype: DATETIME - suggested_properties: - Time stop: + end_date: description: The date and time the event ended. datatype: DATETIME - -SourceEvent: - inherit_from_suggested: - - Event - obligatory_properties: - Position: - datatype: LIST<Position> - Gear: - recommended_properties: - Hol: + EventType: + igsn_doi: datatype: TEXT - Station Number: + level: datatype: TEXT - IGSN: + locality_description: datatype: TEXT - DOI: - datatype: TEXT - suggested_properties: - Station ID: - description: The name/identifier of a station where regular sampling activity takes place. + locality_name: datatype: TEXT + Sphere: + Responsible_Person: + datatype: LIST(Person) Platform: inherit_from_obligatory: @@ -222,9 +193,10 @@ SampleType: inherit_from_obligatory: - ControlledRecordType -SampleTypeSpecific: - inherit_from_obligatory: - - ControlledRecordType +SampleMethod: + recommended_properties: + Parent_SampleMethod: + datatype: SampleMethod StartPosition: inherit_from_obligatory: @@ -239,25 +211,6 @@ RegisterSampleTemplate: column_names: datatype: LIST<TEXT> -MeshPoreSize: - inherit_from_obligatory: - - ControlledRecordType - recommended_properties: - enumValue: - datatype: TEXT - -Fixation: - inherit_from_obligatory: - - ControlledRecordType - recommended_properties: - enumValue: - -StorageTemperature: - inherit_from_obligatory: - - ControlledRecordType - recommended_properties: - enumValue: - Campaign: inherit_from_obligatory: - ControlledRecordType @@ -267,27 +220,7 @@ Sphere: inherit_from_obligatory: - ControlledRecordType -Sample context: - inherit_from_obligatory: - - ControlledRecordType - PDFReport: inherit_from_obligatory: - ControlledRecordType description: PDF files which contain reports and additional information about samples, collections, or containers - -commonGermanName: - inherit_from_obligatory: - - ControlledRecordType - -commonEnglishName: - inherit_from_obligatory: - - ControlledRecordType - -Taxonomic group/Species: - inherit_from_obligatory: - - ControlledRecordType - -Sample container: - inherit_from_obligatory: - - ControlledRecordType diff --git a/sample-management-custom/caosdb-server/caosdb-webui/build.properties.d/50.sample-management.conf b/sample-management-custom/caosdb-server/caosdb-webui/build.properties.d/50.sample-management.conf index 185d75ef202aff0b38c806fec8f0a3654be04931..6727bd1ff64c2dbb8add5a605ba9ed0a8f6e0d8a 100644 --- a/sample-management-custom/caosdb-server/caosdb-webui/build.properties.d/50.sample-management.conf +++ b/sample-management-custom/caosdb-server/caosdb-webui/build.properties.d/50.sample-management.conf @@ -22,6 +22,8 @@ BUILD_MODULE_EXT_SSS_MARKDOWN=ENABLED # Showing id in labels helps for the container/sample id columns BUILD_MODULE_SHOW_ID_IN_LABEL=ENABLED -# BIS specific variables -BUILD_MODULE_EXT_BIS_SAMPLEMANAGEMENT=ENABLED -BUILD_MODULE_EXT_BIS_STOCKMANAGEMENT=ENABLED +# sample-management specific variables +BUILD_MODULE_EXT_SAMPLEMANAGEMENT=ENABLED +BUILD_MODULE_EXT_STOCKMANAGEMENT=ENABLED +BUILD_MODULE_EXT_SAMPLEMANGEMENT_ID_LABEL="Sample ID" +BUILD_MODULE_EXT_STOCKMANAGEMENT_ID_LABEL="Container ID" \ No newline at end of file diff --git a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/bis_custom_reference_resolver.js b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_sample_reference_resolver.js similarity index 92% rename from sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/bis_custom_reference_resolver.js rename to sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_sample_reference_resolver.js index 780a0c20983149530d8cbfdcdba34c06c9f763d2..d237d621d07fd54088234b80530471375b2447c0 100644 --- a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/bis_custom_reference_resolver.js +++ b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_sample_reference_resolver.js @@ -1,7 +1,8 @@ -var bis_custom_reference_resolver = new function (getEntityID, getEntityName, getParents, getProperty, query) { +var sample_management_reference_resolver = new function (getEntityID, getEntityName, getParents, getProperty, query) { + // TODO note explicitly what data model assumptions are made here - const lastname_prop_name = "Last name"; - const firstname_prop_name = "First name"; + const lastname_prop_name = "last_name"; + const firstname_prop_name = "first_name"; const person_rt_name = "Person"; const bis_label_prop_name = "BIS label"; const custom_label_prop_name = "Custom label" diff --git a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_samplemanagemet.js b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_samplemanagement.js similarity index 87% rename from sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_samplemanagemet.js rename to sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_samplemanagement.js index 4228d6423c37ec8d71d7608448fa4361de281c3e..e8e46e0d88ac2cc40b4fe89fe5032ecbbdd3cb82 100644 --- a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_samplemanagemet.js +++ b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_samplemanagement.js @@ -6,9 +6,9 @@ 'use strict'; /** - * This module adds to forms to BIS that relate to sample management. - * One form allows to register samples. - * The other allows to update registered samples. + * This module adds to forms to LinkAhead that relate to sample + * management; there are forms for sample registration, upload, and + * export. * * @module ext_flight_preview * @version 0.1 @@ -16,7 +16,7 @@ * @requires nothing yet * (pass the dependencies as arguments) */ -const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_panel, bis_custom_reference_resolver) { +const ext_samplemanagement = function($, navbar, log, form_elements, form_panel, sample_management_reference_resolver) { const logger = log.getLogger("samplemanagement_form"); const tool_box = "Sample Management" const upload_sample_template_title = "Upload sample template"; // title of the form and text in the toolbox @@ -29,6 +29,7 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa const upload_pdf_title = "Upload PDF file"; const export_csv_id = "export_sample_csv"; const export_csv_title = "Export Samples to CSV"; + const sample_id_label = ("${BUILD_MODULE_EXT_SAMPLEMANGEMENT_ID_LABEL}" != "") ? "${BUILD_MODULE_EXT_SAMPLEMANGEMENT_ID_LABEL}" : "Sample ID"; const accepted_file_formats = [ ".csv", ".CSV", @@ -36,85 +37,47 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa "text/csv", ]; const required_column_names = [ - "BIS ID", - "Date collected start", + "LinkAhead ID", "Main User", - "Gear", - "Gear configuration", "Latitude start", "Longitude start", - "Collection", - "PI", + "Elevation start", + "Start date", "Storage ID", - "Nagoya case number", - "Sample container", - "Fixation", - "StorageTemperature", - "SampleType", - "SampleTypeSpecific", - "Sphere", - "Sample Context", + "Device" ]; const requiredColumnNamesChildren = [ - "BIS ID", - "Date sampled start", - "Fixation", + "LinkAhead ID", "Main User", - "Parent BIS ID", - "Sample Context", - "Sample container", + "Parent LinkAhead ID", "SampleType", - "SampleTypeSpecific", - "Sphere", "Storage ID", - "StorageTemperature", ]; const non_sample_rt_column_names = [ - "Platform", + "Biome", "Campaign", - "Station ID", - "Station number", - "Subevent", - "Time collected start", - "Date collected stop", - "Time collected stop", - "Timezone", - "Hol", - "Sampling depth start", - "Sampling depth stop", - "Water depth start", - "Water depth stop", + "Start date", + "End date", + "IGSN DOI", + "Level", + "Locality description", + "Locality name", + "Sphere", + "Event responsible", "Latitude stop", "Longitude stop", - "Taxonomic group/Species", - "AphiaID", - "On-board sampleID", - "Extraction ID", - "Duplicate number", - "Other fixations", - "Notes / remarks", - "Forlmaldehyde_box", - "OSIS URL", - "Ecotaxa URL", + "Elevation stop", "PDFReport", - "Date sampled start", - "Date sampled stop", - "Time sampled start", - "Time sampled stop", - "Parent BIS ID" - ] + "Parent LinkAhead ID", + ]; const allColumnNames = non_sample_rt_column_names.concat(requiredColumnNamesChildren).concat(required_column_names); const unused_property_names = [ - 'Container', - 'Event', "Main User", + "Event", + "Container", 'NagoyaCase', - "Parent Sample", - "Person", - "SampleType", - "Time start", - "Time stop" + "Parent_sample", ] const upload_sample_template_form_config = { @@ -134,12 +97,12 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa script: "export_sample_csv.py", fields: [{ type: "text", - name: "bis_ids", - label: "BIS IDS", + name: "ids", + label: sample_id_label, required: false, pattern: "((([0-9]+\\s*-\\s*){0,1}[0-9]+)\\s*,\\s*)*(([0-9]+\\s*-\\s*){0,1}[0-9]+)", cached: false, - help: "Comma-separated list of single BIS IDs or ranges, e.g. '124, 126-139, 242'", + help: `Comma-separated list of single ${sample_id_label}s or ranges, e.g. '124, 126-139, 242'`, }, { type: "text", name: "query_string", @@ -181,7 +144,7 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa name: "responsible_person", label: "Main user", query: "FIND RECORD Person", - make_desc: bis_custom_reference_resolver.resolve_person_reference, + make_desc: sample_management_reference_resolver.resolve_person_reference, required: true }, { @@ -262,7 +225,7 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa name: "main_user", label: "Main user", query: "FIND RECORD Person", - make_desc: bis_custom_reference_resolver.resolve_person_reference, + make_desc: sample_management_reference_resolver.resolve_person_reference, required: true }, { type: "select", @@ -398,7 +361,7 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa }); }; - + const is_authorized = function() { // return true //for testing return isAuthenticated() //&& (userHasRole("stockmanager") || userHasRole("administration")) @@ -416,14 +379,14 @@ const ext_bis_samplemanagement = function($, navbar, log, form_elements, form_pa return { init: init, }; -}($, navbar, log, form_elements, form_panel, bis_custom_reference_resolver); +}($, navbar, log, form_elements, form_panel, sample_management_reference_resolver); // this will be replaced by require.js in the future. $(document).ready(function() { // use a variable starting with `BUILD_MODULE_` to enable your module // the build variable has to be enabled in the `build.properties.d/` directory. // Otherwise the module will not be activated. - if ("${BUILD_MODULE_EXT_BIS_SAMPLEMANAGEMENT}" === "ENABLED") { - caosdb_modules.register(ext_bis_samplemanagement); + if ("${BUILD_MODULE_EXT_SAMPLEMANAGEMENT}" === "ENABLED") { + caosdb_modules.register(ext_samplemanagement); } }); diff --git a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_stockmanagement.js b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_stockmanagement.js similarity index 84% rename from sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_stockmanagement.js rename to sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_stockmanagement.js index b8630a40836d7bc971ffe749af898d0f583cebf1..4775b9dff5327cf026106a530da05023f0858f8d 100644 --- a/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_bis_stockmanagement.js +++ b/sample-management-custom/caosdb-server/caosdb-webui/src/ext/js/ext_stockmanagement.js @@ -6,18 +6,19 @@ 'use strict'; /** - * @module ext_bis_stockmanagement + * @module ext_stockmanagement * @version 0.2 * - * @requires($, navbar, log, form_elements, form_panel, bis_custom_reference_resolver) + * @requires($, navbar, log, form_elements, form_panel, sample_management_reference_resolver) */ -const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_panel, bis_custom_reference_resolver) { +const ext_stockmanagement = function($, navbar, log, form_elements, form_panel, sample_management_reference_resolver) { const tool_box = "Storage Management" // The item that is shown in the top-navbar ///////////////// REGISTER NEW CONTAINERS START const register_new_containers_title = "Register new Containers"; const register_new_containers_panel_id = "register_new_containers_form_panel"; + const container_id_label = ("${BUILD_MODULE_EXT_STOCKMANAGEMENT_ID_LABEL}" != "") ? "${BUILD_MODULE_EXT_STOCKMANAGEMENT_ID_LABEL}" : "Container ID"; const register_new_containers_form_config = { // the script can be found in profiles/default/custom/caosdb-server/scripting/bin/ // it is called like ./register_new_containers.py form.json where form.json @@ -34,7 +35,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa name: "responsible_entity", label: "Responsible entity", query: "FIND RECORD Responsible", - make_desc: getEntityName, + make_desc: sample_management_reference_resolver.resolve_person_reference, required: true }, { @@ -76,7 +77,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa name: "parent_id", label: "Parent container", query: "FIND RECORD Container", - make_desc: bis_custom_reference_resolver.resolve_default, + make_desc: sample_management_reference_resolver.resolve_default, required: false }, ], @@ -114,12 +115,12 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa script: "export_container_csv.py", fields: [{ type: "text", - name: "bis_ids", - label: "BIS IDS", + name: "container_ids", + label: container_id_label, required: true, pattern: "((([0-9]+\\s*-\\s*){0,1}[0-9]+)\\s*,\\s*)*(([0-9]+\\s*-\\s*){0,1}[0-9]+)", cached: false, - help: "Comma-separated list of single BIS IDs or ranges, e.g. '124, 126-139, 242'" + help: `Comma-separated list of single ${container_id_label}s or ranges, e.g. '124, 126-139, 242'` }] }; ///////////////// EXPORT CONTAINER CSV END @@ -130,7 +131,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa * dropdown for Container registration and for uploading filled-in container * forms. */ - const init_show_stockmanagement_panel_buttons = function () { + const init_show_stockmanagement_panel_buttons = function() { create_register_new_containers_button(); create_update_containers_button(); @@ -159,7 +160,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa }); }; - const create_register_new_containers_button = function () { + const create_register_new_containers_button = function() { navbar.add_tool(register_new_containers_title, tool_box, { callback: form_panel.create_show_form_callback( register_new_containers_panel_id, @@ -168,7 +169,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa }); }; - const create_update_containers_button = function () { + const create_update_containers_button = function() { navbar.add_tool(update_containers_title, tool_box, { callback: form_panel.create_show_form_callback( update_containers_panel_id, @@ -183,7 +184,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa * Check if the user is authenticated and has at least stockmanager or * administrator role. */ - const is_authorized = function () { + const is_authorized = function() { return isAuthenticated() && (userHasRole("Scientist") || userHasRole("Stock manager") || userHasRole("administration") || userHasRole("Curator")); } @@ -192,7 +193,7 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa * top-navbar is_authorized() returns true, i.e. the user is authenticated * and has either stockmanager or administration role. */ - const init = function () { + const init = function() { if (is_authorized()) { init_show_stockmanagement_panel_buttons(); } @@ -202,14 +203,14 @@ const ext_bis_stockmanagement = function ($, navbar, log, form_elements, form_pa return { init: init, }; -}($, navbar, log, form_elements, form_panel, bis_custom_reference_resolver); // this will be replaced by require.js in the future. +}($, navbar, log, form_elements, form_panel, sample_management_reference_resolver); // this will be replaced by require.js in the future. -$(document).ready(function () { +$(document).ready(function() { // use a variable starting with `BUILD_MODULE_` to this module // the build variable has to be enabled in the `build.properties.d/` directory. // Otherwise the module will not be activated. - if ("${BUILD_MODULE_EXT_BIS_STOCKMANAGEMENT}" === "ENABLED") { - caosdb_modules.register(ext_bis_stockmanagement); + if ("${BUILD_MODULE_EXT_STOCKMANAGEMENT}" === "ENABLED") { + caosdb_modules.register(ext_stockmanagement); } }); // const form = form_elements.make_form(config); diff --git a/sample-management-custom/caosdb-server/scripting/bin/bis_utils.py b/sample-management-custom/caosdb-server/scripting/bin/bis_utils.py index d70503062881b0df7701fff1d7e6b6fc936a6921..00699e559a62b6cbfb208f144ba83f0cdf078d01 100644 --- a/sample-management-custom/caosdb-server/scripting/bin/bis_utils.py +++ b/sample-management-custom/caosdb-server/scripting/bin/bis_utils.py @@ -32,84 +32,12 @@ from caosadvancedtools.serverside.helper import send_mail from caoscrawler.config import get_config_setting from linkahead import get_entity_by_name -SPECIAL_TREATMENT_SAMPLE = [ - "BIS ID", - "Collection", - "Date collected start", - "Date collected stop", - "Date sampled start", - "Date sampled stop", - "Gear configuration", - "Gear", - "Hol", - "Latitude start", - "Latitude stop", - "Longitude start", - "Longitude stop", - "Main User", - "Nagoya case number", - "PDFReport", - "PI", - "Parent BIS ID", - "Person", - "Sampling Person", - "Sampling depth start", - "Sampling depth stop", - "Sphere", - "Station ID", - "Station number", - "Storage Container Label", - "Storage ID", - "Subevent", - "Time collected start", - "Time collected stop", - "Time sampled start", - "Time sampled stop", - "Timezone", - "Water depth start", - "Water depth stop", -] - -IGNORED_COLUMN_NAMES_SAMPLE = [ - "BIS URL", - "Date", - "IGSN URL", - "IGSN", # TODO This will be relevant for external IGSNs in the future. - "Parent Sample", - "Sampling depth", - "Storage chain", - "Water depth", -] - - -# This can be used for hard-coded descriptions -COLUMN_DESCRIPTIONS = { - "BIS ID": "An ID generated by the BIS (either integer or URL to this entity). Do not change this column!", - "BIS label": "The label automatically assigned by the BIS. Do not change this column.", - "Collection": "Collection(s) to which this container or samples belongs (separated by ';')", - "Container Contents": "Notes concerning the contents of this container", - "Container size": "The size of the container (e.g 81 spaces)", - "Container type": "The type of container. Do not change this column.", - "Custom label": "Any additional custom label you put on the container", - "Date collected start": "The start date in format YYYY-MM-DD", - "Date collected stop": "The end date in format YYYY-MM-DD", - "Gear configuration": "How gear is configured, meshsize of the net, filter pore size, what kind of trawl", - "Latitude start": "Latitude where sampling started; in decimal degrees,use \".\" as decimal sign, use - for S", - "Latitude stop": "Latitude where sampling ended; in decimal degrees,use \".\" as decimal sign, use - for S", - "Longitude start": "Longitude where sampling started; in decimal degrees,use \".\" as decimal sign, use - for W", - "Longitude stop": "Longitude where sampling ended; in decimal degrees,use \".\" as decimal sign, use - for W", - "Notes / remarks": "Field for notes", - "PDFReport": "A PDF containing additional information on the container or sampe contents", - "PI": "The PI (by abbreviation) of the container or sample", - "Parent container": "The BIS ID (integer or URL) or name or BIS label of the container where this container is stored.", - "Sampling depth start": "The depth in meters where the sampling started as a positive value, not bottom depth", - "Sampling depth stop": "The depth in meters where the sampling ended as a positive value,not bottom depth", - "Storage ID": "BIS ID of the Container that holds this sample", - "Subevent": "Subevent given by D-Ship in Stationlog. Expl. AL123_14-4", - "Timezone": "Timezone: Either UTC, CET, .... or in the form +hh[:mm], -hh:[mm].", - "Water depth start": "The bottom depth in meters where the sampling started as a positive value", - "Water depth stop": "The bottom depth in meters where the sampling ended as a positive value", -} +from sample_helpers.sample_upload_column_definitions import ( + IGNORED_COLUMN_NAMES_SAMPLE, SPECIAL_TREATMENT_SAMPLE) +from sample_helpers.utils import CONSTANTS, return_value_if_not_none + + +COLUMN_DESCRIPTIONS = CONSTANTS["csv_column_descriptions"] def get_do_not_insert_type_names(override_names: list[str] = []): @@ -203,22 +131,6 @@ def replace_entity_urls_by_ids(data: pd.DataFrame, eid_columns: list[str] = ["BI return data -def return_value_if_not_none(val): - """Workaround for somewhat weird pandas behavior. Return value if actual - value, otherwise, return None. - - """ - if isinstance(val, list): - if len(val) == 0: - return None - if len(val) == 1: - if pd.isnull(val[0]) or val[0] is None or f"{val[0]}".lower() == "nan" or f"{val[0]}" == "": - return None - elif pd.isnull(val) or val is None or f"{val}".lower() == "nan" or f"{val}" == "": - return None - return val - - def whitespace_cleanup_in_df(df: pd.DataFrame): """Strip all leading and trailing whitespaces from all str values in df.""" diff --git a/sample-management-custom/caosdb-server/scripting/bin/crawl_sample_data_async.py b/sample-management-custom/caosdb-server/scripting/bin/crawl_sample_data_async.py index 1c578f9e6e6b56c53805857996b13b90f4199e4b..ace23eef4c481fdc20057fce83ef44d4c45dbca5 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/crawl_sample_data_async.py +++ b/sample-management-custom/caosdb-server/scripting/bin/crawl_sample_data_async.py @@ -47,6 +47,11 @@ from bis_utils import (get_do_not_insert_type_names, return_value_if_not_none, send_mail_with_defaults, SPECIAL_TREATMENT_SAMPLE) +from sample_helpers.sample_upload_add_special_properties import add_special_properties +from sample_helpers.sample_upload_get_event import add_event_to_sample +from sample_helpers.sample_upload_get_person import get_person +from sample_helpers.sample_upload_post_processing import post_process_samples +from sample_helpers.utils import (get_column_header_name, get_entity_name, update_property) # suppress warning of diff function apilogger = logging.getLogger("linkahead.apiutils") @@ -66,373 +71,6 @@ def _is_ignored_column_name(name, parent_suffix="_parent"): return name in IGNORED_COLUMN_NAMES_SAMPLE or name.endswith(parent_suffix) -def _update_property(entity: db.Record, property_id: int, value, property_name="", datatype=None): - """ - Set the property of an entity. - - If the entity already has the property, just the value is set. - Else, the property is added to the entity - """ - # TODO: Replace by assure_property_is etc. - - # If the value in the spreadsheet is empty (nan) - if ((isinstance(value, list) and len(value) == 0) - or (not isinstance(value, list) and pd.isna(value))): - # Remove the property from te Entity if it has it - try: - entity.get_properties().get_by_name(property_name) - entity.remove_property(property_name) - except KeyError: - pass - return entity - if entity.get_property(property_id) is None: - if datatype: - entity.add_property(id=property_id, value=value, name=property_name, datatype=datatype) - else: - entity.add_property(id=property_id, value=value, name=property_name) - logger.debug("{}: Adding {} = {}".format(entity.id, property_id, value.id if - isinstance(value, db.Entity) else value)) - else: - if isinstance(value, list) and not entity.get_property(property_id).datatype.startswith("LIST"): - entity.get_property(property_id).datatype = db.LIST( - entity.get_property(property_id).datatype) - entity.get_property(property_id).value = value - logger.debug("{}: Setting {} = {}".format(entity.id, property_id, value.id if - isinstance(value, db.Entity) else value)) - return entity - - -def _treat_date(date_val: str): - - date_pattern = r"^[0-9]{4,4}-[0-9]{2,2}-[0-9]{2,2}$" - # Check general pattern since dateutil.parse is unreliable with incomplete - # dates (e.g., 2024-01) or wrong formats (e.g., 01.12.2024 is parsed as - # 2024-01-12). - if re.match(date_pattern, date_val) is None: - # ParserError for simplified error handling down the line. - raise dateparser.ParserError(f"{date_val} is not of the format YYYY-MM-DD.") - # Use dateutils.parser despite having checked the pattern to exclude - # nonsense dates like 2024-13-54. - return str(dateparser.parse(date_val).date()) - - -def _append_times_to_entity(ent, data, propname_prefix="Time", colname_time_prefix="Time collected", colname_date_prefix="Date collected"): - propname_start = f"{propname_prefix} start" - propname_stop = f"{propname_prefix} stop" - prop_start = cached_get_entity_by(name=propname_start) - prop_stop = cached_get_entity_by(name=propname_stop) - colname_time_start = f"{colname_time_prefix} start" - colname_time_stop = f"{colname_time_prefix} stop" - colname_date_start = f"{colname_date_prefix} start" - colname_date_stop = f"{colname_date_prefix} stop" - - date_start = None - date_stop = None - time_start = None - time_stop = None - timezone = None - # Time start - if colname_date_start in data and return_value_if_not_none(data[colname_date_start]) is not None: - date_start = return_value_if_not_none(data[colname_date_start]) - try: - date_start = _treat_date(date_start) - except dateparser.ParserError as perr: - logger.error( - f"There is a problem in '{colname_date_start}': {date_start}" - f" of sample {data['BIS ID']}: {perr}" - ) - raise DataInconsistencyError - if colname_time_start in data and return_value_if_not_none(data[colname_time_start]) is not None: - if not "Timezone" in data or return_value_if_not_none(data["Timezone"]) is None: - logger.error(f"{colname_time_start} but no timezone given for sample " - f"{data['BIS ID']}.") - raise DataInconsistencyError - time_start = return_value_if_not_none(data[colname_time_start]) - timezone = return_value_if_not_none(data["Timezone"]) - if date_start is None: - logger.error( - f"{colname_time_start} is given but {colname_date_start} is missing for sample {data['BIS ID']}.") - raise DataInconsistencyError - try: - _val = str(dateparser.parse(f"{date_start}T{time_start}{timezone}")) - ent = _update_property(ent, prop_start.id, property_name=prop_start.name, value=_val) - except dateparser.ParserError as perr: - logger.error( - f"Couldn't parse {colname_time_start}: {time_start} with timezone {timezone} " - f"of sample {data['BIS ID']}: {perr}" - ) - raise DataInconsistencyError - elif date_start is not None: - ent = _update_property(ent, prop_start.id, value=date_start, property_name=prop_start.name) - - # Time stop; raise error in case of stop without start - if colname_date_stop in data and return_value_if_not_none(data[colname_date_stop]) is not None: - date_stop = return_value_if_not_none(data[colname_date_stop]) - if date_stop is not None and date_start is None: - logger.error( - f"{colname_date_stop} is given but {colname_date_start} is missing for sample {data['BIS ID']}.") - raise DataInconsistencyError - if date_stop is None: - _date_stop = date_start - else: - try: - _date_stop = _treat_date(date_stop) - except dateparser.ParserError as perr: - logger.error( - f"There is a problem in '{colname_date_stop}': {date_stop}" - f" of sample {data['BIS ID']}: {perr}" - ) - raise DataInconsistencyError - - if colname_time_stop in data and return_value_if_not_none(data[colname_time_stop]) is not None: - time_stop = return_value_if_not_none(data[colname_time_stop]) - if time_start is None: - logger.error( - f"{colname_time_stop} is given but {colname_time_start} is missing for sample {data['BIS ID']}.") - raise DataInconsistencyError - # timezone is set by time start; if it hadn't been there, we would already have an error. - try: - _val = str(dateparser.parse(f"{_date_stop}T{time_stop}{timezone}")) - except dateparser.ParserError as perr: - logger.error( - f"Couldn't parse {colname_time_stop}: {time_stop} with timezone {timezone} " - f"of sample {data['BIS ID']}: {perr}" - ) - raise DataInconsistencyError - ent = _update_property(ent, prop_stop.id, property_name=prop_stop.name, value=_val) - elif date_stop is not None: - # We check date_stop but we used the cleaned-up _date_stop as value - ent = _update_property(ent, prop_stop.id, property_name=prop_stop.name, value=_date_stop) - - return ent - - -def get_container(data): - """ - Retrun the BIS ID of the Container Record that is identified by 'Storage contianer' in data. - A Container can either be identified via a BIS ID or via a BIS Label. - - If no Container can be identified, an Error is raised, since creating/registering new - Containers has to be done before registering samples. - """ - identified_by_label = False - container_identifier = data["Storage ID"] - # If the ID is not spcified, try to get the label - if "Storage Container Label" in data and pd.isnull(container_identifier): - container_identifier = data["Storage Container Label"] - identified_by_label = True - - if identified_by_label: - container = _get_container_by_label(container_identifier) - else: - container = _get_container_by_id(container_identifier) - - if container is not None: - return container - else: - msg = "Container: '{}' could not be identified.".format(container_identifier) - raise DataInconsistencyError(msg) - - -def _get_container_by_id(id): - res = db.execute_query("FIND RECORD Container WITH id = '{}'".format(id)) - if len(res) > 0: - return res[0] - else: - return None - - -def _get_container_by_label(label): - res = db.execute_query("FIND RECORD Container WITH 'BIS Label' = '{}'".format(label)) - if len(res) > 0: - return res[0] - else: - return None - - -def get_event(data, gear_id): - # Events only have names if they have the subevent property. - if "Subevent" in data and return_value_if_not_none(data["Subevent"]) is not None: - event_name = f"{data['Subevent']}" - return _create_new_source_event(event_name, data, gear_id) - - return _create_new_source_event(name=None, data=data, gear_id=gear_id) - - -def _create_new_source_event(name, data, gear_id) -> db.Record: - event = db.Record(name) - event.add_parent("SourceEvent") - event = _append_times_to_entity(event, data) - - event.add_property(name="Gear", value=gear_id) - event.add_property(name="Position", value=_get_positions( - data), datatype=db.common.datatype.LIST("Position")) # type: ignore - if "Station ID" in data and return_value_if_not_none(data["Station ID"]) is not None: - event.add_property(name="Station ID", value=str(data["Station ID"])) - if "Station number" in data and return_value_if_not_none(data["Station number"]) is not None: - event.add_property(name="Station number", value=str(data["Station number"])) - if "Hol" in data and return_value_if_not_none(data["Hol"]) is not None: - event.add_property(name="Hol", value=str(data["Hol"])) - return event - - -def _get_positions(data): - # TODO: Refactor - if "Latitude start" in data: - latitude_start = return_value_if_not_none(data["Latitude start"]) - else: - latitude_start = None - if "Latitude stop" in data: - latitude_stop = return_value_if_not_none(data["Latitude stop"]) - else: - latitude_stop = None - if "Longitude start" in data: - longitude_start = return_value_if_not_none(data["Longitude start"]) - else: - longitude_start = None - if "Longitude stop" in data: - longitude_stop = return_value_if_not_none(data["Longitude stop"]) - else: - longitude_stop = None - if "Sampling depth start" in data: - sampling_depth_start = return_value_if_not_none(data["Sampling depth start"]) - else: - sampling_depth_start = None - if "Sampling depth stop" in data: - sampling_depth_stop = return_value_if_not_none(data["Sampling depth stop"]) - else: - sampling_depth_stop = None - if "Water depth start" in data: - water_depth_start = return_value_if_not_none(data["Water depth start"]) - else: - water_depth_start = None - if "Water depth stop" in data: - water_depth_stop = return_value_if_not_none(data["Water depth stop"]) - else: - water_depth_stop = None - # insert start position - position_start = db.Record() - position_start.add_parent("StartPosition") - position_start.add_property(name="Latitude", value=latitude_start) - position_start.add_property(name="Longitude", value=longitude_start) - if not pd.isna(sampling_depth_start): - if sampling_depth_start < 0.0: - sampling_depth_start *= -1.0 - # identifiable, so add even if it is None - position_start.add_property(name="Sampling depth", value=sampling_depth_start) - if not pd.isna(water_depth_start): - if water_depth_start < 0: - water_depth_start *= -1 - # identifiable, so add even if it is None - position_start.add_property(name="Water depth", value=water_depth_start) - - # A stop position may be specified by depth stop alone: - if not (pd.isna(sampling_depth_stop) and pd.isna(water_depth_stop)): - # Copy all empty info from start position - if pd.isna(latitude_stop) and pd.isna(longitude_stop): - latitude_stop = latitude_start - longitude_stop = longitude_start - if pd.isna(sampling_depth_stop): - sampling_depth_stop = sampling_depth_start - if pd.isna(water_depth_stop): - water_depth_stop = water_depth_start - # If there is an endposition: insert endposition - if not (pd.isna(latitude_stop) or pd.isna(longitude_stop)): - - position_end = db.Record() - # position_end = db.Record("({}, {})".format(latitude_stop, longitude_stop)) - position_end.add_parent("StopPosition") - position_end.add_property(name="Latitude", value=latitude_stop) - position_end.add_property(name="Longitude", value=longitude_stop) - if not pd.isna(sampling_depth_stop): - if sampling_depth_stop < 0: - sampling_depth_stop *= -1 - # position_end.name = position_end.name + " at -{}m".format(sampling_depth_stop) - # identifiable, so add even if it is None - position_end.add_property(name="Sampling depth", value=sampling_depth_stop) - if not pd.isna(water_depth_stop): - if water_depth_stop < 0: - water_depth_stop *= -1 - # identifiable, so add even if it is None - position_end.add_property(name="Water depth", value=water_depth_stop) - # position_end.insert(unique=False) - return [position_start, position_end] - else: - return [position_start] - - -def get_gear(data): - """ - Return the BIS ID of the Gear that is specified by 'Gear' and 'Gear configuration' in data. - - If no Such Gear Record exists, a new Gear Record is created. - """ - - qtext = f"FIND RECORD '{data['Gear']}'" - if "Gear configuration" in data and pd.notnull(data["Gear configuration"]): - qtext += f" WITH 'Configuration'='{data['Gear configuration']}'" - try: - res = db.execute_query(qtext, unique=True) - except db.exceptions.EmptyUniqueQueryError: - raise DataInconsistencyError(f"The query\n{qtext}\nreturned no results.") - except db.exceptions.QueryNotUniqueError: - raise DataInconsistencyError(f"The query\n{qtext}\nreturned more than one result.") - return res - - -def get_nagoya_case(data): - """Create and retrun a NagoyaCase Record.""" - nagoya_case_number = return_value_if_not_none(data["Nagoya case number"]) - - nagoya_case = db.Record(nagoya_case_number) - nagoya_case.add_parent(name="NagoyaCase") - nagoya_case.add_property(name="Nagoya Case Number", value=nagoya_case_number) - - return nagoya_case - - -def get_person(text) -> db.Record: - """ - Return the BIS ID of the person that is specifed as 'Main User' or 'Sampling Person' in data. - - If the Person is not present in the database, an Exception is raised. Creating new Person Reconrds can only be done by a priviledged user. - """ - # Check in which format the person is identified: - person_identifier = text.split(", ") - if len(person_identifier) == 1: - person = _get_person_by_abbreviation(person_identifier[0]) - else: - person = _get_person_by_fullname(person_identifier[1], person_identifier[0]) - - return person - - -def _get_person_by_fullname(first_name, last_name): - # seach for person in db - res = db.execute_query( - "FIND RECORD Person WITH 'First name' = '{}' AND 'Last name' = '{}'".format(first_name, last_name)) - # if person doesn't exist in database... - if len(res) == 0: - # There is not enought data in the template to create a new Person record. Hence, we have to raise an Exception - error_msg = "There is no person Record with 'First name' = '{}' AND 'Last name' = '{}' in the database. ".format( - first_name, last_name) - raise DataInconsistencyError(error_msg) - else: - return res[0] - - -def _get_person_by_abbreviation(abbreviation): - # seach for person in db - res = db.execute_query("FIND RECORD Person WITH 'Abbreviation' = '{}'".format(abbreviation)) - # if person doesn't exist in database... - if len(res) == 0: - # There is not enought data in the template to create a new Person record. Hence, we have to raise an Exception - error_msg = "There is no Person Record with Abbreviation = '{}'".format(abbreviation) - raise DataInconsistencyError(error_msg) - else: - return res[0] - - def synchroize(records, additional_property_ents, htmluserlog_public): crawler = Crawler(securityMode=SecurityMode.UPDATE) identifiables_definition_file = os.path.expanduser("~/identifiables.yml") @@ -468,15 +106,6 @@ def update_sample_records(data, htmluserlog_public): # TODO Check data first and if there are Errors in the data: Provide the user with a download # link to a template with Error descriptions. - # Get property ids: - person_property_id = db.get_entity_by_name("Main User").id - sampling_person_property_id = db.get_entity_by_name("Sampling Person").id - nagoya_case_property_id = get_id_of_datatype("NagoyaCase") - container_property_id = get_id_of_datatype("Container") - event_property_id = get_id_of_datatype("SourceEvent") - pdfreport_property_id = get_id_of_datatype("PDFReport") - parent_sample_property_id = db.get_entity_by_name("Parent Sample").id - additional_properties = data.keys().to_list() additional_property_ids = {} # name-> id additional_property_ents = {} # name-> Entity @@ -502,65 +131,21 @@ def update_sample_records(data, htmluserlog_public): for index, row in data.iterrows(): - sample_id_exists = not pd.isnull(row["BIS ID"]) + sample_id_exists = not pd.isnull(row[get_column_header_name("entity_id")]) if not sample_id_exists: raise DataInconsistencyError(f"Missing sample ID in row {index}") try: sample = db.execute_query( - "FIND RECORD Sample WITH id = {}".format(row["BIS ID"]), unique=True) + "FIND RECORD Sample WITH id = {}".format( + row[get_column_header_name("entity_id")]), + unique=True) except db.exceptions.EmptyUniqueQueryError: - msg = "There is no Sample with ID = {} in the system.".format(row["BIS ID"]) + msg = "There is no Sample with ID = {} in the system.".format( + row[get_column_header_name("entity_id")]) raise DataInconsistencyError(msg) - sample = _update_property(entity=sample, property_id=person_property_id, - property_name="Main User", - value=get_person(row["Main User"])) - if "Parent BIS ID" in row and return_value_if_not_none(row["Parent BIS ID"]) is not None: - sample = _update_property(entity=sample, property_id=parent_sample_property_id, - value=row["Parent BIS ID"]) - if ("Sampling Person" in row - and return_value_if_not_none(row["Sampling Person"]) is not None): - sample = _update_property(entity=sample, property_id=sampling_person_property_id, - property_name="Sampling Person", - value=get_person(row["Sampling Person"])) - if "PI" in row and return_value_if_not_none(row["PI"]) is not None: - sample = _update_property(entity=sample, property_id=db.get_entity_by_name("PI").id, - property_name="PI", - value=get_person(row["PI"])) - if "Nagoya case number" in row and return_value_if_not_none(row["Nagoya case number"]) is not None: - sample = _update_property(entity=sample, property_id=nagoya_case_property_id, - property_name="NagoyaCase", - value=get_nagoya_case(row)) - if "Storage ID" in row and return_value_if_not_none(row["Storage ID"]) is not None: - sample = _update_property(entity=sample, property_id=container_property_id, - property_name="Container", - value=get_container(row)) - if "Collection" in row and return_value_if_not_none(row["Collection"]) is not None: - sample = _update_property(entity=sample, - property_id=db.get_entity_by_name("Collection").id, - property_name="Collection", - datatype=db.LIST("Collection"), - value=[db.Record(name=el) - .add_parent(name="Collection") - for el in row["Collection"]]) - if "Sphere" in row and return_value_if_not_none(row["Sphere"]) is not None: - sample = _update_property(entity=sample, - property_id=db.get_entity_by_name("Sphere").id, - property_name="Sphere", - datatype=db.LIST("Sphere"), - value=[db.Record(name=el) - .add_parent(name="Sphere") - for el in row["Sphere"]]) - - if "Date collected start" in row and return_value_if_not_none(row["Date collected start"]) is not None: - sample = _update_property(entity=sample, property_id=event_property_id, property_name='SourceEvent', value=get_event( - row, get_gear(row))) - if "PDFReport" in data.columns: - sample = _update_property( - entity=sample, property_id=pdfreport_property_id, property_name="PDFReport", value=row["PDFReport"]) - if "Date sampled start" in data.columns: - sample = _append_times_to_entity(ent=sample, data=row, propname_prefix="Time sampled", - colname_time_prefix="Time sampled", colname_date_prefix="Date sampled") + # All special properties are added here + sample = add_special_properties(sample, row) # Add additional properties for property_name in additional_property_ids.keys(): @@ -581,12 +166,16 @@ def update_sample_records(data, htmluserlog_public): name=ent.properties[0].name, value=return_value_if_not_none(row[property_name])) else: value = return_value_if_not_none(row[property_name]) - sample = _update_property( + sample = update_property( entity=sample, property_id=additional_property_ids[property_name], value=value, property_name=property_name) + # Now, treat events and event data + sample = add_event_to_sample(sample, row) samples.append(sample) + # Samples might need additional post processing + samples = post_process_samples(samples, data) synchroize(samples, additional_property_ents, htmluserlog_public) diff --git a/sample-management-custom/caosdb-server/scripting/bin/export_container_csv.py b/sample-management-custom/caosdb-server/scripting/bin/export_container_csv.py index a23c47a849941e089d81f1b8afccf26e4ef02b98..13a2a1b3f77b0b35573e1c6c3246fab2ac1ff668 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/export_container_csv.py +++ b/sample-management-custom/caosdb-server/scripting/bin/export_container_csv.py @@ -36,18 +36,18 @@ from linkahead.exceptions import (EmptyUniqueQueryError, from bis_utils import (create_email_with_link_text, get_description_row, get_email_from_username, - send_mail_with_defaults) + get_options_row, send_mail_with_defaults) +from sample_helpers.utils import (CONSTANTS, get_column_header_name, + get_entity_name) # suppress warning of diff function -apilogger = logging.getLogger("caosdb.apiutils") +apilogger = logging.getLogger("linkahead.apiutils") apilogger.setLevel(logging.ERROR) logger = logging.getLogger("caosadvancedtools") -CACHE = {} - -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] def cached_query(query, unique=False): @@ -66,18 +66,20 @@ def reverse_semicolon_separated_list(value): def generate_label_text(entity): - if entity.get_property("Custom label") is not None and entity.get_property("Custom label").value: - label = entity.get_property("Custom label").value + custom_label_prop_name = get_entity_name("custom_label_prop") + if entity.get_property(custom_label_prop_name) is not None and entity.get_property(custom_label_prop_name).value: + label = entity.get_property(custom_label_prop_name).value else: - label = entity.get_property("BIS label").value if entity.get_property( - "BIS label") is not None else None + container_label_prop_name = get_entity_name("container_label_prop") + label = entity.get_property(container_label_prop_name).value if entity.get_property( + container_label_prop_name) is not None else None if not label: if not entity.name: return entity.id return f"{entity.id} {entity.name}" if f"{label}" == f"{entity.id}": - # prevent special case of BisLabel = BisID resulting in a preview + # prevent special case of Label = ID resulting in a preview # that looks like "id, id". return entity.id return f"{entity.id} {label}" @@ -85,7 +87,9 @@ def generate_label_text(entity): def extract_parent_container(record, key): - possible_parents = cached_query(f"FIND Container WHICH REFERENCES {record.id}") + possible_parents = cached_query( + f"FIND {get_entity_name('container_rt')} WHICH REFERENCES {record.id}" + ) values = [] for par in possible_parents: @@ -96,16 +100,18 @@ def extract_parent_container(record, key): return reverse_semicolon_separated_list(values) -def extract_bis_id(record, key): +def extract_eid(record, key): return record.id def extract_custom_label(record, key): - if record.get_property("Custom label") is not None: + custom_label_prop_name = get_entity_name("custom_label_prop") + + if record.get_property(custom_label_prop_name) is not None: - return record.get_property("Custom label").value + return record.get_property(custom_label_prop_name).value return None @@ -117,8 +123,9 @@ def extract_pi(record, key): pi_id = record.get_property("PI").value pi_rec = cached_query(f"FIND RECORD Person WITH ID={pi_id}", unique=True) - if pi_rec.get_property("Abbreviation") is not None and pi_rec.get_property("Abbreviation").value is not None: - return pi_rec.get_property("Abbreviation").value + abbr_prop_name = get_entity_name("abbreviation_prop") + if pi_rec.get_property(abbr_prop_name) is not None and pi_rec.get_property(abbr_prop_name).value is not None: + return pi_rec.get_property(abbr_prop_name).value return pi_id return None @@ -145,11 +152,14 @@ def extract_collection(record, key): def extract_container_type(record, key): - if record.get_property("ContainerType") is not None and record.get_property("ContainerType").value is not None: - ct_id = record.get_property("ContainerType").value - ctr = cached_query(f"FIND ContainerType WITH ID={ct_id}", unique=True) - if ctr.get_property("Abbreviation") is not None and ctr.get_property("Abbreviation").value is not None: - return ctr.get_property("Abbreviation").value + abbr_prop_name = get_entity_name("abbreviation_prop") + containertype_rt_name = get_entity_name("containertype_rt") + + if record.get_property(containertype_rt_name) is not None and record.get_property(containertype_rt_name).value is not None: + ct_id = record.get_property(containertype_rt_name).value + ctr = cached_query(f"FIND {containertype_rt_name} WITH ID={ct_id}", unique=True) + if ctr.get_property(abbr_prop_name) is not None and ctr.get_property(abbr_prop_name).value is not None: + return ctr.get_property(abbr_prop_name).value return ct_id return None @@ -159,7 +169,7 @@ def extract_storage_chain(record, key): def find_referencing_containers(eid): containing_containers = [] - candidates = cached_query(f"FIND CONTAINER WHICH REFERENCES {eid}") + candidates = cached_query(f"FIND {get_entity_name('container_rt')} WHICH REFERENCES {eid}") if len(candidates) > 1: logger.debug(f"Entity {eid} is referenced by more than one container.") return [] @@ -183,25 +193,25 @@ def default_extractor(record, key): EXTRACTORS = { - "BIS ID": extract_bis_id, - "BIS label": default_extractor, - "Collection": extract_collection, - "Container Contents": default_extractor, - "Container size": default_extractor, - "Container type": extract_container_type, - "Custom label": extract_custom_label, - "PDFReport": default_extractor, - "PI": extract_pi, - "Parent container": extract_parent_container, - "Storage chain": extract_storage_chain, + get_column_header_name("entity_id"): extract_eid, + get_column_header_name("container_label_prop"): default_extractor, + get_column_header_name("Collection"): extract_collection, + get_column_header_name("container_size_prop"): default_extractor, + get_column_header_name("containertype_rt"): extract_container_type, + get_column_header_name("custom_label_prop"): extract_custom_label, + get_column_header_name("PDFReport"): default_extractor, + get_column_header_name("PI"): extract_pi, + get_column_header_name("Parent container"): extract_parent_container, + get_column_header_name("Storage chain"): extract_storage_chain, } # List of sample properties to be ignored because they are treated # otherwise. Similar, but not identical to SPECIAL TREATMENT. IGNORE_KEYS = [ - "Child container", # To be changed by updating the child, not the parent. - "ContainerType", # handled separately above - "Responsible", # Not to be changed via upload + # To be changed by updating the child, not the parent. + get_column_header_name("child_container_prop"), + get_column_header_name("containertype_rt"), # handled separately above + get_column_header_name("Responsible"), # Not to be changed via upload ] @@ -225,29 +235,29 @@ def retrieve_containers(data): container = [] not_found = [] - for bis_id in data: - if isinstance(bis_id, int): + for eid in data: + if isinstance(eid, int): try: container.append( cached_query( - f"FIND RECORD Container WITH id = '{bis_id}'", + f"FIND RECORD {get_entity_name('container_rt')} WITH id = '{eid}'", unique=True)) except EmptyUniqueQueryError as e: # we want to warn about these - not_found.append(bis_id) + not_found.append(eid) else: found_at_least_one_in_range = False - for next_bis_id in bis_id: + for next_eid in eid: try: container.append( cached_query( - f"FIND RECORD Container WITH id = '{next_bis_id}'", + f"FIND RECORD {get_entity_name('container_rt')} WITH id = '{next_eid}'", unique=True)) found_at_least_one_in_range = True except EmptyUniqueQueryError as e: pass if not found_at_least_one_in_range: - not_found.append(f"{bis_id.start}-{bis_id.stop-1}") + not_found.append(f"{eid.start}-{eid.stop-1}") return container, not_found @@ -315,7 +325,7 @@ def main(): with open(args.filename) as form_json: form_data = json.load(form_json) - tmp = form_data["bis_ids"].split(",") + tmp = form_data["container_ids"].split(",") data = [] for d in tmp: if "-" in d: @@ -355,7 +365,7 @@ def main(): f"{ke}\n\nPlease contact your admin." ) else: - msg = "{}export_sample_csv.py was called without the JSON file in args.{}".format( + msg = "{} export_container_csv.py was called without the JSON file in args. {}".format( ERROR_PREFIX, ERROR_SUFFIX) logger.error(msg) diff --git a/sample-management-custom/caosdb-server/scripting/bin/export_sample_csv.py b/sample-management-custom/caosdb-server/scripting/bin/export_sample_csv.py index 4f47b48eb46557cc49e5e77d9dcef25a7ac15445..7311fc101929f2012269d4001904c1c34edf3f09 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/export_sample_csv.py +++ b/sample-management-custom/caosdb-server/scripting/bin/export_sample_csv.py @@ -25,33 +25,30 @@ import logging import os import sys import urllib -from datetime import date, datetime -from typing import List import linkahead as db import pandas as pd -from caosadvancedtools.datainconsistency import DataInconsistencyError from caosadvancedtools.serverside import helper from caosadvancedtools.table_export import BaseTableExporter -from caosadvancedtools.table_importer import CSVImporter -from caoscrawler import Crawler, SecurityMode -from caoscrawler.crawl import ForbiddenTransaction -from caoscrawler.identifiable_adapters import CaosDBIdentifiableAdapter +from caoscrawler.config import get_config_setting from caoscrawler.logging import configure_server_side_logging from linkahead.cached import cached_get_entity_by, cached_query as cquery from linkahead.common.datatype import get_id_of_datatype from linkahead.exceptions import (EmptyUniqueQueryError, QueryNotUniqueError, TransactionError) -from dateutil import parser as dateparser -from dateutil.parser import isoparse from bis_utils import (create_email_with_link_text, get_description_row, get_email_from_username, - get_options_row, send_mail_with_defaults, - SPECIAL_TREATMENT_SAMPLE as SPECIAL_TREATMENT) + get_options_row, send_mail_with_defaults) from export_container_csv import (generate_label_text, extract_storage_chain as container_storage_chain) -from upload_sample_template import DATATYPE_DEFINITIONS +from sample_helpers.sample_registration_get_person_identifier import get_person_identifier_from_rec +from sample_helpers.sample_upload_column_definitions import ( + DATATYPE_DEFINITIONS, SPECIAL_TREATMENT_SAMPLE as + SPECIAL_TREATMENT, use_custom_names) +from sample_helpers.utils import (CONSTANTS, get_column_header_name, + get_entity_name) + # suppress warning of diff function apilogger = logging.getLogger("linkahead.apiutils") @@ -59,21 +56,8 @@ apilogger.setLevel(logging.ERROR) logger = logging.getLogger("caosadvancedtools") -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' - - -# Declare mapping of different time/date column names to property names -TIME_NAME_MAPPINGS = { - "date collected start": "time start", - "time collected start": "time start", - "date collected stop": "time stop", - "time collected stop": "time stop", - "date sampled start": "time sampled start", - "time sampled start": "time sampled start", - "date sampled stop": "time sampled stop", - "time sampled stop": "time sampled stop", -} +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] def cached_record(i): @@ -91,7 +75,10 @@ def cached_query(query, unique=False): def reverse_semicolon_separated_list(value): if isinstance(value, list): - return ";".join([str(val) for val in value if val is not None]) + ret_str = ";".join([str(val) for val in value if val is not None]) + if ',' in ret_str: + ret_str = f"\"{ret_str}\"" + return ret_str else: return value @@ -144,7 +131,7 @@ def extract_value_as_list(record, key): def extract_storage_id(record, key): - return extract_value_as_list(record, "Container") + return extract_value_as_list(record, get_entity_name("container_rt")) def extract_pdf_id(record, key): @@ -152,19 +139,20 @@ def extract_pdf_id(record, key): return prop.value if prop is not None else None -def extract_storage_container_label(record, key): - ids = extract_value_as_list(record, "Container") - return retrieve_values(ids, 'BIS Label') - - -def extract_nagoya_case_number(record, key): - ids = extract_value_as_list(record, "NagoyaCase") - return retrieve_values(ids, key) - - def extract_person(record, key): ids = extract_value_as_list(record, key) - return retrieve_values(ids, 'Abbreviation') + person_recs = [cached_record(i) for i in ids] + return [get_person_identifier_from_rec(r) for r in person_recs] + + +def extract_event_responsible(record_key): + evt = retrieve_event(record) + if len(evt) == 0: + return None + elif len(evt) > 1: + logger.debug(f"Sample {record.id} references more than one event.") + return None + return extract_person(evt[0], get_entity_name("responsible_person_event")) def extract_parent_sample(record, key): @@ -179,74 +167,14 @@ def extract_reference_name(record, key): for i in ids if i is not None] -def retrieve_source_event(record): - ids = extract_value_as_list(record, 'SourceEvent') - if record.get_property("SourceEvent") is None: +def retrieve_event(record): + ids = extract_value_as_list(record, get_entity_name("event_rt")) + if record.get_property(get_entity_name("event_rt")) is None: # there are cases where this property is named "Event" - ids = extract_value_as_list(record, 'Event') + ids = extract_value_as_list(record, get_entity_name("event_rt")) return [cached_record(i) for i in ids] -def retrieve_gear(record): - ids = [e.get_property("Gear").value for e in retrieve_source_event(record) - if e.get_property("Gear") is not None] - return [cached_query(f"SELECT 'parent', 'Configuration' FROM ENTITY WITH id = '{i}'", unique=True) for i in ids] - - -def extract_gear(record, key): - return [e.get_parents()[0].name for e in retrieve_gear(record)] - - -def extract_gear_configuration(record, key): - return [e.get_property("Configuration").value for e in - retrieve_gear(record) - if e.get_property("Configuration") is not None] - - -def extract_date_time(record, p): - if p.lower() == "time start" or p.lower() == "time stop": - # these are attached to the source event directly - return [e.get_property(p).value for e in retrieve_source_event(record) if - e.get_property(p) is not None and e.get_property(p).value is not None] - else: - return extract_value_as_list(record, p) - - -def extract_date(record, key): - if key.lower() in TIME_NAME_MAPPINGS.keys(): - time_key = TIME_NAME_MAPPINGS[key.lower()] - else: - time_key = key - date_times = extract_date_time(record, time_key) - return [isoparse(t).date().isoformat() for t in date_times] - - -def extract_time(record, key): - if key.lower() in TIME_NAME_MAPPINGS.keys(): - time_key = TIME_NAME_MAPPINGS[key.lower()] - else: - time_key = key - date_times = extract_date_time(record, time_key) - return [isoparse(t).time().isoformat() for t in date_times] - - -def extract_time_zone(record, key): - date_times = extract_date_time(record, "Time start") - return [isoparse(t).tzname() for t in date_times] - - -def extract_station_number(record, key): - source_ev = retrieve_source_event(record) - return [e.get_property(key).value for e in source_ev if - e.get_property(key) is not None] - - -def extract_station_id(record, key): - source_ev = retrieve_source_event(record) - return [e.get_property(key).value for e in source_ev if - e.get_property(key) is not None] - - def retrieve_positions(source_ev): pos_ids = extract_value_as_list(source_ev, "Position") return [cached_record(i) for i in pos_ids] @@ -258,7 +186,7 @@ def has_parent(r, par): def extract_position(record, position, component): - source_evs = retrieve_source_event(record) + source_evs = retrieve_event(record) result = [] for ev in source_evs: _pos = [pos for pos in retrieve_positions(ev)] @@ -275,83 +203,113 @@ def extract_position(record, position, component): return [pos.get_property(component).value for pos in result if pos.get_property(component) is not None] +def extract_ele_start(record, key): + return extract_position(record, get_entity_name("StartPosition"), get_entity_name("elevation")) + + +def extract_ele_stop(record, key): + return extract_position(record, get_entity_name("StopPosition"), get_entity_name("elevation")) + + def extract_lat_start(record, key): - return extract_position(record, "StartPosition", "Latitude") + return extract_position(record, get_entity_name("StartPosition"), get_entity_name("latitude")) def extract_lat_stop(record, key): - return extract_position(record, "StopPosition", "Latitude") + return extract_position(record, get_entity_name("StopPosition"), get_entity_name("latitude")) def extract_lng_start(record, key): - return extract_position(record, "StartPosition", "Longitude") + return extract_position(record, get_entity_name("StartPosition"), get_entity_name("longitude")) def extract_lng_stop(record, key): - return extract_position(record, "StopPosition", "Longitude") + return extract_position(record, get_entity_name("StopPosition"), get_entity_name("longitude")) -def extract_sampling_depth_start(record, key): - return extract_position(record, "StartPosition", "Sampling depth") +def extract_linkahead_url(record, key): + # base_uri = db.get_config().get("Connection", "url") + base_uri = get_config_setting("public_host_url") + return urllib.parse.urljoin(base_uri, f"Entity/{record.id}") -def extract_sampling_depth_stop(record, key): - return extract_position(record, "StopPosition", "Sampling depth") +def extract_doi(record, key): + source_evs = retrieve_event(record) + if len(source_evs) > 1: + logger.error( + f"Sample {record.id} references more than one event so no unique DOI can be exported.") + return None + elif len(source_evs) == 0: + return None + ev = source_evs[0] + return ev.get_property(get_entity_name("igsn_doi_prop")).value if ev.get_property("igsn_doi_prop") is not None else None -def extract_water_depth_start(record, key): - return extract_position(record, "StartPosition", "Water depth") +def _extract_event_prop(record, key, ref=False): + evt = retrieve_event(record) + if len(evt) == 0: + return None + elif len(evt) > 1: + logger.debug(f"Sample {record.id} references more than one event.") + return None -def extract_water_depth_stop(record, key): - return extract_position(record, "StopPosition", "Water depth") + if ref: + return extract_reference_name(evt[0], key) + return extract_value_as_list(evt[0], key) -def extract_source_event_name(record, key): - return [e.name for e in retrieve_source_event(record)] +def extract_biome(record, key): -def extract_hol(record, key): - source_ev = retrieve_source_event(record) - return [e.get_property(key).value for e in source_ev if - e.get_property(key) is not None] + return _extract_event_prop(record, get_entity_name("Biome"), ref=True) -def extract_bis_url(record, key): - # base_uri = db.get_config().get("Connection", "url") - base_uri = "https://biosamples.geomar.de/" - return urllib.parse.urljoin(base_uri, f"Entity/{record.id}") +def extract_campaign(record, key): + return _extract_event_prop(record, get_entity_name("Campaign"), ref=True) -def extract_igsn(record, key): - source_evs = retrieve_source_event(record) - if len(source_evs) > 1: - logger.error( - f"Sample {record.id} references more than one SourceEvent so no unique IGSN can be exported.") - return None - elif len(source_evs) == 0: - return None - ev = source_evs[0] - return ev.get_property(key).value if ev.get_property(key) is not None else None +def extract_device(record, key): -def extract_doi(record, key): - source_evs = retrieve_source_event(record) - if len(source_evs) > 1: - logger.error( - f"Sample {record.id} references more than one SourceEvent so no unique DOI can be exported.") - return None - elif len(source_evs) == 0: - return None - ev = source_evs[0] - return ev.get_property("DOI").value if ev.get_property("DOI") is not None else None + return _extract_event_prop(record, get_entity_name("Device"), ref=True) + + +def extract_end_date(record, key): + + return _extract_event_prop(record, get_entity_name("end_date_prop")) + + +def extract_start_date(record, key): + + return _extract_event_prop(record, get_entity_name("start_date_prop")) + + +def extract_level(record, key): + + return _extract_event_prop(record, get_entity_name("level")) + + +def extract_sphere(record, key): + + return _extract_event_prop(record, get_entity_name("Sphere"), ref=True) + + +def extract_locality_descr(record, key): + + return _extract_event_prop(record, get_entity_name("locality_description_prop")) + + +def extract_locality_name(record, key): + + return _extract_event_prop(record, get_entity_name("locality_name_prop")) def extract_storage_chain(record, key): - if record.get_property("Container") is not None and record.get_property("Container").value: + if record.get_property(get_entity_name("container_rt")) is not None and record.get_property(get_entity_name("container_rt")).value: - cont_id = record.get_property("Container").value + cont_id = record.get_property(get_entity_name("container_rt")).value if isinstance(cont_id, list): if len(cont_id) > 1: logger.debug(f"Sample {record.id} has multiple containers.") @@ -368,103 +326,71 @@ def extract_storage_chain(record, key): def extract_event_url(record, key): - events = retrieve_source_event(record) + events = retrieve_event(record) if not events: return None if len(events) == 1: - return urllib.parse.urljoin("https://biosamples.geomar.de", f"Entity/{events[0].id}") + return urllib.parse.urljoin(get_config_setting("public_host_url"), f"Entity/{events[0].id}") logger.debug(f"Sample {record.id} has multiple events.") return None -# must be same keys as SPECIAL_TREATMENT -EXTRACTORS = { - "BIS ID": lambda record, key: record.id, - "Parent BIS ID": extract_parent_sample, - "AphiaID": default_find, - "Collection": extract_reference_name, - "Date collected start": extract_date, - "Date collected stop": extract_date, - "Date sampled start": extract_date, - "Date sampled stop": extract_date, +# must include all keys from SPECIAL_TREATMENT +EXTRACTORS = use_custom_names({ + "entity_id": lambda record, key: record.id, "Main User": extract_person, - "Sampling Person": extract_person, - "PI": extract_person, - "Person": extract_person, - "Gear": extract_gear, - "Gear configuration": extract_gear_configuration, + "Biome": extract_biome, + "Campaign": extract_campaign, + "Collector": extract_person, + "Curator": extract_person, + "Device": extract_device, + "Elevation start": extract_ele_start, + "Elevation stop": extract_ele_stop, + "Embargo": default_find, + "End date": extract_end_date, "Latitude start": extract_lat_start, - "Longitude start": extract_lng_start, - "Storage ID": extract_storage_id, - "Nagoya case number": extract_nagoya_case_number, - "PDFReport": extract_pdf_id, - "Subevent": extract_source_event_name, - "Time collected start": extract_time, - "Time collected stop": extract_time, - "Time sampled start": extract_time, - "Time sampled stop": extract_time, - "Timezone": extract_time_zone, - "Station ID": extract_station_id, - "Station number": extract_station_number, - "Sampling depth start": extract_sampling_depth_start, - "Sampling depth stop": extract_sampling_depth_stop, - "Water depth start": extract_water_depth_start, - "Water depth stop": extract_water_depth_stop, "Latitude stop": extract_lat_stop, + "Level": extract_level, + "LinkAhead URL": extract_linkahead_url, + "Longitude start": extract_lng_start, "Longitude stop": extract_lng_stop, - "Storage chain": extract_storage_chain, - "Storage Container Label": extract_storage_container_label, - "Hol": extract_hol, + "PDFReport": extract_pdf_id, "Sampling method": default_find, - # "Publications": TODO never used - # "NCBI BioProject": TODO never used - # "NCBI BioSample": TODO never used - # "NCBI Accession": TODO never used - "BIS URL": extract_bis_url, - "IGSN": extract_igsn, - "IGSN URL": extract_doi, - "Sphere": default_find, - "URL SourceEvent": extract_event_url, -} - -REVERSE_COLUMN_CONVERTER = { - "Collection": collection_value, - "PI": person_value, - "Person": person_value, -} + "Sphere": extract_sphere, + "Start date": extract_start_date, + "Storage ID": extract_storage_id, + "Storage chain": extract_storage_chain, + "URL Event": extract_event_url, + "igsn_doi_prop": extract_doi, + "locality_description_prop": extract_locality_descr, + "locality_name_prop": extract_locality_name, + "parent_sample_prop": extract_parent_sample, + "responsible_person_event": extract_event_responsible +}) + +REVERSE_COLUMN_CONVERTER = use_custom_names({ +}) # List of sample properties to be ignored because they are treated # otherwise. Similar, but not identical to SPECIAL TREATMENT. -IGNORE_KEYS = [ - "NagoyaCase", - "Parent Sample", - "Container", - "SourceEvent", - "Event", - "Date", - "Sampling depth", - "Water depth" -] +IGNORE_KEYS = use_custom_names([ + "parent_sample_prop", + "container_rt", + "event_rt", +]) # Additional list of keys to be ignored when extracting parent sample information -IGNORE_KEYS_PARENT = IGNORE_KEYS + [ - "BIS ID", -] +IGNORE_KEYS_PARENT = IGNORE_KEYS + use_custom_names([ + "entity_id", +]) # List of columns to be exported although they are not known to or ignored by # the import. -ADDITIONAL_EXPORTS = [ - "BIS URL", - "Date sampled start", - "Date sampled stop", - "IGSN URL", - "IGSN", - "Parent BIS ID", +ADDITIONAL_EXPORTS = use_custom_names([ + "LinkAhead URL", + "parent_sample_prop", "Storage chain", - "Time sampled start", - "Time sampled stop", - "URL SourceEvent" -] +]) def extract_value(r, e): @@ -593,25 +519,25 @@ def to_csv(samples): def retrieve_samples(data): container = [] not_found = [] - for bis_id in data: - if isinstance(bis_id, int): + for eid in data: + if isinstance(eid, int): try: container.append( - cached_get_entity_by(query=f"FIND RECORD SAMPLE WITH id='{bis_id}'")) + cached_get_entity_by(query=f"FIND RECORD SAMPLE WITH id='{eid}'")) except EmptyUniqueQueryError as e: # we want to warn about these - not_found.append(bis_id) + not_found.append(eid) else: found_at_least_one_in_range = False - for next_bis_id in bis_id: + for next_eid in eid: try: container.append( - cached_get_entity_by(query=f"FIND RECORD Sample WITH id='{next_bis_id}'")) + cached_get_entity_by(query=f"FIND RECORD Sample WITH id='{next_eid}'")) found_at_least_one_in_range = True except EmptyUniqueQueryError as e: pass if not found_at_least_one_in_range: - not_found.append(f"{bis_id.start}-{bis_id.stop-1}") + not_found.append(f"{eid.start}-{eid.stop-1}") return container, not_found @@ -679,19 +605,6 @@ def main(): logger.info("Removing empty columns from export") no_empty_columns = True - if "from_date" in form_data: - # Inserted after ... - data = [el.id for el in db.execute_query( - "SELECT id FROM sample WHICH REFERENCES A SourceEvent " - "WHICH HAS AN IGSN AND " - f"(WHICH WAS INSERTED SINCE {form_data['from_date']})") - ] - # ... + update after - data += [el.id for el in db.execute_query( - "SELECT id FROM sample WHICH REFERENCES A SourceEvent " - "WHICH HAS AN IGSN AND " - f"(WHICH WAS UPDATED SINCE {form_data['from_date']})") - ] elif "query_string" in form_data and form_data["query_string"]: query_string = form_data["query_string"] if not query_string.lower().startswith("find ") and not query_string.lower().startswith("select "): @@ -710,11 +623,11 @@ def main(): ) return else: - if not form_data["bis_ids"]: + if not form_data["ids"]: logger.error( "Please specify the samples to be exported either by query or by id(s).") return - tmp = form_data["bis_ids"].split(",") + tmp = form_data["ids"].split(",") data = [] for d in tmp: if "-" in d: diff --git a/sample-management-custom/caosdb-server/scripting/bin/register_new_containers.py b/sample-management-custom/caosdb-server/scripting/bin/register_new_containers.py index f64e9341540ccc81506f1878c4f1c3048f21da43..fb6bd65d6f784188eff9330dc8367d7415cf8e00 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/register_new_containers.py +++ b/sample-management-custom/caosdb-server/scripting/bin/register_new_containers.py @@ -23,23 +23,26 @@ # import csv -import datetime import json import logging import sys -import caosdb as db +import linkahead as db from caosadvancedtools.serverside import helper from caosadvancedtools.serverside.logging import configure_server_side_logging from bis_utils import (create_email_with_link_text, get_description_row, get_email_from_username, get_options_row, send_mail_with_defaults) +from sample_helpers.container_name import get_container_name_prefix +from sample_helpers.container_registration_identifier import get_container_identifier +from sample_helpers.container_registration_post_processing import ( + post_process_containers, post_process_parent_container) +from sample_helpers.utils import CONSTANTS, get_column_header_name, get_entity_name -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' -BIS_URL = 'https://biosamples.geomar.de/Entity/' +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] logger = logging.getLogger("caosadvancedtools") @@ -50,125 +53,95 @@ def get_parser(): return par -def get_current_year(): - currentDateTime = datetime.datetime.now() - date = currentDateTime.date() - year = date.strftime("%Y") - return year - - -def get_abbreviation(id): - try: - rd = db.execute_query("FIND {}".format(id))[0] - return rd.get_property("Abbreviation").value - except: - entity_url = BIS_URL + str(id) - logger.error('{}There was a problem while querying for the abbreviation of <a href="{}">this BIS Entity</a>.{}'.format( - ERROR_PREFIX, entity_url, ERROR_SUFFIX)) - - def create_new_label_counter(container_type_id, value=1): counter = db.Record() - counter.add_parent(name="LabelCounter") - counter.add_property(name="Counter", value=value) - counter.add_property(name="ContainerType", value=container_type_id) + counter.add_parent(name=get_entity_name("labelcounter_rt")) + counter.add_property(name=get_entity_name("labelcounter_prop"), value=value) + counter.add_property(name=get_entity_name("containertype_rt"), value=container_type_id) counter.insert() def get_label_counter(container_type_id): counter = db.execute_query( - "FIND LabelCounter WITH ContainerType = {}".format(container_type_id)) + f"FIND '{get_entity_name('labelcounter_rt')}' WITH " + f"'{get_entity_name('containertype_rt')}' = {container_type_id}" + ) if len(counter) == 0: create_new_label_counter(container_type_id) return 1 else: - return counter[0].get_property("Counter").value + return counter[0].get_property(get_entity_name("labelcounter_prop")).value def set_label_counter(container_type_id, value): counter = db.execute_query( - "FIND LabelCounter WITH ContainerType = {}".format(container_type_id)) + f"FIND '{get_entity_name('labelcounter_rt')}' WITH " + f"'{get_entity_name('containertype_rt')}' = {container_type_id}" + ) if len(counter) == 0: create_new_label_counter(container_type_id, value) else: try: - counter[0].get_property("Counter").value = value + counter[0].get_property(get_entity_name("labelcounter_prop")).value = value counter.update() except: logger.error('{}There was an error when updating the internal label counter {}.{}'.format( ERROR_PREFIX, counter[0], ERROR_SUFFIX)) -def get_bis_label_prefix(responsible_entity): - """ - @return The prefix for the BIS Label, i.e. the BIS lable without the container number. - """ - try: - if responsible_entity.get_parent("Person") is not None: - research_unit = db.execute_query("FIND ResearchUnit WITH id = {}".format( - responsible_entity.get_property("ResearchUnit").value))[0] - return "{}_{}_{}".format(get_bis_label_prefix(research_unit), get_abbreviation(responsible_entity.id), get_current_year()) - elif responsible_entity.get_parent("ResearchUnit") is not None: - research_division = db.execute_query("FIND ResearchDivision WITH id = {}".format( - responsible_entity.get_property("ResearchDivision").value))[0] - return "{}_{}".format(get_bis_label_prefix(research_division), get_abbreviation(responsible_entity.id)) - else: - return "{}".format(get_abbreviation(responsible_entity.id)) - except: - logger.error('{}The system could not auto-generate the BIS Label for {}.{}'.format( - ERROR_PREFIX, responsible_entity, ERROR_SUFFIX)) +def create_container_entities(responsible_id, container_type_id, container_size, number_of_containers, parent_container): + """Create the container entities specified via the "Register + Containers" form in the web interface. + Returns + ------- + conts : db.Container + A container of the newly registered container entities. -def create_container_entities(responsible_id, container_type_id, container_size, number_of_containers, parent_container): """ - Create the BIS Entities specified via the "Register Containers" Form in - ext_bis_stockmanagement.js. - @return A list of the newly registered entities.""" # Create Batch for insert batch = db.Container() # Get BIS Label for responsible person responsible_entity = db.execute_query("FIND {}".format(responsible_id))[0] - container_type_abbreviation = get_abbreviation(container_type_id) - bis_label_prefix = get_bis_label_prefix(responsible_entity) + bis_label_prefix = get_container_name_prefix(responsible_id, container_type_id) bis_label_counter = get_label_counter(container_type_id) for i in range(number_of_containers): - bis_label = "{}_{}_{}".format( - bis_label_prefix, container_type_abbreviation, bis_label_counter + i) + bis_label = f"{bis_label_prefix}_{bis_label_counter+i}" container = db.Record(bis_label) - container.add_parent(name="Container") - container.add_property(name="Responsible", value=responsible_id) - container.add_property(name="ContainerType", value=container_type_id) - container.add_property(name="Container size", value=container_size) - container.add_property(name="BIS label", value=bis_label) - container.add_property(name="Child container", value=[]) + container.add_parent(name=get_entity_name("container_rt")) + container.add_property(name=get_entity_name("responsible_rt"), value=responsible_id) + container.add_property(name=get_entity_name("containertype_rt"), value=container_type_id) + container.add_property(name=get_entity_name("container_size_prop"), value=container_size) + container.add_property(name=get_entity_name("child_container_prop"), value=[]) batch += [container] + + # Post process child containers + batch = post_process_containers(batch) try: batch.insert() - for ent in batch: - ent.retrieve_acl() - ent.grant(role='Stock Manager', priority=False, permission="EDIT:ACL") - ent.update_acl() - except: - logger.error("{}The registered containers could not be imported to the system.{}".format( + except Exception as e: + logger.error("{} The registered containers could not be imported to the system. {}".format( ERROR_PREFIX, ERROR_SUFFIX)) + logger.error(e) return None # TODO: Reset the internal_container_label_counter of person to its previous value if nothing has be inserted # Add to parent container if parent_container is not None: + child_cont_prop = db.Property(name=get_entity_name("child_container_prop")).retrieve() new_childs = [] for entity in batch: new_childs += [entity.id] # Check if property Child container exists: - if parent_container.get_property("Child container") is None: - child_cont_prop = db.Property(name="Child container").retrieve() + if parent_container.get_property(child_cont_prop.name) is None: parent_container.add_property(id=child_cont_prop.id, value=new_childs) else: # Check if the container allready has childs - if parent_container.get_property("Child container").value is None: - parent_container.get_property("Child container").value = new_childs + if parent_container.get_property(child_cont_prop.name).value is None: + parent_container.get_property(child_cont_prop.name).value = new_childs else: - parent_container.get_property("Child container").value += new_childs + parent_container.get_property(child_cont_prop.name).value += new_childs + parent_container = post_process_parent_container(parent_container) parent_container.update() set_label_counter(container_type_id=container_type_id, @@ -187,7 +160,10 @@ def get_parent_container(id): parent = db.execute_query("FIND {}".format(id))[0] return parent except: - logger.info("No parent container specified. The BIS ID or BIS label of the parent container has to be entered in the template before it is uploaded.") + logger.info( + "No parent container specified. The Contianer ID or label of the " + "parent container has to be entered in the template before it is uploaded." + ) return None @@ -204,21 +180,23 @@ def create_csv_template(template_internal_path, container_entities, container_ty if parent_container is None: parent_container_label = "" else: - parent_container_label = parent_container.get_property("BIS label").value + parent_container_label = get_container_identifier(parent_container) with open(template_internal_path, 'w') as template_csv: writer = csv.writer(template_csv) # Write header headers = [ - "BIS ID", - "Container type", - "Container size", - "BIS label", + "entity_id", + "container_type_rt", + "container_size_prop", + "container_label", "PI", "Collection", "Parent container", "Custom label", "Container Contents", - "PDFReport"] + "PDFReport" + ] + headers = [get_column_header_name(header) for header in headers] writer.writerow(headers) # Write description with leading '#' descriptions = get_description_row(headers) @@ -233,7 +211,7 @@ def create_csv_template(template_internal_path, container_entities, container_ty entity.id, container_type.name, container_size, - entity.get_property("BIS label").value, + get_container_identifier(entity), "", "", parent_container_label, @@ -287,8 +265,6 @@ def main(): template_name = get_template_name(container_entities, file_format_extension) template_display_path, template_internal_path = helper.get_shared_filename( template_name) - logger.info("Internal path: {}, Display path: {}".format( - template_internal_path, template_display_path)) if (file_format_extension == "csv"): create_csv_template( template_internal_path=template_internal_path, diff --git a/sample-management-custom/caosdb-server/scripting/bin/register_new_samples.py b/sample-management-custom/caosdb-server/scripting/bin/register_new_samples.py index 7724b577012ee3592a8de12a987b3f16054f6559..15e22bd9c17a67098519e38f6fc8f9c976f09718 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/register_new_samples.py +++ b/sample-management-custom/caosdb-server/scripting/bin/register_new_samples.py @@ -11,10 +11,13 @@ from caosadvancedtools.serverside.logging import configure_server_side_logging from bis_utils import (create_email_with_link_text, get_description_row, get_email_from_username, get_options_row, send_mail_with_defaults) +from sample_helpers.sample_registration_get_person_identifier import get_person_identifier +from sample_helpers.sample_registration_post_processing import post_process_samples +from sample_helpers.utils import CONSTANTS, get_column_header_name, get_entity_name + +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' -BIS_URL = 'https://localhost:10443/Entity/' logger = logging.getLogger("caosadvancedtools") @@ -23,58 +26,28 @@ def get_parser(): return par -def get_responsible_person_id(data): - return int(data["responsible_person"]) - - -def get_responsible_person_abbreviation(data): - person = db.execute_query("FIND {}".format(get_responsible_person_id(data)))[0] - return person.get_property("Abbreviation").value - - -def get_number_of_samples(data): - return int(data["number_of_samples"]) -# def is_samples_from_cruise(data): -# return ("samples_from_cruise" in data and data["samples_from_cruise"] == "on") -# def has_start_and_end_time(data): -# return ("start_and_end_time" in data and data["start_and_end_time"] == "on") -# def get_number_of_locations(data): -# number_of_locations = int(data["number_of_locations"]) -# if number_of_locations < 1: -# number_of_locations = 1 # At least one location is OBLIGATORY -# return number_of_locations - - -def get_reference_properties(data): - return data["reference_properties"] - - def get_column_names(data): # Make sure each header starts with BIS ID and Main User - starting_names = ["BIS ID", "Main User"] + starting_names = [get_column_header_name(name) for name in ["entity_id", "Main User"]] # required_column_names is just a string of column names separated by # commas. other_names = data["required_column_names"].split(',') + data["column_names"] starting_names.extend([name.strip() for name in other_names if name.strip() not in starting_names]) - return starting_names + return [get_column_header_name(name) for name in starting_names] def create_sample_entities(data): - responsible_person_id = get_responsible_person_id(data) - number_of_samples = get_number_of_samples(data) - add_state = len(db.execute_query("FIND 'Sample life-cycle'")) + responsible_person_id = int(data["responsible_person"]) + number_of_samples = int(data["number_of_samples"]) batch = db.Container() for index in range(number_of_samples): sample = db.Record() - sample.add_parent(name="Sample") - sample.add_property(name="Main User", value=responsible_person_id) - sample.add_property("Container") - sample.add_property("NagoyaCase") - if add_state: - sample.state = db.State(name="Registered", model="Sample life-cycle") + sample.add_parent(name=get_entity_name("Sample")) + sample.add_property(name=get_entity_name("Main User"), value=responsible_person_id) batch += [sample] + batch = post_process_samples(batch) batch.insert() return batch @@ -119,12 +92,8 @@ def read_input(args, log=False): def create_csv_template(template_name, samples, form_input): template_display_path, template_internal_path = helper.get_shared_filename(template_name) # Read form_input - responsible_person_abbreviation = get_responsible_person_abbreviation(form_input) - number_of_samples = get_number_of_samples(form_input) - # samples_from_cruise = is_samples_from_cruise(form_input) - # start_and_end_time = has_start_and_end_time(form_input) - # number_of_locations = get_number_of_locations(form_input) - # reference_properties = get_reference_properties(form_input) + responsible_person_abbreviation = get_person_identifier(form_input) + number_of_samples = int(form_input["number_of_samples"]) header = get_column_names(form_input) with open(template_internal_path, 'w') as csv_template: diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_name.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_name.py new file mode 100644 index 0000000000000000000000000000000000000000..efcab85761fe841ff23dec3be219d62fcee3f3e7 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_name.py @@ -0,0 +1,67 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +"""This module serves a function that generates a human-readable +container name or label. + +""" + +import linkahead as db + +from .utils import get_entity_name + + +def get_record_string(rec: db.Record) -> str: + + if rec.get_property(get_entity_name("abbreviation_prop")) is not None and rec.get_property(get_entity_name("abbreviation_prop")).value: + return rec.get_property(get_entity_name("abbreviation_prop")).value + + if rec.name: + return rec.name + + return "" + + +def get_container_name_prefix(responsible_id: int, container_type_id: int) -> str: + """Generate a container name prefix from the container type and + the responsible entity. + + Parameters + ---------- + responsible_id : int + LinkAhead id of the responsible entity (person, research + institution, ...) + container_type_id: int + LinkAhead id of the ContainerType Record corresponding to the + type of the given container. + + Returns + ------- + container_name_prefix : str + Name in the format `<responsible>_<container_type_name>` + + """ + resp = db.cached.cached_get_entity_by(eid=responsible_id) + ct = db.cached.cached_get_entity_by(eid=container_type_id) + + resp_str = get_record_string(resp) + ct_str = get_record_string(ct) + + if resp_str: + return f"{resp_str}_{ct_str}" + + return ct_str diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_identifier.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_identifier.py new file mode 100644 index 0000000000000000000000000000000000000000..52943d77a2ac4189120a8161d0f61cb81a090add --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_identifier.py @@ -0,0 +1,43 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from .utils import get_entity_name + + +def get_container_identifier(cont: db.Record) -> str: + """Return an identifier string that uniquely identifies the given + container entity in a registration/upload/export table file. + + Parameters + ---------- + cont : db.Record + The container record to be identified. + + Returns + ------- + identifier : str + The identifying string that uniquly identifies the given + container. + """ + if (cont.get_property(get_entity_name("container_label_prop")) is not None and + cont.get_property(get_entity_name("container_label_prop")).value): + return cont.get_property(get_entity_name("container_label_prop")).value + elif cont.name: + return cont.name + return cont.id diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_post_processing.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_post_processing.py new file mode 100644 index 0000000000000000000000000000000000000000..8ab70ba108910651decd80d7a5b34a9b60e95edd --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_registration_post_processing.py @@ -0,0 +1,53 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + + +def post_process_containers(conts: db.Container): + """Dummy function to be overwritten by custom code if desired. + + Parameters + ---------- + conts : db.Container + Container of container entities to be post-processed. + + Returns + ------- + conts : db.Container + Container of post-processed container entities to be inserted. + """ + + return conts + + +def post_process_parent_container(parent: db.Record): + """Dummy function to be overwritten by custom code if desired. + + Parameters + ---------- + parent : db.Record + parent container entitiy to be post-processed. + + Returns + ------- + parent : db.Record + Post-processed parent container entity to be + updated. + """ + + return parent diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_get_parent.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_get_parent.py new file mode 100644 index 0000000000000000000000000000000000000000..4d11a3b74453298e13b2185ac4fbcf5650f2bb0f --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_get_parent.py @@ -0,0 +1,35 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from .utils import get_entity_name + + +def get_container_by_identifier(identifier: str) -> db.Record: + """Return a container identified by the given string identifier.""" + # First try (integer) ID + try: + identifier = int(identifier) + query = f"FIND '{get_entity_name('container_rt')}' WITH ID={identifier}" + except ValueError: + # Then fall back to name or label + query = (f"FIND '{get_entity_name('container_rt')}' WITH " + f"name='{identifier}' OR WITH " + f"'{get_entity_name('container_label_prop')}'='{identifier}'") + + return db.cached.cached_query(query) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_post_processing.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_post_processing.py new file mode 100644 index 0000000000000000000000000000000000000000..e8f696b7c6fbe6352fe618d37a138ac796614cd3 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/container_update_post_processing.py @@ -0,0 +1,33 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + + +def post_process_containers_before_sync(child_containers): + + return child_containers + + +def post_process_parent_containers_before_sync(parent_containers): + + return parent_containers + + +def post_process_inserts_and_updates(inserts, updates): + + return inserts, updates diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/default_constants.yml b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/default_constants.yml new file mode 100644 index 0000000000000000000000000000000000000000..c575cb61f4cbd50676d8b944f7af7631963907bf --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/default_constants.yml @@ -0,0 +1,79 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +csv_column_names: + entity_id: "LinkAhead ID" + custom_label_prop: "Custom Label" + start_date_prop: "Start date" + end_date_prop: "End date" + igsn_doi_prop: "IGSN DOI" + locality_description_prop: "Locality description" + locality_name_prop: "Locality name" + responsible_person_event: "Event responsible" + parent_sample_prop: "Parent LinkAhead ID" + +csv_column_descriptions: + LinkAhead ID: "An ID generated by LinkAhead (either integer or URL to this entity). Do not change this column!" + container_label: "The label automatically assigned by LinkAhead sample management. Do not change this column." + Collection: "Collection(s) to which this container or samples belongs (separated by ';')" + Container Contents: "Notes concerning the contents of this container" + Container size: "The size of the container (e.g 81 spaces)" + Container type: "The type of container. Do not change this column." + Custom label: "Any additional custom label you put on the container" + Date start: "The start date or datetime in ISO format" + Date stop: "The end date or datetime in ISO format" + Gear configuration: "How gear is configured, meshsize of the net, filter pore size, what kind of trawl" + Latitude start: "Latitude where sampling started; in decimal degrees,use \".\" as decimal sign, use - for S" + Latitude stop: "Latitude where sampling ended; in decimal degrees,use \".\" as decimal sign, use - for S" + Longitude start: "Longitude where sampling started; in decimal degrees,use \".\" as decimal sign, use - for W" + Longitude stop: "Longitude where sampling ended; in decimal degrees,use \".\" as decimal sign, use - for W" + Notes / remarks: "Field for notes" + PDFReport: "A PDF containing additional information on the container or sampe contents" + PI: "The PI (by abbreviation) of the container or sample" + Parent container: "The LinkAhead ID (integer or URL) or name or BIS label of the container where this container is stored." + Sampling depth start: "The depth in meters where the sampling started as a positive value, not bottom depth" + Sampling depth stop: "The depth in meters where the sampling ended as a positive value,not bottom depth" + Storage ID: "LinkAhead ID of the Container that holds this sample" + Subevent: "Subevent given by D-Ship in Stationlog. Expl. AL123_14-4" + Timezone: "Timezone: Either UTC, CET, .... or in the form +hh[:mm], -hh:[mm]." + Water depth start: "The bottom depth in meters where the sampling started as a positive value" + Water depth stop: "The bottom depth in meters where the sampling ended as a positive value" + +entity_names: + abbreviation_prop: abbreviation + child_container_prop: "Child_Container" + container_label_prop: container_label + container_rt: Container + container_size_prop: "Container size" + containertype_rt: ContainerType + custom_label_prop: custom_label + end_date_prop: end_date + event_rt: Event + first_name_prop: first_name + igsn_doi_prop: igsn_doi + labelcounter_prop: counter + labelcounter_rt: LabelCounter + last_name_prop: last_name + parent_sample_prop: Parent_sample + responsible_rt: Responsible + start_date_prop: start_date + locality_description_prop: locality_description + locality_name_prop: locality_name + responsible_person_event: Responsible_Person + +error_prefix: "Something went wrong:" +error_suffix: "Please contact your administrator(s)." diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_get_person_identifier.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_get_person_identifier.py new file mode 100644 index 0000000000000000000000000000000000000000..396bf0ef054275f901fb6fa6056dc95e6a624516 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_get_person_identifier.py @@ -0,0 +1,49 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from .utils import get_entity_name + + +def get_person_identifier(form_data: dict) -> str: + """Retrieve a person record from the given id, and replace by + abbreviation if present, else "last name, first name". + + """ + person_rec = db.cached.cached_get_entity_by(eid=form_data["responsible_person"]) + + return get_person_identifier_from_rec(person_rec) + + +def get_person_identifier_from_rec(person_rec: db.Record) -> str: + # Use abbreviation if present + if (person_rec.get_property(get_entity_name("abbreviation_prop")) is not None and + person_rec.get_property(get_entity_name("abbreviation_prop")).value): + return person_rec.get_property(get_entity_name("abbreviation_prop")).value + first_name = person_rec.get_property(get_entity_name("first_name_prop")).value if person_rec.get_property( + get_entity_name("first_name_prop")) is not None else "" + last_name = person_rec.get_property(get_entity_name("last_name_prop")).value if person_rec.get_property( + get_entity_name("last_name_prop")) is not None else "" + # Else use first and last name ... + if first_name and last_name: + return f"{last_name}, {first_name}" + # ... or just last name if no first name + if last_name: + return last_name + # Last fallback id + return person_rec.id diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_post_processing.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_post_processing.py new file mode 100644 index 0000000000000000000000000000000000000000..6d8fe9d623dc644423006b00a1b271330ffb5871 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_registration_post_processing.py @@ -0,0 +1,26 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + + +def post_process_samples(samples: db.Container) -> db.Container: + """Post process the container of samples to be registerd before + they are inserted. Dummy function to be overwritten. + + """ + return samples diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_add_special_properties.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_add_special_properties.py new file mode 100644 index 0000000000000000000000000000000000000000..9ec2fffef113f842440b1907f153b0e01cbe5050 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_add_special_properties.py @@ -0,0 +1,78 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db +import pandas as pd + +from caosadvancedtools.datainconsistency import DataInconsistencyError + +from .sample_upload_get_container import get_container +from .sample_upload_get_device import get_device +from .sample_upload_get_person import get_person +from .utils import get_column_header_name, get_entity_name, update_property, return_value_if_not_none + + +def add_special_properties(sample: db.Record, data: pd.Series) -> db.Record: + """This function adds all the properties that can not be added + trivially from the given data row to the sample. + + """ + + main_user_prop = db.get_entity_by_name(get_entity_name("Main User")) + pi_prop = db.get_entity_by_name(get_entity_name("PI")) + container_rt = db.get_entity_by_name(get_entity_name("container_rt")) + device_rt = db.get_entity_by_name(get_entity_name("Device")) + parent_sample_prop = db.get_entity_by_name(get_entity_name("parent_sample_prop")) + + if (get_column_header_name("PI") in data and + return_value_if_not_none(data[get_column_header_name("PI")]) is not None): + pi_user = get_person(return_value_if_not_none(data[get_column_header_name("PI")])) + sample = update_property(sample, pi_prop.id, pi_user, property_name=pi_prop.name) + + if (get_column_header_name("Main User") in data and + return_value_if_not_none(data[get_column_header_name("Main User")]) is not None): + main_user = get_person(return_value_if_not_none(data[get_column_header_name("Main User")])) + sample = update_property(sample, main_user_prop.id, main_user, property_name=main_user_prop.name) + + if (get_column_header_name("Storage ID") in data and + return_value_if_not_none(data[get_column_header_name("Storage ID")]) is not None): + container = get_container(return_value_if_not_none( + data[get_column_header_name("Storage ID")])) + sample = update_property(sample, container_rt.id, container, property_name=container_rt.name) + + if (get_column_header_name("Device") in data and + return_value_if_not_none(data[get_column_header_name("Device")]) is not None): + device = get_device(return_value_if_not_none( + data[get_column_header_name("Device")])) + sample = update_property(sample, device_rt.id, device, property_name=device_rt.name) + + if (get_column_header_name("Parent LinkAhead ID") in data and + return_value_if_not_none(data[get_column_header_name("Parent LinkAhead ID")]) is not None): + parent = return_value_if_not_none( + data[get_column_header_name("Parent LinkAhead ID")]) + sample = update_property(sample, parent_sample_prop.id, parent, property_name=parent_sample_prop.name) + + for name in ["Curator", "Collector"]: + prop = db.get_entity_by_name(get_entity_name(name)) + if (get_column_header_name(name) in data and + return_value_if_not_none(data[get_column_header_name(name)]) is not None): + persons = [get_person(id_str) for id_str in return_value_if_not_none( + data[get_column_header_name(name)])] + sample = update_property(sample, prop.id, persons, + datatype=db.LIST(get_entity_name("Person")), property_name=prop.name) + + return sample diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_column_definitions.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_column_definitions.py new file mode 100644 index 0000000000000000000000000000000000000000..058452266722257db5023f076031d2cc570a54dd --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_column_definitions.py @@ -0,0 +1,131 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import re + +from datetime import date +from dateutil.relativedelta import relativedelta +from typing import Union + +from .utils import get_column_header_name + + +def semicolon_separated_list(text): + return [el.strip() for el in text.split(";") if el != ""] + + +def _embargo_converter(text: str): + + datepattern = r"^(?P<year>\d{4,4})-(?P<month>\d{2,2})-(?P<day>\d{2,2})" + matches = re.match(datepattern, str(text)) + if matches: + return date(int(matches.groupdict()["year"]), int(matches.groupdict()["month"]), int(matches.groupdict()["day"])) + if f"{text}".lower() in ["true", "yes"]: + # yes means embargo until today in one year + return date.today() + relativedelta(years=1) + if f"{text}".lower() in ["false", "no"]: + return "" + raise ValueError( + f"The embargo should be either a date in YYYY-MM-DD format, or 'true'/'yes' or 'false'/'no', but is {text}.") + + +def use_custom_names(definition: Union[list, dict]): + """Replace names in list or dict keys by custom names with + `utils.get_column_header_name`. + + """ + if isinstance(definition, list): + return [get_column_header_name(name) for name in definition] + elif isinstance(definition, dict): + return {get_column_header_name(key): value for key, value in definition.items()} + + raise ValueError(f"Expected dict or list, but got {type(definition)}.") + + +DATATYPE_DEFINITIONS = use_custom_names({ + "entity_id": str, + "Main User": str, + "Campaign": str, + "Elevation start": float, + "Elevation stop": float, + "End date": str, + "Latitude start": float, + "Latitude stop": float, + "Longitude start": float, + "Longitude stop": float, + "PI": str, + "Parent LinkAhead ID": str, + "SampleMethod": str, + "SampleType": str, + "Start date": str, + "Storage ID": str, +}) + +# Obligatory columns: Must exist and must not be empty +# Must exist +OBLIGATORY_COLUMNS = use_custom_names([ + "entity_id", + "Start date", +]) + +OBLIGATORY_COLUMNS_CHILD = use_custom_names([ + "entity_id", + "Parent LinkAhead ID", +]) + +COLUMN_CONVERTER = use_custom_names({ + "Collector": semicolon_separated_list, + "Curator": semicolon_separated_list, + "Embargo": _embargo_converter, + "Event responsible": semicolon_separated_list, + "Sphere": semicolon_separated_list, +}) + +SPECIAL_TREATMENT_SAMPLE = use_custom_names([ + "Biome", + "Campaign", + "Collector", + "Curator", + "Device", + "Elevation start", + "Elevation stop", + "Embargo", + "End date", + "responsible_person_event", + "igsn_doi_prop", + "Latitude start", + "Latitude stop", + "Level", + "locality_description_prop", + "locality_name_prop", + "Longitude start", + "Longitude stop", + "Main User", + "PI", + "PDFReport", + "parent_sample_prop", + "Sphere", + "Start date", + "Storage ID", + "entity_id", +]) + +IGNORED_COLUMN_NAMES_SAMPLE = use_custom_names([ + "LinkAhead URL", + "Parent_sample", + "Storage chain", +]) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_container.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_container.py new file mode 100644 index 0000000000000000000000000000000000000000..40e2e735fa97fafced6410403c19b57decd19ebb --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_container.py @@ -0,0 +1,43 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from caosadvancedtools.datainconsistency import DataInconsistencyError + +from .utils import get_entity_name + + +def get_container(identifier: str) -> db.Record: + """Return the container record for a given identifier string. By + default, that's simply the id but may be overwritten for + customization. + + """ + try: + return db.cached.cached_get_entity_by( + query=f"FIND '{get_entity_name('container_rt')}' WITH ID='{identifier}'" + ) + except db.EmptyUniqueQueryError: + try: + return db.cached.cached_get_entity_by( + query=f"FIND '{get_entity_name('container_rt')}' WITH name='{identifier}'" + ) + except db.EmptyUniqueQueryError: + raise DataInconsistencyError( + f"Couldn't find a {get_entity_name('container_rt')} with name or id {identifier}." + ) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_device.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_device.py new file mode 100644 index 0000000000000000000000000000000000000000..08cf3f0ac5b3148361fc8517cdbcbabf6f48e63e --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_device.py @@ -0,0 +1,42 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from caosadvancedtools.datainconsistency import DataInconsistencyError + +from .utils import get_entity_name + +def get_device(identifier: str) -> db.Record: + """Return the container record for a given identifier string. By + default, that's simply the id but may be overwritten for + customization. + + """ + try: + return db.cached.cached_get_entity_by( + query=f"FIND '{get_entity_name('Device')}' WITH ID='{identifier}'" + ) + except db.EmptyUniqueQueryError: + try: + return db.cached.cached_get_entity_by( + query=f"FIND '{get_entity_name('Device')}' WITH name='{identifier}'" + ) + except db.EmptyUniqueQueryError: + raise DataInconsistencyError( + f"Couldn't find a {get_entity_name('Device')} with name or id {identifier}." + ) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_event.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_event.py new file mode 100644 index 0000000000000000000000000000000000000000..c15df79805965ec982012775a113704979ecdf1b --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_event.py @@ -0,0 +1,148 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db +import pandas as pd + +from caosadvancedtools.datainconsistency import DataInconsistencyError + +from .sample_upload_get_person import get_person +from .utils import get_column_header_name, get_entity_name, update_property, return_value_if_not_none + + +def add_event_to_sample(sample: db.Record, data: pd.Series) -> db.Record: + """Take a given sample and attach an event Record from the + data. Then, return the sample. + + """ + + _perform_sanity_checks(sample, data) + event_rt = db.get_entity_by_name(get_entity_name("event_rt"), role="RECORDTYPE") + event = db.Record().add_parent(event_rt) + + # We performed the sanity checks so we can assume that if the + # Start/Stop Latitude exists, all start/stop data exist. + positions = [] + position_prop = db.get_entity_by_name(get_entity_name("Position")) + + # Take care of positions + for mode in ["start", "stop"]: + if (get_column_header_name(f"Latitude {mode}") in data and + return_value_if_not_none(data[get_column_header_name(f"Latitude {mode}")]) is not None): + positions.append( + _create_position( + mode=mode, + lat=return_value_if_not_none(data[get_column_header_name(f"Latitude {mode}")]), + lng=return_value_if_not_none(data[get_column_header_name(f"Longitude {mode}")]), + ele=return_value_if_not_none(data[get_column_header_name(f"Elevation {mode}")]), + ) + ) + if positions: + event = update_property(event, position_prop.id, positions, + datatype=db.LIST(get_entity_name("Position")), property_name=position_prop.name) + + # Further event properties + for event_p in ["Biome", "Campaign", "Device", "EventType", + "igsn_doi_prop", "level", "locality_description_prop", + "locality_name_prop", "Sphere"]: + prop = db.get_entity_by_name(get_entity_name(event_p)) + if (get_column_header_name(event_p) in data and + return_value_if_not_none(data[get_column_header_name(event_p)]) is not None): + event = update_property(event, prop.id, return_value_if_not_none( + data[get_column_header_name(event_p)]), property_name=prop.name) + + # Special treatment for (list of) persons + resp_prop = db.get_entity_by_name(get_entity_name("responsible_person_event")) + if (get_column_header_name("responsible_person_event") in data and + return_value_if_not_none(data[get_column_header_name("responsible_person_event")]) is not None): + resps = return_value_if_not_none(data[get_column_header_name("responsible_person_event")]) + if not isinstance(resps, list): + resps = [resps] + recs = [get_person(resp) for resp in resps] + event = update_property(event, resp_prop.id, recs, datatype=db.LIST( + "Person"), property_name=resp_prop.name) + + for time_p in ["start_date_prop", "end_date_prop"]: + prop = db.get_entity_by_name(get_entity_name(time_p)) + if (get_column_header_name(time_p) in data and + return_value_if_not_none(data[get_column_header_name(time_p)]) is not None): + event = update_property(event, prop.id, return_value_if_not_none( + data[get_column_header_name(time_p)]), property_name=prop.name) + + # only add if there was any event data at all: + if len(event.properties) > 0: + sample = update_property(sample, event_rt.id, event, property_name=event_rt.name) + + return sample + + +def _create_position(mode: str, lat: float, lng: float, ele: float): + pos = db.Record() + if mode.lower() == "start": + pos.add_parent(get_entity_name("StartPosition")) + elif mode.lower() == "stop": + pos.add_parent(get_entity_name("StopPosition")) + else: + pos.add_parent(get_entity_name("Position")) + + pos.add_property(name=get_entity_name("latitude"), value=lat) + pos.add_property(name=get_entity_name("longitude"), value=lng) + pos.add_property(name=get_entity_name("elevation"), value=ele) + + return pos + + +def _perform_sanity_checks(sample, data): + + if (get_column_header_name("end_date") in data and + return_value_if_not_none(data[get_column_header_name("end_date")]) is not None): + if (get_column_header_name("start_date") not in data or + return_value_if_not_none(data[get_column_header_name("start_date")]) is None): + raise DataInconsistencyError( + f"Sample with {get_entity_name('entity_id')} {sample.id} has a " + f"{get_column_header_name('end_date')} but no valid " + f"{get_column_header_name('start_date')}." + ) + + for name in ["start", "stop"]: + bool_list = [get_column_header_name(f"{val}_{name}") in data for val in [ + "latitude", "longitude", "elevation"]] + raise_error = False + if any(bool_list): + if not all(bool_list): + raise_error = True + elif any([return_value_if_not_none(data[get_column_header_name(f"{val}_{name}")]) is None for val in ["latitude", "longitude", "elevation"]]): + raise_error = True + + if raise_error: + raise DataInconsistencyError( + f"Sample with {get_entity_name('entity_id')} {sample.id} has an " + f"invalid {name} position. Please make sure that latitude, longitude, " + "and elevation are provided." + ) + + # only need to check lat since we already checked that if lat is + # present, lng and ele are present, too + if (get_column_header_name("latitude_stop") in data and + return_value_if_not_none(data[get_column_header_name("latitude_stop")]) is not None): + if (get_column_header_name("latitude_start") not in data or + return_value_if_not_none(data[get_column_header_name("latitude_start")]) is not None): + raise DataInconsistencyError( + f"Sample with {get_entity_name('entity_id')} {sample.id} has a " + f"{get_entity_name('StopPosition')} but no valid " + f"{get_entity_name('StartPosition')}." + ) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_person.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_person.py new file mode 100644 index 0000000000000000000000000000000000000000..3e947804d60d10e20a6cd05219c02861b0c067a0 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_get_person.py @@ -0,0 +1,79 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db + +from caosadvancedtools.datainconsistency import DataInconsistencyError + +from .utils import get_entity_name + +from linkahead.cached import cached_get_entity_by, cached_query as cquery + +def get_person(text: str) -> db.Record: + """Return the Person Record that is specifed as 'Main User' or + 'Sampling Person' in data. + + If the Person is not present in the database, an Exception is + raised. Creating new Person Reconrds can only be done by a + priviledged user. + + """ + # Check in which format the person is identified: + person_identifier = text.split(", ") + if len(person_identifier) == 1: + person = _get_person_by_id(person_identifier[0]) + else: + person = _get_person_by_fullname(person_identifier[1], person_identifier[0]) + + return person + + +def _get_person_by_fullname(first_name: str, last_name: str) -> db.Record: + # seach for person in db + res = db.execute_query( + f"FIND '{get_entity_name('Person')}' WITH " + f"'{get_entity_name('first_name_prop')}'='{first_name}' " + f"AND '{get_entity_name('last_name_prop')}'='{last_name}'" + ) + # if person doesn't exist in database... + if len(res) == 0: + # There is not enought data in the template to create a new Person record. Hence, we have to raise an Exception + error_msg = ( + f"There is no person Record with '{get_entity_name('first_name_prop')}' " + f"= '{first_name}' " + f"AND '{get_entity_name('last_name_prop')}' = '{last_name}' in the database. ") + raise DataInconsistencyError(error_msg) + else: + return res[0] + + +def _get_person_by_id(ident: str) -> db.Record: + options = cquery(f"FIND '{get_entity_name('Person')}' WITH ID='{ident}' ") + if len(options)==1: + return options[0] + options = cquery(f"FIND '{get_entity_name('Person')}' WITH " + f"'{get_entity_name('abbreviation_prop')}'='{ident}' ") + if len(options) == 1: + return options[0] + elif len(options) == 0: + raise DataInconsistencyError( + f"Could not find a {get_entity_name('Person')} with ID {ident}." + ) + else: + raise DataInconsistencyError( + f"Found multiple {get_entity_name('Person')} with {get_entity_name('abbreviation_prop')}={ident}." + ) diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_post_processing.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_post_processing.py new file mode 100644 index 0000000000000000000000000000000000000000..54a034775389a16d5dc0d02d57195b37dcd94ca8 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/sample_upload_post_processing.py @@ -0,0 +1,23 @@ +# +# Copyright (C) 2025 Indiscale GmbH <info@indiscale.com> +# Copyright (C) 2025 Florian Spreckelsen <f.spreckelsen@indiscale.com> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as +# published by the Free Software Foundation, either version 3 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <https://www.gnu.org/licenses/>. +# +import linkahead as db +import pandas as pd + +def post_process_samples(samples: db.Container, data: pd.DataFrame) -> db.Container: + + return samples diff --git a/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/utils.py b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/utils.py new file mode 100644 index 0000000000000000000000000000000000000000..d047e6bbfd6a39cae610526d17238d3675533447 --- /dev/null +++ b/sample-management-custom/caosdb-server/scripting/bin/sample_helpers/utils.py @@ -0,0 +1,79 @@ +import os +import yaml + +from pathlib import Path + +import linkahead as db +import pandas as pd + +with open(os.path.join(Path(__file__).parent, "default_constants.yml")) as yaml_file: + CONSTANTS = yaml.safe_load(yaml_file) + +if os.path.isfile(os.path.join(Path(__file__).parent, "constants.yml")): + with open(os.path.join(Path(__file__).parent, "constants.yml")) as custom_yaml_file: + CONSTANTS.update(yaml.safe_load(custom_yaml_file)) + + +def get_entity_name(name: str): + + if name in CONSTANTS["entity_names"]: + + return CONSTANTS["entity_names"][name] + + return name + + +def get_column_header_name(name: str): + + if name in CONSTANTS["csv_column_names"]: + return CONSTANTS["csv_column_names"][name] + + return get_entity_name(name) + + +def update_property(entity: db.Record, property_id: int, value, property_name="", datatype=None): + """ + Set the property of an entity. + + If the entity already has the property, just the value is set. + Else, the property is added to the entity + """ + # TODO: Replace by assure_property_is etc. + + # If the value in the spreadsheet is empty (nan) + if ((isinstance(value, list) and len(value) == 0) + or (not isinstance(value, list) and pd.isna(value))): + # Remove the property from te Entity if it has it + try: + entity.get_properties().get_by_name(property_name) + entity.remove_property(property_name) + except KeyError: + pass + return entity + if entity.get_property(property_id) is None: + if datatype: + entity.add_property(id=property_id, value=value, name=property_name, datatype=datatype) + else: + entity.add_property(id=property_id, value=value, name=property_name) + else: + if isinstance(value, list) and not entity.get_property(property_id).datatype.startswith("LIST"): + entity.get_property(property_id).datatype = db.LIST( + entity.get_property(property_id).datatype) + entity.get_property(property_id).value = value + return entity + + +def return_value_if_not_none(val): + """Workaround for somewhat weird pandas behavior. Return value if actual + value, otherwise, return None. + + """ + if isinstance(val, list): + if len(val) == 0: + return None + if len(val) == 1: + if pd.isnull(val[0]) or val[0] is None or f"{val[0]}".lower() == "nan" or f"{val[0]}" == "": + return None + elif pd.isnull(val) or val is None or f"{val}".lower() == "nan" or f"{val}" == "": + return None + return val diff --git a/sample-management-custom/caosdb-server/scripting/bin/update_containers.py b/sample-management-custom/caosdb-server/scripting/bin/update_containers.py index 6334551716ec80205b3765d81907f71e69a7026d..4a03f1d5bec3207a17d971d98fd96a42722e2070 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/update_containers.py +++ b/sample-management-custom/caosdb-server/scripting/bin/update_containers.py @@ -36,13 +36,17 @@ from linkahead.cached import cached_query, cached_get_entity_by from bis_utils import (get_do_not_insert_type_names, replace_entity_urls_by_ids, whitespace_cleanup_in_df) +from sample_helpers.container_update_get_parent import get_container_by_identifier +from sample_helpers.container_update_post_processing import ( + post_process_containers_before_sync, post_process_parent_containers_before_sync, post_process_inserts_and_updates) +from sample_helpers.utils import (CONSTANTS, get_column_header_name, get_entity_name) # suppress warning of diff function apilogger = logging.getLogger("linkahead.apiutils") apilogger.setLevel(logging.ERROR) -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] logger = logging.getLogger("caosadvancedtools") @@ -55,17 +59,6 @@ def _value_in_row(key, row): return True -def _get_parent_by_identifier(parent_identifier): - """Get parent specified either by BIS ID, name, or BIS label.""" - try: - parent_identifier = int(parent_identifier) - query = f"FIND Container WITH ID={parent_identifier}" - except ValueError: - query = (f"FIND Container WITH name='{parent_identifier}' " - f"OR WITH 'BIS label'='{parent_identifier}'") - return cached_query(query) - - def get_parser(): par = helper.get_argument_parser() return par @@ -96,29 +89,35 @@ def main(): # Get referenced container entities child_containers = db.Container() parent_containers = db.Container() + id_column_name = get_column_header_name("entity_id") + parent_column_name = get_column_header_name("Parent container") for index, row in data.iterrows(): - if not _value_in_row("BIS ID", row): - logger.error(f"BIS ID is missing in row {index+1}. Nothing was updated.") + if not _value_in_row(id_column_name, row): + logger.error(f"{id_column_name} is missing in row {index+1}. Nothing was updated.") return 1 try: - child = db.Record(id=int(row["BIS ID"])) + child = db.Record(id=int(row[id_column_name])) except ValueError: logger.error( - f"Invalid BIS ID {row['BIS ID']} in row {index + 1}. Nothing was updated.") + f"Invalid {id_column_name} {row[id_column_name]} in row {index + 1}. Nothing was updated.") return 1 - child.add_parent("Container") + child.add_parent(get_entity_name("container_rt")) child_containers.append(child) - if _value_in_row("Parent container", row): - parent_identifier = row["Parent container"] - parent = _get_parent_by_identifier(parent_identifier) + if _value_in_row(parent_column_name, row): + parent_identifier = row[parent_column_name] + parent = get_container_by_identifier(parent_identifier) if len(parent) == 0: logger.error( - f"Couldn't find parent with identifier '{parent_identifier}' in row {index+1}.") + f"Couldn't find parent with identifier '{parent_identifier}' " + f"in row {index+1}." + ) return 1 elif len(parent) > 1: - logger.error(f"Parent with identifier '{parent_identifier}' in row {index+1} was not unique. " - "Please specify with BIS ID instead.") + logger.error( + f"Parent with identifier '{parent_identifier}' in row " + f"{index+1} was not unique. Please specify with " + f"{id_column_name} instead.") return 1 parent = parent[0] try: @@ -132,34 +131,44 @@ def main(): return 1 # Get IDs of proerperties - child_container_prop = cached_get_entity_by(query="FIND Property WITH name = 'Child container'") - custom_label_prop = cached_get_entity_by(query="FIND Property WITH name = 'Custom label'") - pdf_rt = cached_get_entity_by(query="FIND RECORDTYPE WITH name=PDFReport") + child_container_prop = cached_get_entity_by( + query=f"FIND Property WITH name = '{get_entity_name('child_container_prop')}'" + ) + custom_label_prop = cached_get_entity_by( + query=f"FIND Property WITH name = '{get_entity_name('custom_label_prop')}'" + ) + pdf_rt = cached_get_entity_by( + query=f"FIND RECORDTYPE WITH name='{get_entity_name('PDFReport')}'" + ) # Update (/create) container entities for index, row in data.iterrows(): # Add child to parent parent = None - if _value_in_row("Parent container", row): - parent_identifier = row["Parent container"] + if _value_in_row(parent_column_name, row): + parent_identifier = row[parent_column_name] # This has already been checked above for uniqueness - candidate = _get_parent_by_identifier(parent_identifier)[0] + candidate = get_container_by_identifier(parent_identifier)[0] # A bit redundant, but we need the exact Python object here that is in the parent_containers list. parent = parent_containers.get_entity_by_id(candidate.id) if parent.get_property(child_container_prop.id) is None: parent.add_property(id=child_container_prop.id, - name=child_container_prop.name, value=[int(row["BIS ID"])]) + name=child_container_prop.name, value=[int(row[id_column_name])]) else: if parent.get_property(child_container_prop.id).value is None: - parent.get_property(child_container_prop.id).value = [int(row["BIS ID"])] + parent.get_property(child_container_prop.id).value = [int(row[id_column_name])] else: - if int(row["BIS ID"]) not in parent.get_property(child_container_prop.id).value: + if int(row[id_column_name]) not in parent.get_property(child_container_prop.id).value: parent.get_property(child_container_prop.id).value.append( - int(row["BIS ID"])) - - # remove the current child from all other parents (don't do anything if the parent didn't change) - old_parents = cached_query(f"FIND Container WHICH REFERENCES {int(row['BIS ID'])}") + int(row[id_column_name])) + + # remove the current child from all other parents (don't + # do anything if the parent didn't change) + old_parents = cached_query( + f"FIND {get_entity_name('container_rt')} WHICH REFERENCES " + f"{int(row[id_column_name])}" + ) for old_parent in old_parents: if parent is not None and old_parent.id == parent.id: # old parent also is new parent @@ -169,29 +178,30 @@ def main(): old_parent = parent_containers.get_entity_by_id(old_parent.id) except KeyError: parent_containers.append(old_parent) - old_parent.remove_value_from_property("Child container", int( - row["BIS ID"]), remove_if_empty_afterwards=False) - if old_parent.get_property("Child container").value is None: - old_parent.get_property("Child container").value = [] + old_parent.remove_value_from_property(child_container_prop.name, int( + row[id_column_name]), remove_if_empty_afterwards=False) + if old_parent.get_property(child_container_prop.name).value is None: + old_parent.get_property(child_container_prop.name).value = [] # Add custom label o child - child = child_containers.get_entity_by_id(id=int(row["BIS ID"])) - if _value_in_row("Custom label", row): - child.name = row["Custom label"] + child = child_containers.get_entity_by_id(id=int(row[id_column_name])) + if _value_in_row(get_column_header_name("custom_label_prop"), row): + child.name = row[get_column_header_name("custom_label_prop")] if child.get_property(custom_label_prop.id) is None: child.add_property(id=custom_label_prop.id, - name=custom_label_prop.name, value=row["Custom label"]) + name=custom_label_prop.name, value=row[get_column_header_name("custom_label_prop")]) else: - child.get_property(custom_label_prop.id).value = row["Custom label"] + child.get_property( + custom_label_prop.id).value = row[get_column_header_name("custom_label_prop")] # Treat PI - if _value_in_row("PI", row): - pi = row["PI"] - pi_prop = cached_get_entity_by(query="FIND PROPERTY Pi") + if _value_in_row(get_column_header_name("PI"), row): + pi = row[get_column_header_name("PI")] + pi_prop = cached_get_entity_by(query=f"FIND PROPERTY {get_entity_name('PI')}") try: query = f"FIND RECORD Person WITH ID={int(pi)}" except ValueError: - query = f"FIND RECORD Person WITH AN Abbreviation='{pi}'" + query = f"FIND RECORD Person WITH AN '{get_entity_name('abbreviation_prop')}'='{pi}'" try: pi_rec = cached_get_entity_by(query=query) if child.get_property(pi_prop.name) is not None: @@ -199,67 +209,63 @@ def main(): else: child.add_property(id=pi_prop.id, name=pi_prop.name, value=pi_rec.id) except db.EmptyUniqueQueryError: - logger.warning(f"There is no PI with BIS ID or abbreviation {pi}. Skipping.") + logger.warning(f"There is no PI with ID or {get_entity_name('abbreviation_prop')} {pi}. Skipping.") # Collection(s) - if _value_in_row("Collection", row): - collection_rt = cached_get_entity_by(query="FIND RECORDTYPE Collection") - if not ";" in str(row["Collection"]): - collections = [row["Collection"]] + if _value_in_row(get_column_header_name("Collection"), row): + collection_rt = cached_get_entity_by(query=f"FIND RECORDTYPE '{get_entity_name('Collection')}'") + if not ";" in str(row[get_column_header_name("Collection")]): + collections = [row[get_column_header_name("Collection")]] else: - collections = [coll.strip() for coll in str(row["Collection"]).split(';')] + collections = [coll.strip() for coll in str(row[get_column_header_name("Collection")]).split(';')] prop_val = [] for coll in collections: try: - query = f"FIND RECORD Collection WITH ID={int(coll)}" + query = f"FIND '{collection_rt.name}' WITH ID={int(coll)}" except ValueError: - query = f"FIND RECORD Collection WITH name='{coll}'" + query = f"FIND RECORD '{collection_rt.name}' WITH name='{coll}'" try: coll_rec = cached_get_entity_by(query=query) prop_val.append(coll_rec.id) except db.EmptyUniqueQueryError: - logger.warning(f"There is no collection with name or BIS ID {coll}. Skipping.") + logger.warning(f"There is no {collection_rt.name} with name or ID {coll}. Skipping.") continue if prop_val: - if child.get_property("Collection") is not None: - child.get_property("Collection").datatype = db.LIST("Collection") - child.get_property("Collection").value = prop_val + if child.get_property(collection_rt.name) is not None: + child.get_property(collection_rt.name).datatype = db.LIST(collection_rt.name) + child.get_property(collection_rt.name).value = prop_val else: child.add_property(id=collection_rt.id, name=collection_rt.name, datatype=db.LIST( - "Collection"), value=prop_val) + collection_rt.name), value=prop_val) # Treat Container Contents - if _value_in_row("Container Contents", row): - if not (_value_in_row("PI", row) and _value_in_row("Collection", row)): - logger.error( - f"Container Contents are given for container {child.id} but it " - "is missing PI and/or Collection info. No updates have been performed." - ) - return 1 - contents_prop = cached_get_entity_by(query="FIND PROPERTY 'Container Contents'") + if _value_in_row(get_column_header_name("Container Contents"), row): + contents_prop = cached_get_entity_by( + query=f"FIND PROPERTY '{get_entity_name('Container Contents')}'" + ) if child.get_property(contents_prop.name) is not None: - child.get_property(contents_prop.name).value = row["Container Contents"] + child.get_property(contents_prop.name).value = row[get_column_header_name("Container Contents")] else: child.add_property(id=contents_prop.id, name=contents_prop.name, - value=row["Container Contents"]) + value=row[get_column_header_name("Container Contents")]) # Treat PDF Report - if _value_in_row("PDFReport", row): - pdf_id = row["PDFReport"] + if _value_in_row(get_column_header_name("PDFReport"), row): + pdf_id = row[get_column_header_name("PDFReport")] try: pdf_id = int(pdf_id) - pdf_rec = cached_query(f"FIND FILE PDFReport WITH ID={pdf_id}") + pdf_rec = cached_query(f"FIND FILE {get_entity_name('PDFReport')} WITH ID={pdf_id}") if not pdf_rec: logger.warning( - f"There is no PDFReport with Bis ID {pdf_id}, so no PDF is attached to container {child.id}.") + f"There is no PDFReport with ID {pdf_id}, so no PDF is attached to container {child.id}.") else: - if child.get_property("PDFReport") is not None: - child.get_property("PDFReport").value = pdf_id + if child.get_property(get_entity_name("PDFReport")) is not None: + child.get_property(get_property("PDFReport")).value = pdf_id else: child.add_property(id=pdf_rt.id, name=pdf_rt.name, value=pdf_id) except ValueError: logger.warning( - f"There is no valid Bis ID provided for container {child.id}." + f"There is no valid ID provided for container {child.id}." f"Provided was {pdf_id}. Skipping") # This is a workaround for weird merging errors in the @@ -267,13 +273,13 @@ def main(): # following release. merged = [] for par in parent_containers: - if (data['BIS ID'] == par.id).any(): + if (data[id_column_name] == par.id).any(): # A container to be updated is used as another containers parent: child = child_containers.get_entity_by_id(par.id) # All parents have a child sample property with a value (which may # be empty). No child sample has this property, so the following is # okay without checks: - prop = par.get_property("Child container") + prop = par.get_property(child_container_prop.name) child.add_property(name=prop.name, id=prop.id, value=prop.value) merged.append(par) for par in merged: @@ -282,6 +288,8 @@ def main(): parent_containers.remove(par) # TODO Add notes as CommentAnnotation crawler = Crawler(securityMode=SecurityMode.UPDATE) + child_containers = post_process_containers_before_sync(child_containers) + parent_containers = post_process_parent_containers_before_sync(parent_containers) to_be_synchronized = child_containers + parent_containers inserts, updates = crawler.synchronize( @@ -291,10 +299,8 @@ def main(): if "SHARED_DIR" in os.environ: _notify_about_inserts_and_updates(len(inserts), len(updates), htmluserlog_public, crawler.run_id) - for ent in inserts + updates: - ent.retrieve_acl() - ent.grant(role='Stock Manager', priority=False, permission="EDIT:ACL") - ent.update_acl() + inserts, updates = post_process_inserts_and_updates(inserts, updates) + logger.info(f"Successfully processed {len(child_containers)} containers and " f"{len(parent_containers)} parent containers.") diff --git a/sample-management-custom/caosdb-server/scripting/bin/upload_sample_template.py b/sample-management-custom/caosdb-server/scripting/bin/upload_sample_template.py index f66744a9b118f90311718f802741f8e546e98e5c..716000720b5a8595dd0add4a31e870c9dac7b275 100755 --- a/sample-management-custom/caosdb-server/scripting/bin/upload_sample_template.py +++ b/sample-management-custom/caosdb-server/scripting/bin/upload_sample_template.py @@ -27,12 +27,9 @@ import json import logging import os import pandas as pd -import re import subprocess import sys -from datetime import date, datetime -from dateutil.relativedelta import relativedelta from pathlib import Path from tempfile import NamedTemporaryFile @@ -45,128 +42,21 @@ from caoscrawler.logging import configure_server_side_logging from bis_utils import (replace_entity_urls_by_ids, SPECIAL_TREATMENT_SAMPLE, whitespace_cleanup_in_df) +from sample_helpers.sample_upload_column_definitions import ( + COLUMN_CONVERTER, DATATYPE_DEFINITIONS, + OBLIGATORY_COLUMNS, OBLIGATORY_COLUMNS_CHILD, SPECIAL_TREATMENT_SAMPLE) +from sample_helpers.utils import CONSTANTS, get_column_header_name # suppress warning of diff function apilogger = logging.getLogger("linkahead.apiutils") apilogger.setLevel(logging.ERROR) -def semicolon_separated_list(text): - return [el.strip() for el in text.split(";") if el != ""] - - -def _embargo_converter(text: str): - - datepattern = r"^(?P<year>\d{4,4})-(?P<month>\d{2,2})-(?P<day>\d{2,2})" - matches = re.match(datepattern, str(text)) - if matches: - return date(int(matches.groupdict()["year"]), int(matches.groupdict()["month"]), int(matches.groupdict()["day"])) - if f"{text}".lower() in ["true", "yes"]: - # yes means embargo until today in one year - return date.today() + relativedelta(years=1) - if f"{text}".lower() in ["false", "no"]: - return "" - raise ValueError( - f"The embargo should be either a date in YYYY-MM-DD format, or 'true'/'yes' or 'false'/'no', but is {text}.") - - -ERROR_PREFIX = 'Something went wrong: ' -ERROR_SUFFIX = ' Please conatct <a href="mailto:biosamples@geomar.de">biosamples@geomar.de</a> if you encounter this issue.' +ERROR_PREFIX = CONSTANTS["error_prefix"] +ERROR_SUFFIX = CONSTANTS["error_suffix"] # Column datatypes -DATATYPE_DEFINITIONS = { - "AphiaID": int, - "BIS ID": str, - "Campaign": str, - "Date collected start": str, - "Date collected stop": str, - "Date sampled start": str, - "Date sampled stop": str, - "Fixation": str, - "Gear configuration": str, - "Gear": str, - "Hol": int, - "Latitude start": float, - "Latitude stop": float, - "Longitude start": float, - "Longitude stop": float, - "Main User": str, - "Nagoya case number": str, - "PI": str, - "Parent BIS ID": str, - "Platform": str, - "Sample Context": str, - "Sample container": str, - "SampleType": str, - "SampleTypeSpecific": str, - "Sampling Person": str, - "Sampling depth start": float, - "Sampling depth stop": float, - "Sampling method": str, - "Station ID": str, - "Station number": str, - "Storage Container Label": str, - "Storage ID": str, - "StorageTemperature": str, - "Subevent": str, - "Time collected start": str, - "Time collected stop": str, - "Time sampled start": str, - "Time sampled stop": str, - "Timezone": str, - "Water depth start": float, - "Water depth stop": float, -} - -# Obligatory columns: Must exist and must not be empty -# Must exist -OBLIGATORY_COLUMNS = [ - "BIS ID", - "Collection", - "Date collected start", - "Fixation", - "Gear", - "Latitude start", - "Longitude start", - "Main User", - "Nagoya case number", - "PI", - "Sample Context", - "Sample container", - "SampleType", - "SampleTypeSpecific", - "Sphere", - "Storage ID", - "StorageTemperature", -] - -OBLIGATORY_COLUMNS_CHILD = [ - "BIS ID", - "Date sampled start", - "Fixation", - "Main User", - "Parent BIS ID", - "Sample Context", - "Sample container", - "SampleType", - "SampleTypeSpecific", - "Sphere", - "Storage ID", - "StorageTemperature", -] - -COLUMN_CONVERTER = { - "Collection": semicolon_separated_list, - "Ecotaxa URL": semicolon_separated_list, - "NCBI Accession": semicolon_separated_list, - "NCBI BioProject": semicolon_separated_list, - "NCBI BioSample": semicolon_separated_list, - "OSIS URL": semicolon_separated_list, - "Embargo": _embargo_converter, - "Publications": semicolon_separated_list, - "Sphere": semicolon_separated_list, -} logger = logging.getLogger("caosadvancedtools") @@ -179,7 +69,7 @@ def get_parser(): def _is_child_sample_table(filename): tmp_data = pd.read_csv(filename, sep=',') if 'Parent BIS ID' in tmp_data.columns: - return not tmp_data["Parent BIS ID"].isnull().all() + return not tmp_data[get_column_header_name("Parent LinkAhead ID")].isnull().all() return False @@ -266,25 +156,27 @@ def main(): property_name = eprop.name if property_name in SPECIAL_TREATMENT_SAMPLE: continue - if db.apiutils.is_reference(eprop): - rt = db.get_entity_by_id(eprop.id) - if len(rt.properties) == 1: - converter = _get_converter_from_property_datatype(rt.properties[0].datatype) - elif len(rt.properties) == 1: - converter = str + if property_name not in DATATYPE_DEFINITIONS: + if db.apiutils.is_reference(eprop): + rt = db.get_entity_by_id(eprop.id) + if len(rt.properties) == 1: + converter = _get_converter_from_property_datatype(rt.properties[0].datatype) + elif len(rt.properties) < 1: + converter = str + else: + converter = None else: - converter = None - else: - converter = _get_converter_from_property_datatype(eprop.datatype) - if converter is None: - continue - DATATYPE_DEFINITIONS[property_name] = converter + converter = _get_converter_from_property_datatype(eprop.datatype) + if converter is None: + continue + DATATYPE_DEFINITIONS[property_name] = converter if sample.get_importance(property_name) == db.OBLIGATORY: # This is only needed if the sample is not a child sample OBLIGATORY_COLUMNS.append(property_name) try: data = read_data_from_file(path) - data = replace_entity_urls_by_ids(data, ["BIS ID", "Storage ID", "Parent BIS ID"]) + data = replace_entity_urls_by_ids(data, [get_column_header_name(name) for name in [ + "entity_id", "Storage ID", "Parent LinkAhead ID"]]) pickle_out = NamedTemporaryFile(delete=False, suffix=".pkl") data.to_pickle(pickle_out.name) except DataInconsistencyError as err: diff --git a/sample-management-custom/caosdb-server/scripting/home/identifiables.yml b/sample-management-custom/caosdb-server/scripting/home/identifiables.yml index f37c5c380ca38a583d74a8f09314f3850b810e47..c3df242f791a8224fadd31da7c0ffe75889ef775 100644 --- a/sample-management-custom/caosdb-server/scripting/home/identifiables.yml +++ b/sample-management-custom/caosdb-server/scripting/home/identifiables.yml @@ -1,40 +1,24 @@ -NagoyaCase: - - 'Nagoya case number' -Device: +Campaign: - name -SourceEvent: - - Gear - - Time start - - Position -Gear: - # TODO? - - name -Fixation: - # TODO? - - enumValue -Platform: - # TODO? - - name -FunctionalGroup: - # TODO? +ControlledRecordType: - name +Event: + - start_date + - Position SampleType: - # TODO? - - name -Collection: - name -SampleTypeSpecific: - # TODO? +SampleMethod: - name Sphere: - name StartPosition: - - Latitude - - Longitude - - Water depth - - Sampling depth + - latitude + - longitude + - elevation StopPosition: - - Latitude - - Longitude - - Water depth - - Sampling depth + - latitude + - longitude + - elevation +Person: + - first_name + - last_name diff --git a/test-profile/custom/other/restore/caosdb.sql b/test-profile/custom/other/restore/caosdb.2025-03-11T15:15:22.815622880+00:00.dump.sql similarity index 81% rename from test-profile/custom/other/restore/caosdb.sql rename to test-profile/custom/other/restore/caosdb.2025-03-11T15:15:22.815622880+00:00.dump.sql index 1182c12275a990e9755ec2c415a23eae0ca27a65..de206c453444440da48c8688c07420d83707ea0f 100644 --- a/test-profile/custom/other/restore/caosdb.sql +++ b/test-profile/custom/other/restore/caosdb.2025-03-11T15:15:22.815622880+00:00.dump.sql @@ -2,7 +2,7 @@ -- -- Host: sqldb Database: caosdb -- ------------------------------------------------------ --- Server version 10.11.9-MariaDB-ubu2204 +-- Server version 11.4.5-MariaDB-ubu2404 /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; @@ -36,7 +36,7 @@ CREATE TABLE `archive_collection_type` ( CONSTRAINT `archive_collection_type_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_collection_type_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_collection_type_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -45,6 +45,11 @@ CREATE TABLE `archive_collection_type` ( LOCK TABLES `archive_collection_type` WRITE; /*!40000 ALTER TABLE `archive_collection_type` DISABLE KEYS */; +INSERT INTO `archive_collection_type` VALUES +(0,0,113,'LIST',1), +(0,192,162,'LIST',1), +(0,194,162,'LIST',1), +(0,196,162,'LIST',1); /*!40000 ALTER TABLE `archive_collection_type` ENABLE KEYS */; UNLOCK TABLES; @@ -71,7 +76,7 @@ CREATE TABLE `archive_data_type` ( CONSTRAINT `archive_data_type_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_data_type_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_data_type_ibfk_4` FOREIGN KEY (`datatype`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -81,7 +86,14 @@ CREATE TABLE `archive_data_type` ( LOCK TABLES `archive_data_type` WRITE; /*!40000 ALTER TABLE `archive_data_type` DISABLE KEYS */; INSERT INTO `archive_data_type` VALUES -(0,188,177,105,1); +(0,0,113,111,1), +(0,0,120,118,1), +(0,192,163,124,1), +(0,194,163,124,1), +(0,196,163,124,1), +(0,192,162,139,1), +(0,194,162,139,1), +(0,196,162,139,1); /*!40000 ALTER TABLE `archive_data_type` ENABLE KEYS */; UNLOCK TABLES; @@ -107,7 +119,7 @@ CREATE TABLE `archive_date_data` ( CONSTRAINT `archive_date_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_date_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_date_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -116,6 +128,13 @@ CREATE TABLE `archive_date_data` ( LOCK TABLES `archive_date_data` WRITE; /*!40000 ALTER TABLE `archive_date_data` DISABLE KEYS */; +INSERT INTO `archive_date_data` VALUES +(0,196,147,20231220,'FIX',2,1), +(0,196,148,20231220,'FIX',3,1), +(0,194,147,20231220,'FIX',2,1), +(0,194,148,20231220,'FIX',3,1), +(0,192,147,20231220,'FIX',2,1), +(0,192,148,20231220,'FIX',3,1); /*!40000 ALTER TABLE `archive_date_data` ENABLE KEYS */; UNLOCK TABLES; @@ -142,7 +161,7 @@ CREATE TABLE `archive_datetime_data` ( CONSTRAINT `archive_datetime_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_datetime_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_datetime_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -165,7 +184,7 @@ CREATE TABLE `archive_desc_overrides` ( `domain_id` int(10) unsigned NOT NULL, `entity_id` int(10) unsigned NOT NULL, `property_id` int(10) unsigned NOT NULL, - `description` text NOT NULL, + `description` mediumtext NOT NULL, `_iversion` int(10) unsigned NOT NULL, UNIQUE KEY `archive_desc_overrides-d-e-p-v` (`domain_id`,`entity_id`,`property_id`,`_iversion`), KEY `domain_id` (`domain_id`,`entity_id`,`_iversion`), @@ -175,7 +194,7 @@ CREATE TABLE `archive_desc_overrides` ( CONSTRAINT `archive_desc_overrides_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_desc_overrides_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_desc_overrides_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -210,7 +229,7 @@ CREATE TABLE `archive_double_data` ( CONSTRAINT `archive_double_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_double_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_double_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -231,7 +250,7 @@ DROP TABLE IF EXISTS `archive_entities`; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `archive_entities` ( `id` int(10) unsigned NOT NULL, - `description` text DEFAULT NULL, + `description` mediumtext DEFAULT NULL, `role` enum('RECORDTYPE','RECORD','FILE','_REPLACEMENT','PROPERTY','DATATYPE','ROLE','QUERYTEMPLATE') NOT NULL, `acl` int(10) unsigned DEFAULT NULL, `_iversion` int(10) unsigned NOT NULL, @@ -239,7 +258,7 @@ CREATE TABLE `archive_entities` ( KEY `acl` (`acl`), CONSTRAINT `archive_entities_ibfk_1` FOREIGN KEY (`id`, `_iversion`) REFERENCES `entity_version` (`entity_id`, `_iversion`) ON DELETE CASCADE, CONSTRAINT `archive_entities_ibfk_2` FOREIGN KEY (`acl`) REFERENCES `entity_acl` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -249,16 +268,18 @@ CREATE TABLE `archive_entities` ( LOCK TABLES `archive_entities` WRITE; /*!40000 ALTER TABLE `archive_entities` DISABLE KEYS */; INSERT INTO `archive_entities` VALUES -(188,NULL,'RECORD',2,1), +(113,NULL,'PROPERTY',2,1), +(120,NULL,'PROPERTY',2,1), +(184,NULL,'RECORD',2,1), +(185,NULL,'RECORD',2,1), +(186,NULL,'RECORD',2,1), +(187,NULL,'RECORD',2,1), +(189,NULL,'RECORD',2,1), +(190,NULL,'RECORD',2,1), +(191,NULL,'RECORD',2,1), (192,NULL,'RECORD',2,1), -(193,NULL,'RECORD',2,1), -(193,NULL,'RECORD',3,2), (194,NULL,'RECORD',2,1), -(195,NULL,'RECORD',2,1), -(195,NULL,'RECORD',3,2), -(196,NULL,'RECORD',2,1), -(197,NULL,'RECORD',2,1), -(198,NULL,'RECORD',2,1); +(196,NULL,'RECORD',2,1); /*!40000 ALTER TABLE `archive_entities` ENABLE KEYS */; UNLOCK TABLES; @@ -284,7 +305,7 @@ CREATE TABLE `archive_enum_data` ( CONSTRAINT `archive_enum_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_enum_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_enum_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -305,13 +326,13 @@ DROP TABLE IF EXISTS `archive_files`; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `archive_files` ( `file_id` int(10) unsigned NOT NULL, - `path` text NOT NULL, + `path` mediumtext NOT NULL, `size` bigint(20) unsigned NOT NULL, `hash` binary(64) DEFAULT NULL, `_iversion` int(10) unsigned NOT NULL, PRIMARY KEY (`file_id`,`_iversion`), CONSTRAINT `archive_files_ibfk_1` FOREIGN KEY (`file_id`, `_iversion`) REFERENCES `entity_version` (`entity_id`, `_iversion`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -346,7 +367,7 @@ CREATE TABLE `archive_integer_data` ( CONSTRAINT `archive_integer_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_integer_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_integer_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -356,9 +377,7 @@ CREATE TABLE `archive_integer_data` ( LOCK TABLES `archive_integer_data` WRITE; /*!40000 ALTER TABLE `archive_integer_data` DISABLE KEYS */; INSERT INTO `archive_integer_data` VALUES -(0,192,137,1,'FIX',0,1,NULL), -(0,194,137,1,'FIX',0,1,NULL), -(0,196,137,1,'FIX',0,1,NULL); +(0,187,162,1,'FIX',0,1,NULL); /*!40000 ALTER TABLE `archive_integer_data` ENABLE KEYS */; UNLOCK TABLES; @@ -378,7 +397,7 @@ CREATE TABLE `archive_isa` ( KEY `child` (`child`,`child_iversion`), CONSTRAINT `archive_isa_ibfk_1` FOREIGN KEY (`parent`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_isa_ibfk_2` FOREIGN KEY (`child`, `child_iversion`) REFERENCES `entity_version` (`entity_id`, `_iversion`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -388,18 +407,16 @@ CREATE TABLE `archive_isa` ( LOCK TABLES `archive_isa` WRITE; /*!40000 ALTER TABLE `archive_isa` DISABLE KEYS */; INSERT INTO `archive_isa` VALUES -(188,1,100,0), -(188,1,101,0), -(188,1,106,1), -(193,1,111,1), -(192,1,136,1), -(195,1,111,1), -(193,2,111,1), -(194,1,136,1), -(197,1,111,1), -(195,2,111,1), -(196,1,136,1), -(198,1,118,1); +(187,1,131,1), +(191,1,111,1), +(190,1,111,1), +(189,1,111,1), +(186,1,118,1), +(185,1,118,1), +(184,1,118,1), +(196,1,126,1), +(194,1,126,1), +(192,1,126,1); /*!40000 ALTER TABLE `archive_isa` ENABLE KEYS */; UNLOCK TABLES; @@ -426,7 +443,7 @@ CREATE TABLE `archive_name_data` ( CONSTRAINT `archive_name_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_name_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_name_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -436,11 +453,11 @@ CREATE TABLE `archive_name_data` ( LOCK TABLES `archive_name_data` WRITE; /*!40000 ALTER TABLE `archive_name_data` DISABLE KEYS */; INSERT INTO `archive_name_data` VALUES -(0,193,20,'fad_mru_myn_2025_bld_1','FIX',0,1), -(0,195,20,'fad_mru_myn_2025_rom_1','FIX',0,1), -(0,193,20,'fad_mru_myn_2025_bld_1','FIX',0,2), -(0,197,20,'fad_mru_myn_2025_fri_1','FIX',0,1), -(0,195,20,'fad_mru_myn_2025_rom_1','FIX',0,2); +(0,113,20,'Child container','FIX',0,1), +(0,191,20,'MM_sb_3','FIX',0,1), +(0,190,20,'MM_sb_2','FIX',0,1), +(0,189,20,'MM_sb_1','FIX',0,1), +(0,120,20,'Parent sample','FIX',0,1); /*!40000 ALTER TABLE `archive_name_data` ENABLE KEYS */; UNLOCK TABLES; @@ -465,7 +482,7 @@ CREATE TABLE `archive_name_overrides` ( CONSTRAINT `archive_name_overrides_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_name_overrides_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_name_overrides_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -475,11 +492,7 @@ CREATE TABLE `archive_name_overrides` ( LOCK TABLES `archive_name_overrides` WRITE; /*!40000 ALTER TABLE `archive_name_overrides` DISABLE KEYS */; INSERT INTO `archive_name_overrides` VALUES -(0,193,177,'BIS label',1), -(0,193,177,'BIS label',2), -(0,195,177,'BIS label',1), -(0,195,177,'BIS label',2), -(0,197,177,'BIS label',1); +(0,187,162,'counter',1); /*!40000 ALTER TABLE `archive_name_overrides` ENABLE KEYS */; UNLOCK TABLES; @@ -504,7 +517,7 @@ CREATE TABLE `archive_null_data` ( CONSTRAINT `archive_null_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_null_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_null_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -514,13 +527,9 @@ CREATE TABLE `archive_null_data` ( LOCK TABLES `archive_null_data` WRITE; /*!40000 ALTER TABLE `archive_null_data` DISABLE KEYS */; INSERT INTO `archive_null_data` VALUES -(0,193,178,'FIX',0,1), -(0,195,178,'FIX',0,1), -(0,193,178,'FIX',0,2), -(0,197,178,'FIX',0,1), -(0,195,178,'FIX',0,2), -(0,198,111,'FIX',1,1), -(0,198,119,'FIX',2,1); +(0,191,162,'FIX',0,1), +(0,190,162,'FIX',0,1), +(0,189,162,'FIX',0,1); /*!40000 ALTER TABLE `archive_null_data` ENABLE KEYS */; UNLOCK TABLES; @@ -533,11 +542,11 @@ DROP TABLE IF EXISTS `archive_query_template_def`; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `archive_query_template_def` ( `id` int(10) unsigned NOT NULL, - `definition` mediumtext NOT NULL, + `definition` longtext NOT NULL, `_iversion` int(10) unsigned NOT NULL, PRIMARY KEY (`id`,`_iversion`), CONSTRAINT `archive_query_template_def_ibfk_1` FOREIGN KEY (`id`, `_iversion`) REFERENCES `entity_version` (`entity_id`, `_iversion`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -574,7 +583,7 @@ CREATE TABLE `archive_reference_data` ( CONSTRAINT `archive_reference_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_reference_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_reference_data_ibfk_4` FOREIGN KEY (`value`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -584,32 +593,32 @@ CREATE TABLE `archive_reference_data` ( LOCK TABLES `archive_reference_data` WRITE; /*!40000 ALTER TABLE `archive_reference_data` DISABLE KEYS */; INSERT INTO `archive_reference_data` VALUES -(0,188,177,187,NULL,'FIX',0,1), -(0,188,105,177,NULL,'REPLACEMENT',4,1), -(0,193,101,188,NULL,'FIX',0,1), -(0,193,110,189,NULL,'FIX',1,1), -(0,193,117,177,NULL,'REPLACEMENT',3,1), -(0,193,112,178,NULL,'REPLACEMENT',4,1), -(0,192,110,189,NULL,'FIX',1,1), -(0,195,101,188,NULL,'FIX',0,1), -(0,195,110,190,NULL,'FIX',1,1), -(0,195,117,177,NULL,'REPLACEMENT',3,1), -(0,195,112,178,NULL,'REPLACEMENT',4,1), -(0,193,101,188,NULL,'FIX',0,2), -(0,193,110,189,NULL,'FIX',1,2), -(0,193,117,177,NULL,'REPLACEMENT',3,2), -(0,193,112,178,NULL,'REPLACEMENT',4,2), -(0,194,110,190,NULL,'FIX',1,1), -(0,197,101,188,NULL,'FIX',0,1), -(0,197,110,191,NULL,'FIX',1,1), -(0,197,117,177,NULL,'REPLACEMENT',3,1), -(0,197,112,178,NULL,'REPLACEMENT',4,1), -(0,195,101,188,NULL,'FIX',0,2), -(0,195,110,190,NULL,'FIX',1,2), -(0,195,117,177,NULL,'REPLACEMENT',3,2), -(0,195,112,178,NULL,'REPLACEMENT',4,2), -(0,196,110,191,NULL,'FIX',1,1), -(0,198,124,188,NULL,'FIX',0,1); +(0,187,132,162,NULL,'REPLACEMENT',0,1), +(0,187,107,181,NULL,'FIX',1,1), +(0,191,101,182,NULL,'FIX',0,1), +(0,191,107,181,NULL,'FIX',1,1), +(0,191,113,162,NULL,'REPLACEMENT',3,1), +(0,190,101,182,NULL,'FIX',0,1), +(0,190,107,181,NULL,'FIX',1,1), +(0,190,113,162,NULL,'REPLACEMENT',3,1), +(0,189,101,182,NULL,'FIX',0,1), +(0,189,107,181,NULL,'FIX',1,1), +(0,189,113,162,NULL,'REPLACEMENT',3,1), +(0,186,121,182,NULL,'FIX',0,1), +(0,185,121,182,NULL,'FIX',0,1), +(0,184,121,182,NULL,'FIX',0,1), +(0,196,162,197,NULL,'FIX',0,1), +(0,196,139,162,NULL,'REPLACEMENT',0,1), +(0,196,163,177,NULL,'FIX',0,1), +(0,196,124,163,NULL,'REPLACEMENT',1,1), +(0,194,162,195,NULL,'FIX',0,1), +(0,194,139,162,NULL,'REPLACEMENT',0,1), +(0,194,163,177,NULL,'FIX',0,1), +(0,194,124,163,NULL,'REPLACEMENT',1,1), +(0,192,162,193,NULL,'FIX',0,1), +(0,192,139,162,NULL,'REPLACEMENT',0,1), +(0,192,163,177,NULL,'FIX',0,1), +(0,192,124,163,NULL,'REPLACEMENT',1,1); /*!40000 ALTER TABLE `archive_reference_data` ENABLE KEYS */; UNLOCK TABLES; @@ -624,7 +633,7 @@ CREATE TABLE `archive_text_data` ( `domain_id` int(10) unsigned NOT NULL, `entity_id` int(10) unsigned NOT NULL, `property_id` int(10) unsigned NOT NULL, - `value` text NOT NULL, + `value` mediumtext NOT NULL, `status` enum('OBLIGATORY','RECOMMENDED','SUGGESTED','FIX') NOT NULL, `pidx` int(10) unsigned NOT NULL, `_iversion` int(10) unsigned NOT NULL, @@ -635,7 +644,7 @@ CREATE TABLE `archive_text_data` ( CONSTRAINT `archive_text_data_ibfk_1` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_text_data_ibfk_2` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `archive_text_data_ibfk_3` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -645,20 +654,9 @@ CREATE TABLE `archive_text_data` ( LOCK TABLES `archive_text_data` WRITE; /*!40000 ALTER TABLE `archive_text_data` DISABLE KEYS */; INSERT INTO `archive_text_data` VALUES -(0,188,107,'My','FIX',0,1), -(0,188,102,'myn','FIX',1,1), -(0,188,108,'Name','FIX',2,1), -(0,188,109,'my.name@example.com','FIX',3,1), -(0,193,113,'20 rooms','FIX',2,1), -(0,193,177,'fad_mru_myn_2025_bld_1','FIX',0,1), -(0,195,113,'30 m²','FIX',2,1), -(0,195,177,'fad_mru_myn_2025_rom_1','FIX',0,1), -(0,193,113,'20 rooms','FIX',2,2), -(0,193,177,'fad_mru_myn_2025_bld_1','FIX',0,2), -(0,197,113,'300 l','FIX',2,1), -(0,197,177,'fad_mru_myn_2025_fri_1','FIX',0,1), -(0,195,113,'30 m²','FIX',2,2), -(0,195,177,'fad_mru_myn_2025_rom_1','FIX',0,2); +(0,191,114,'32','FIX',2,1), +(0,190,114,'32','FIX',2,1), +(0,189,114,'32','FIX',2,1); /*!40000 ALTER TABLE `archive_text_data` ENABLE KEYS */; UNLOCK TABLES; @@ -681,7 +679,7 @@ CREATE TABLE `collection_type` ( CONSTRAINT `collection_type_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `collection_type_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `collection_type_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -691,16 +689,19 @@ CREATE TABLE `collection_type` ( LOCK TABLES `collection_type` WRITE; /*!40000 ALTER TABLE `collection_type` DISABLE KEYS */; INSERT INTO `collection_type` VALUES -(0,0,112,'LIST'), -(0,0,125,'LIST'), -(0,0,126,'LIST'), -(0,0,127,'LIST'), -(0,0,128,'LIST'), -(0,0,129,'LIST'), -(0,0,141,'LIST'), -(0,0,164,'LIST'), -(0,153,177,'LIST'), -(0,210,178,'LIST'); +(0,0,113,'LIST'), +(0,0,122,'LIST'), +(0,0,123,'LIST'), +(0,0,137,'LIST'), +(0,0,154,'LIST'), +(0,0,160,'LIST'), +(0,126,165,'LIST'), +(0,189,163,'LIST'), +(0,190,163,'LIST'), +(0,191,163,'LIST'), +(0,192,162,'LIST'), +(0,194,162,'LIST'), +(0,196,162,'LIST'); /*!40000 ALTER TABLE `collection_type` ENABLE KEYS */; UNLOCK TABLES; @@ -725,7 +726,7 @@ CREATE TABLE `data_type` ( CONSTRAINT `datatype_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `datatype_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`), CONSTRAINT `datatype_forkey_type` FOREIGN KEY (`datatype`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -735,72 +736,77 @@ CREATE TABLE `data_type` ( LOCK TABLES `data_type` WRITE; /*!40000 ALTER TABLE `data_type` DISABLE KEYS */; INSERT INTO `data_type` VALUES -(0,0,134,12), -(0,0,137,12), -(0,0,144,13), -(0,0,145,13), -(0,0,146,13), -(0,0,147,13), +(0,0,132,12), +(0,0,140,13), +(0,0,141,13), +(0,0,142,13), (0,0,20,14), (0,0,21,14), (0,0,24,14), (0,0,102,14), (0,0,104,14), -(0,0,107,14), -(0,0,108,14), -(0,0,109,14), -(0,0,113,14), +(0,0,105,14), +(0,0,106,14), +(0,0,110,14), +(0,0,112,14), (0,0,114,14), (0,0,115,14), -(0,0,117,14), -(0,0,125,14), -(0,0,126,14), -(0,0,127,14), +(0,0,116,14), (0,0,128,14), -(0,0,129,14), -(0,0,130,14), -(0,0,131,14), -(0,0,139,14), -(0,0,140,14), -(0,0,141,14), -(0,0,142,14), +(0,0,134,14), +(0,0,136,14), +(0,0,137,14), +(0,0,138,14), (0,0,149,14), -(0,0,154,14), -(0,0,155,14), -(0,0,156,14), -(0,0,157,14), -(0,0,158,14), -(0,0,164,14), -(0,0,166,14), -(0,0,132,15), -(0,0,133,15), -(0,0,151,15), -(0,0,152,15), -(0,111,177,101), -(0,105,177,103), -(0,187,177,103), -(0,106,177,105), -(0,188,177,105), -(0,0,116,106), -(0,0,123,106), -(0,0,124,106), -(0,198,177,106), -(0,111,178,110), -(0,136,177,110), -(0,0,112,111), -(0,118,178,111), -(0,198,178,111), -(0,0,122,118), -(0,118,177,119), -(0,198,179,119), -(0,118,179,120), -(0,118,180,121), -(0,119,178,138), -(0,153,177,143), -(0,210,178,143), -(0,150,177,148), -(0,153,178,148), -(0,210,177,148); +(0,0,150,14), +(0,0,151,14), +(0,0,152,14), +(0,0,160,14), +(0,0,147,15), +(0,0,148,15), +(0,0,125,18), +(0,0,129,18), +(0,111,162,101), +(0,0,117,103), +(0,0,121,103), +(0,0,122,103), +(0,0,123,103), +(0,0,154,103), +(0,184,162,103), +(0,185,162,103), +(0,186,162,103), +(0,111,163,107), +(0,131,162,107), +(0,111,165,108), +(0,108,162,109), +(0,180,162,109), +(0,0,113,111), +(0,118,162,111), +(0,0,120,118), +(0,118,163,119), +(0,118,166,124), +(0,126,164,124), +(0,192,163,124), +(0,194,163,124), +(0,196,163,124), +(0,0,145,126), +(0,118,167,126), +(0,0,156,127), +(0,118,168,127), +(0,176,162,127), +(0,189,163,130), +(0,190,163,130), +(0,191,163,130), +(0,135,163,133), +(0,174,163,133), +(0,126,165,139), +(0,192,162,139), +(0,194,162,139), +(0,196,162,139), +(0,126,166,143), +(0,126,162,144), +(0,126,163,146), +(0,126,167,153); /*!40000 ALTER TABLE `data_type` ENABLE KEYS */; UNLOCK TABLES; @@ -824,7 +830,7 @@ CREATE TABLE `date_data` ( CONSTRAINT `date_ov_forkey_dom` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `date_ov_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `date_ov_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -834,7 +840,12 @@ CREATE TABLE `date_data` ( LOCK TABLES `date_data` WRITE; /*!40000 ALTER TABLE `date_data` DISABLE KEYS */; INSERT INTO `date_data` VALUES -(0,210,151,20250101,'FIX',0); +(0,196,147,20231220,'FIX',2), +(0,196,148,20231220,'FIX',3), +(0,194,147,20231220,'FIX',2), +(0,194,148,20231220,'FIX',3), +(0,192,147,20231220,'FIX',2), +(0,192,148,20231220,'FIX',3); /*!40000 ALTER TABLE `date_data` ENABLE KEYS */; UNLOCK TABLES; @@ -859,7 +870,7 @@ CREATE TABLE `datetime_data` ( CONSTRAINT `dat_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `dat_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `dat_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -890,7 +901,7 @@ CREATE TABLE `desc_overrides` ( CONSTRAINT `desc_ov_forkey_dom` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `desc_ov_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `desc_ov_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -923,7 +934,7 @@ CREATE TABLE `double_data` ( CONSTRAINT `dou_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `dou_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `dou_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -933,8 +944,13 @@ CREATE TABLE `double_data` ( LOCK TABLES `double_data` WRITE; /*!40000 ALTER TABLE `double_data` DISABLE KEYS */; INSERT INTO `double_data` VALUES -(0,211,146,1.1,'FIX',0,-1941227323), -(0,211,147,1.2,'FIX',1,-1941227323); +(0,193,140,13.2,'FIX',0,-1941227323), +(0,193,141,1,'FIX',1,-1941227323), +(0,195,140,14,'FIX',0,-1941227323), +(0,195,141,2,'FIX',1,-1941227323), +(0,195,142,23,'FIX',2,NULL), +(0,197,140,16.3,'FIX',0,-1941227323), +(0,197,141,3,'FIX',1,-1941227323); /*!40000 ALTER TABLE `double_data` ENABLE KEYS */; UNLOCK TABLES; @@ -953,7 +969,7 @@ CREATE TABLE `entities` ( PRIMARY KEY (`id`), KEY `entity_entity_acl` (`acl`), CONSTRAINT `entity_entity_acl` FOREIGN KEY (`acl`) REFERENCES `entity_acl` (`id`) -) ENGINE=InnoDB AUTO_INCREMENT=212 DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB AUTO_INCREMENT=198 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -983,116 +999,101 @@ INSERT INTO `entities` VALUES (24,'Description of an entity.','PROPERTY',0), (100,'Abstract type to denote that a RecordType is controlled, i.e.,\nthat no new Records of this type may be created during\nsample/container upload. Add this type as a parent to any\nRecordType you want to be controlled.\n','RECORDTYPE',2), (101,NULL,'RECORDTYPE',2), -(102,'An abbreviation for this entity that is used for creating BIS labels','PROPERTY',2), +(102,'An abbreviation for this entity that is used for identification purposes','PROPERTY',2), (103,NULL,'RECORDTYPE',2), -(104,'A german name for an entitiy. This can be used in queries since it is an alias.','PROPERTY',2), -(105,NULL,'RECORDTYPE',2), -(106,NULL,'RECORDTYPE',2), -(107,NULL,'PROPERTY',2), -(108,NULL,'PROPERTY',2), -(109,NULL,'PROPERTY',2), -(110,NULL,'RECORDTYPE',2), +(104,NULL,'PROPERTY',2), +(105,NULL,'PROPERTY',2), +(106,NULL,'PROPERTY',2), +(107,NULL,'RECORDTYPE',2), +(108,NULL,'RECORDTYPE',2), +(109,NULL,'RECORDTYPE',2), +(110,NULL,'PROPERTY',2), (111,NULL,'RECORDTYPE',2), -(112,NULL,'PROPERTY',2), -(113,'The size of the container, e.g. number of spaces or dimension','PROPERTY',2), -(114,'A custom label for a container that doesn\'t have to follow the BIS labeling guidelines.','PROPERTY',2), -(115,NULL,'PROPERTY',2), +(112,'A standardized label for a container','PROPERTY',2), +(113,NULL,'PROPERTY',2), +(114,'The size of the container, e.g. number of spaces or dimension','PROPERTY',2), +(115,'A custom label for a container that doesn\'t have to follow the automatic labeling guidelines.','PROPERTY',2), (116,NULL,'PROPERTY',2), -(117,'A standardized label for a container in BIS.','PROPERTY',2), +(117,NULL,'PROPERTY',2), (118,NULL,'RECORDTYPE',2), (119,NULL,'RECORDTYPE',2), -(120,NULL,'RECORDTYPE',2), -(121,NULL,'RECORDTYPE',2), +(120,NULL,'PROPERTY',2), +(121,NULL,'PROPERTY',2), (122,NULL,'PROPERTY',2), (123,NULL,'PROPERTY',2), -(124,NULL,'PROPERTY',2), +(124,NULL,'RECORDTYPE',2), (125,NULL,'PROPERTY',2), -(126,'DOIs of publications related to this sample.','PROPERTY',2), -(127,NULL,'PROPERTY',2), +(126,NULL,'RECORDTYPE',2), +(127,NULL,'RECORDTYPE',2), (128,NULL,'PROPERTY',2), (129,NULL,'PROPERTY',2), -(130,'A short description of the sampling methodology used for this sample.','PROPERTY',2), -(131,NULL,'PROPERTY',2), +(130,NULL,'RECORDTYPE',2), +(131,NULL,'RECORDTYPE',2), (132,NULL,'PROPERTY',2), -(133,NULL,'PROPERTY',2), -(134,'An ID for this taxon that allows to uniquely identify it on http://www.marinespecies.org.','PROPERTY',2), +(133,NULL,'RECORDTYPE',2), +(134,'The Marine Regions Geographic Identifier, i.e. a PID (Persistent Identifier) for a geographic region on https://www.marineregions.org','PROPERTY',2), (135,NULL,'RECORDTYPE',2), -(136,NULL,'RECORDTYPE',2), +(136,'A link to the Nagoya documents (or an indication where to find the documents).','PROPERTY',2), (137,NULL,'PROPERTY',2), -(138,NULL,'RECORDTYPE',2), -(139,'The Marine Regions Geographic Identifier, i.e. a PID (Persistent Identifier) for a geographic region on https://www.marineregions.org','PROPERTY',2), -(140,'A link to the Nagoya documents (or an indication where to find the documents).','PROPERTY',2), +(138,'An identifier for nagoya cases. This is managed by the Nagoya-Coordinator and should be taken care of before sampling.','PROPERTY',2), +(139,NULL,'RECORDTYPE',2), +(140,NULL,'PROPERTY',2), (141,NULL,'PROPERTY',2), -(142,'An identifier for nagoya cases. This is managed by the Nagoya-Coordinator and should be taken care of before sampling.','PROPERTY',2), +(142,NULL,'PROPERTY',2), (143,NULL,'RECORDTYPE',2), -(144,'The depth of the water in meters as a positive value','PROPERTY',2), -(145,'The depth in meters where the sampling took place as a positive value','PROPERTY',2), -(146,NULL,'PROPERTY',2), -(147,NULL,'PROPERTY',2), -(148,NULL,'RECORDTYPE',2), +(144,NULL,'RECORDTYPE',2), +(145,NULL,'PROPERTY',2), +(146,'Cruise name or leg description','RECORDTYPE',2), +(147,'The date and time the event started.','PROPERTY',2), +(148,'The date and time the event ended.','PROPERTY',2), (149,NULL,'PROPERTY',2), -(150,NULL,'RECORDTYPE',2), -(151,'The date and time the event started.','PROPERTY',2), -(152,'The date and time the event ended.','PROPERTY',2), +(150,NULL,'PROPERTY',2), +(151,NULL,'PROPERTY',2), +(152,NULL,'PROPERTY',2), (153,NULL,'RECORDTYPE',2), (154,NULL,'PROPERTY',2), -(155,NULL,'PROPERTY',2), +(155,NULL,'RECORDTYPE',2), (156,NULL,'PROPERTY',2), -(157,NULL,'PROPERTY',2), -(158,'The name/identifier of a station where regular sampling activity takes place.','PROPERTY',2), +(157,NULL,'RECORDTYPE',2), +(158,NULL,'RECORDTYPE',2), (159,NULL,'RECORDTYPE',2), -(160,NULL,'RECORDTYPE',2), -(161,NULL,'RECORDTYPE',2), -(162,NULL,'RECORDTYPE',2), -(163,NULL,'RECORDTYPE',2), -(164,NULL,'PROPERTY',2), -(165,NULL,'RECORDTYPE',2), -(166,NULL,'PROPERTY',2), -(167,NULL,'RECORDTYPE',2), -(168,NULL,'RECORDTYPE',2), -(169,'Cruise name or leg description','RECORDTYPE',2), -(170,NULL,'RECORDTYPE',2), -(171,NULL,'RECORDTYPE',2), -(172,'PDF files which contain reports and additional information about samples, collections, or containers','RECORDTYPE',2), -(173,NULL,'RECORDTYPE',2), -(174,NULL,'RECORDTYPE',2), -(175,NULL,'RECORDTYPE',2), -(176,NULL,'RECORDTYPE',2), -(177,NULL,'_REPLACEMENT',0), -(178,NULL,'_REPLACEMENT',0), -(179,NULL,'_REPLACEMENT',0), -(180,NULL,'_REPLACEMENT',0), -(181,NULL,'_REPLACEMENT',0), -(182,NULL,'_REPLACEMENT',0), -(183,NULL,'_REPLACEMENT',0), -(184,NULL,'_REPLACEMENT',0), -(185,NULL,'_REPLACEMENT',0), +(160,NULL,'PROPERTY',2), +(161,'PDF files which contain reports and additional information about samples, collections, or containers','RECORDTYPE',2), +(162,NULL,'_REPLACEMENT',0), +(163,NULL,'_REPLACEMENT',0), +(164,NULL,'_REPLACEMENT',0), +(165,NULL,'_REPLACEMENT',0), +(166,NULL,'_REPLACEMENT',0), +(167,NULL,'_REPLACEMENT',0), +(168,NULL,'_REPLACEMENT',0), +(169,NULL,'RECORD',2), +(170,NULL,'RECORD',2), +(171,NULL,'RECORD',2), +(172,NULL,'RECORD',2), +(173,NULL,'RECORD',2), +(174,NULL,'RECORD',2), +(175,NULL,'RECORD',2), +(176,NULL,'RECORD',2), +(177,NULL,'RECORD',2), +(178,NULL,'RECORD',2), +(179,NULL,'RECORD',2), +(180,NULL,'RECORD',2), +(181,NULL,'RECORD',2), +(182,NULL,'RECORD',2), +(183,NULL,'RECORD',2), +(184,NULL,'RECORD',2), +(185,NULL,'RECORD',2), (186,NULL,'RECORD',2), (187,NULL,'RECORD',2), -(188,NULL,'RECORD',2), (189,NULL,'RECORD',2), (190,NULL,'RECORD',2), (191,NULL,'RECORD',2), (192,NULL,'RECORD',2), -(193,NULL,'RECORD',3), +(193,NULL,'RECORD',2), (194,NULL,'RECORD',2), -(195,NULL,'RECORD',3), +(195,NULL,'RECORD',2), (196,NULL,'RECORD',2), -(197,NULL,'RECORD',3), -(198,NULL,'RECORD',2), -(199,NULL,'RECORDTYPE',2), -(200,NULL,'RECORD',2), -(201,NULL,'RECORD',2), -(202,NULL,'RECORD',2), -(203,NULL,'RECORD',2), -(204,NULL,'RECORD',2), -(205,NULL,'RECORD',2), -(206,NULL,'RECORD',2), -(207,NULL,'RECORD',2), -(208,NULL,'RECORD',2), -(209,NULL,'RECORD',2), -(210,NULL,'RECORD',2), -(211,NULL,'RECORD',2); +(197,NULL,'RECORD',2); /*!40000 ALTER TABLE `entities` ENABLE KEYS */; UNLOCK TABLES; @@ -1108,7 +1109,7 @@ CREATE TABLE `entity_acl` ( `acl` varbinary(65525) NOT NULL, PRIMARY KEY (`id`), KEY `entity_acl_acl` (`acl`(3072)) -) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1119,8 +1120,7 @@ LOCK TABLES `entity_acl` WRITE; /*!40000 ALTER TABLE `entity_acl` DISABLE KEYS */; INSERT INTO `entity_acl` VALUES (0,''), -(2,'[{\"realm\":\"PAM\",\"bitSet\":536608371,\"username\":\"admin\"}]'), -(3,'[{\"role\":\"Stock Manager\",\"bitSet\":1},{\"realm\":\"PAM\",\"bitSet\":536608371,\"username\":\"admin\"}]'); +(2,'[{\"realm\":\"PAM\",\"bitSet\":536608371,\"username\":\"admin\"}]'); /*!40000 ALTER TABLE `entity_acl` ENABLE KEYS */; UNLOCK TABLES; @@ -1226,47 +1226,34 @@ INSERT INTO `entity_ids` VALUES ('161',159), ('162',160), ('163',161), -('164',162), -('165',163), -('166',164), -('167',165), -('168',166), -('169',167), -('170',168), -('171',169), -('172',170), -('173',171), -('174',172), -('175',173), -('176',174), -('177',175), -('178',176), -('179',186), -('180',187), -('181',188), -('182',189), -('183',190), -('184',191), -('185',192), -('186',193), -('187',194), -('188',195), -('189',196), -('190',197), -('191',198), -('192',199), -('193',200), -('194',201), -('195',202), -('196',203), -('197',204), -('198',205), -('199',206), -('200',207), -('201',208), -('202',209), -('203',210), -('204',211); +('164',169), +('165',170), +('166',171), +('167',172), +('168',173), +('169',174), +('170',175), +('171',176), +('172',177), +('173',178), +('174',179), +('175',180), +('176',181), +('177',182), +('178',183), +('179',184), +('180',185), +('181',186), +('182',187), +('184',189), +('185',190), +('186',191), +('187',192), +('188',193), +('189',194), +('190',195), +('191',196), +('192',197); /*!40000 ALTER TABLE `entity_ids` ENABLE KEYS */; UNLOCK TABLES; @@ -1289,7 +1276,7 @@ CREATE TABLE `entity_version` ( KEY `srid` (`srid`), CONSTRAINT `entity_version_ibfk_1` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`) ON DELETE CASCADE, CONSTRAINT `entity_version_ibfk_2` FOREIGN KEY (`srid`) REFERENCES `transactions` (`srid`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1317,119 +1304,108 @@ INSERT INTO `entity_version` VALUES (20,NULL,'9030ad4b0f91c832ddb4a7c9d9b090c1f035b1d3',1,NULL,'cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e'), (21,NULL,'6a693e36a2afae1cbc571cce52c1676065c1ed24',1,NULL,'cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e'), (24,NULL,'d78e2b50954d74d6946d2c113cb7393b3b6146f3',1,NULL,'cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e'), -(100,NULL,'49a9b9af29eee9abeae35df062b9c427583e1f82',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(101,NULL,'d761933f0fb660cb3c456896d7a5cfb9da7b3ec6',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(102,NULL,'3a411f4a8633301cd3775e9eca91c456839ae455',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(103,NULL,'e9435f7be29bab1d255485c653b42323ab472061',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(104,NULL,'511fa3fc006bd78100827dee34a831f53df5bc10',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(105,NULL,'14ede2bd053b53293bf0a08f1f2f2b41d1d5e753',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(106,NULL,'48edf0033a82b59a298788a8fd1134aaf5f0cb69',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(107,NULL,'c07e48a5242cec75dd484535af46788b845fc0c0',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(108,NULL,'ee1eae1aa831261b54a3b302f2a8fe3832caa821',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(109,NULL,'8faa2c5a80f734b894d60dd7f41a69f9686b323f',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(110,NULL,'969e58211cc801b7f29f7d9cf2d386843af3418e',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(111,NULL,'36c265d80cb5abb79d5671a1b431dd62e971e9b5',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(112,NULL,'fb580fe4757eee3dba822814fa51c18a18b761ca',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(113,NULL,'271372b1dfdd98d82b290ad3f40335fbf51d23b1',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(114,NULL,'cf3ab7cfb60c2492e90ddf765784084060508a7d',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(115,NULL,'c3a15cdd5c2c50a8573ed8f58c70f54c437f6acf',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(116,NULL,'b8184c3ce8527029ffd55e5aa9919610fd2f5193',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(117,NULL,'8d70e8707a7954b7d83a25d615e9fbdd8f34519f',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(118,NULL,'68c8f4408696307d47f93015ecf286b079bd50ba',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(119,NULL,'9bfc33ae0c6be37cb027a62afd792d1c311d006a',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(120,NULL,'46a17aea4446446e72aeaffc3a01ee2f2d5d9957',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(121,NULL,'cecee2780f2756778d57180bb6579333c6743420',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(122,NULL,'1ae73a81970c7cffda61f4354d3747495252da95',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(123,NULL,'bba5ab43dbfba399bb2aa344196b390632749bb5',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(124,NULL,'4e1deb501787eab53c87268fb259cbccd869725f',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(125,NULL,'6c1cf7a25eb37e6090c6c96d8beadfa413427e5e',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(126,NULL,'c26bccdc84e3e3c56006a559943cb92ac797a2d8',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(127,NULL,'c554afcf416f8b6d986be567faf6687566eb8a01',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(128,NULL,'f256e3dccb79739fd27c29ee3ad3b971759c1c31',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(129,NULL,'942364fd6b4c82edc48b76fa1fa83084075c4f94',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(130,NULL,'c8d684f752f4bf3e3dd8a9b7bc73549954313ed1',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(131,NULL,'eb16006891bdaedbd9d0b55a3daf1f3b13481724',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(132,NULL,'a8aca96ae9d1d7596fd706b8b8cdc5a35da19f25',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(133,NULL,'610e6c185d51abb0369524c5896151bdff9f197d',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(134,NULL,'e9433547a60b486baaca979aec1608510a2f4541',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(135,NULL,'0f0c45a83c79dbdf7211691715154849d6f4b4f3',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(136,NULL,'6902c75485499ba6513b1faefb733322d9cb0239',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(137,NULL,'acd17699d752e90015eb8010e2f17ab5566a4241',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(138,NULL,'3c63009effd95704bc52400dc38ed7c8ccc63495',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(139,NULL,'54edec758a99a74f50577b74ba8a1ef187e08859',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(140,NULL,'b928d929461a11ab23a17e5031ff48bc00162a57',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(141,NULL,'56b4d616f183a665bd2b1e6de026cbe3a3555aff',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(142,NULL,'dc06b53bfef5dc59ccd33b646cb62cb66a37873e',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(143,NULL,'2a1014ad624089133ffbe85a7a39e82beeab2007',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(144,NULL,'a1d4da47f82593afbb269cbf6fc2311b059812d7',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(145,NULL,'535909011d32dc82dc3368b463c01f471c72b5f6',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(146,NULL,'8ab9de04122cf31364610d3372f0d922b4e110a5',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(147,NULL,'affd3537da482fb6655fc65cd223befedc845cb1',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(148,NULL,'3dc270e8ab4ffded2b7f3412e97469022a389a62',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(149,NULL,'949f9468c06c34881820eeeea6bb999e510c3fac',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(150,NULL,'5a0c0116c267a8165eec4bdb0f6bc9393af4ebac',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(151,NULL,'c5eb1e4ec70baa761c151d708676e4c436f28587',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(152,NULL,'8abed6579e778a2bf94e6e12b38791c7a7308b61',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(153,NULL,'a5e4f99ff9324eae679f20131b8b8da68846b1ad',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(154,NULL,'3f8eb5ecfd51c5a745a4862013547d518b7b2955',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(155,NULL,'afbd70a177d0724af03b005dfa54d7a67825082a',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(156,NULL,'34e04866309d1f335fc6767aa9993ef6ee88dbd3',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(157,NULL,'43e2960c8db90e1bf60369ac0d0eccb5321b26fe',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(158,NULL,'c1d5168ce86e67642bbee45bdc91e18972e62761',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(159,NULL,'728569f7e5de34afd82293e30bafcd11f106440e',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(160,NULL,'2c4ec676f3a5a7e6f7f322d047cef0ccd7e794f7',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(161,NULL,'2e5f4fab2cdd47e184fced9b5ea6dd54ba45cdb3',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(162,NULL,'49fd42751089f19e7d6bb5da57df79ba6cc546d2',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(163,NULL,'ac2b6777efb772874d95a88bc0062c520e4851c9',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(164,NULL,'767121db8cfc7592f4c50814393559b72614384d',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(165,NULL,'162195b8d54378680140bcc63a78109e83756525',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(166,NULL,'98c80742bcab5d3e1461a5be2daf816be849cf61',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(167,NULL,'14e18f38c1d81b861d158bd66d4187b141a94410',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(168,NULL,'2053db8b98bcb86eaaa6447b3e4f1f0f79e6c799',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(169,NULL,'acf024aaf0011c7cc2458fcff05098b34021aeaf',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(170,NULL,'82ecaea92b2db2591aa5259bd9a54462ba642e7c',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(171,NULL,'cb5c35f2ed5dc0df78b79566f0ad1bcdf9cc3443',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(172,NULL,'e797f308f5479d76d5255135b4e37bf0142977b4',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(173,NULL,'a036624431e6dca8807f38db5acba0fc62dc0bc8',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(174,NULL,'f5ab8459832337b7f5e21367039ccee6939334f0',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(175,NULL,'bf5412505c9414f6c376eb92fd327eb12c1af328',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(176,NULL,'f5ae5b2f08624ee694e4a9b05eb59cd0d17501c9',1,NULL,'27f2534b-49b9-4e44-9232-53eadf8ba072'), -(186,NULL,'4f341d574b7e8dede90e20f885bb16de0ef96be2',1,NULL,'ca9db47b-d2d5-4980-84da-7268869f9e3c'), -(187,NULL,'8a275960e8f28e8683d0fde450c1d6670b353199',1,NULL,'84bd4faf-510c-4950-b827-4bab81546887'), -(188,NULL,'d5051e0414a4d4ceddc945eb0fffea1d4ee58a1a',1,NULL,'c6549725-13bb-4076-9d49-219c6b8192f3'), -(188,NULL,'2358d20bb098701deefca886d99a9782ddc5918c',2,1,'55dcaf7b-7372-4778-995a-a6a8bde1b471'), -(189,NULL,'ed034b9364773357f149ad6027b35ba79488c8b6',1,NULL,'ce298544-326b-4b86-96fd-03de11548c9b'), -(190,NULL,'ac6b26b6839869e664429dca95f29f8fbd44d50d',1,NULL,'cad02c5b-f559-4400-9897-80e56c1035ba'), -(191,NULL,'ddb59c87045f1ef36a80194e9859b59a132d3f2c',1,NULL,'a41c5d15-6c9e-4e5c-a8fb-2fa7d38da140'), -(192,NULL,'7c54ab03516afa9119910fcd54105b835a2df455',1,NULL,'9d2c4381-148f-4559-9d30-95a328169bbf'), -(192,NULL,'a051a60190dc9250c80e96905f42e0a4da92323c',2,1,'34fade63-2811-4725-becf-0f2b1f84ad90'), -(193,NULL,'249bf63173256aa2a0199949a901047c2d080ab7',1,NULL,'daa492ef-1495-4123-9966-9951328211b3'), -(193,NULL,'0dc1ff0d0894d7033c1d97678b6a7785ed73d74b',2,1,'dbf1dd0f-405c-4952-b8b6-62e169c79397'), -(193,NULL,'67cfb95fa8340113f44f5fc9a359807e7049c0eb',3,2,'2df660f0-e3a6-447e-837f-4778e5323f5d'), -(194,NULL,'3a6433b58e041efb94320c84ecd5ddc741ee4975',1,NULL,'df3a0e97-6feb-42aa-8800-074ee5520428'), -(194,NULL,'c42e8133496df248dbd53b3935cfa04f60c747b2',2,1,'d0af8a57-7550-42b2-aba9-4b254897976b'), -(195,NULL,'d7bda4827e1f168d92a3bee5582f1aba5240931c',1,NULL,'8205d330-42ef-4519-9e5e-ade1e2465bed'), -(195,NULL,'6bd556548038c036e4ebc8e1832fa9fd522e0e68',2,1,'2d5a4ba7-517f-4b85-9727-b3d96988c485'), -(195,NULL,'54f7e3866e4bf3fcddbfe5ca5300e51b1f36f1b3',3,2,'b29fcae5-7964-4beb-b73a-2f7ce0a4809f'), -(196,NULL,'9db6c0625e7d6ad1c41e5909ad6e753c377697d3',1,NULL,'488967a3-a042-43d4-b74d-485f4bf622a2'), -(196,NULL,'bd9f34005699da67f4f7740642d92de5e18f1c05',2,1,'fe535f2d-6865-41b7-bab1-781f412454f6'), -(197,NULL,'fdc053362de6949898756c2b88da18d5bd86847b',1,NULL,'89a537e0-496a-4e50-b078-74f34cd21a07'), -(197,NULL,'604f5d26dedef5739253efa6722cf181c8405ff8',2,1,'c03fb8ed-a263-4997-a02b-37bf97af9563'), -(198,NULL,'6f97792294445568ab202c93d019065b2a0985ad',1,NULL,'3c931e13-7e9f-4835-99df-d325020b887d'), -(198,NULL,'5f1fe0019202163f814741a0117134c388188cfe',2,1,'032bc38a-28d3-4c59-9d31-1494b62a2ba4'), -(199,NULL,'a56ccd3a7b635129f62520a47996489c5caeca31',1,NULL,'22245d86-e095-4514-917a-7677616c544b'), -(200,NULL,'aff8c1f4b67fa6645d546beeec76715f4c4fae03',1,NULL,'aa019373-730c-4cec-8f83-3eb2f2ee9629'), -(201,NULL,'d94c32746d9f868be39694a4042927d7c38a7d87',1,NULL,'cbb111a6-b62c-4705-b1c4-36f055b3c815'), -(202,NULL,'efe1d8b7f283eb18115e284fbc787b56b3ddfbea',1,NULL,'ada94a1f-e942-46e2-a5de-626a18cd2af9'), -(203,NULL,'c4d16eafa86ba51b38d3001c1876f0776b13817e',1,NULL,'975ea8ec-ba79-452e-bc44-2d9432ce4993'), -(204,NULL,'d8fd7cc6dc9f02b45994c0871b08e1e01f1c36fd',1,NULL,'f284f0d8-aba2-44e9-bfc7-ae68fab970b9'), -(205,NULL,'431fbceb807c61bcb7fbccce56213ff84c27592f',1,NULL,'da56170f-ae54-4ecf-9aa4-342d7512384f'), -(206,NULL,'7296454c481fa252f610cb574e63859a36e1bcc1',1,NULL,'081a60fb-04b9-4d71-b025-d865b503fc7e'), -(207,NULL,'afa0a1402355e1066c5900083baba7db9f522988',1,NULL,'7663f26d-982f-4710-aeab-073a44290f3a'), -(208,NULL,'3198b07efca7137577e72e9b579fd522758cd883',1,NULL,'06147cec-8ac2-4c72-9744-cab44e960e09'), -(209,NULL,'425038e42b7607f1536b5bdda3e31c78486a9f69',1,NULL,'cd4b5be6-82c4-4444-bb3e-143ea3c1fcfe'), -(210,NULL,'9fb32d510fe24863b12988e6a32eb9d9a6322130',1,NULL,'b51ffa82-bcf0-42a1-922c-9f4a845e6f42'), -(211,NULL,'2b5e49feb87fc451f2d3441120adeae539c68f4e',1,NULL,'b51ffa82-bcf0-42a1-922c-9f4a845e6f42'); +(100,NULL,'834cf47992e768c4edd09f66d54ba1167d748bef',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(101,NULL,'865d0206534f6c7ba3cb8c677a66970157dda541',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(102,NULL,'aaf9752ed9d694895ba2adf4d6976333ed77645f',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(103,NULL,'f6cc4aebbea870d15e4877f1a821c3f3a6ad137e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(104,NULL,'3f6e2c74549a96161f8a0b78330736481b3e94aa',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(105,NULL,'d872daea6ca661fcfbf9cb744cd41c011057234f',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(106,NULL,'120b151f61b6bcf4f89d47684daed6cdb2751439',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(107,NULL,'33e571a8d25e04c78636a6c5c8eeb623f6bdfc5e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(108,NULL,'5e3f5708a524b575ef3d55a6df4c2f4a81aec3c1',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(109,NULL,'61ef8dc8b1cd6807836ad5526f57ae79d413c964',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(110,NULL,'6a76d96aaec6b1e08079fd83076aab46af4c423c',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(111,NULL,'a04b5ea25459c6975a29259d814abc917d1f1f9e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(112,NULL,'45fb60897565a442d8f0c96d66658d347e46db44',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(113,NULL,'0e71149f13d2da29b8029b777b8c6c6de05f4996',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(113,NULL,'7c39bd377923a83d468358b1fe8c24b573196543',2,1,'0c173ae6-d2cd-4b30-a759-2d91919b0a1e'), +(114,NULL,'b95a5c65518f8f655353f5d0abfd1459a6052aa3',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(115,NULL,'fe88e0751ddee0266ae82e8724b7a7e0a0997452',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(116,NULL,'d3616c2e6bbbebd25c248d8e6e9265107bc7673d',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(117,NULL,'686cb190bc76c1da153a2239d76748ab7142861b',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(118,NULL,'ae61dfaadb02184cac431d8aca91566b89c36a33',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(119,NULL,'97053962d7c33fef71ae5651d7a1b719952e4109',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(120,NULL,'38f0f1b1878ffab11e43bd58d06d50330d270706',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(120,NULL,'45cacbd7f51d912a16613135e52766d79576cf7b',2,1,'dcd7eccb-3e38-4c04-9a98-0159bda1e2c6'), +(121,NULL,'6762fa62a28b2e16bbd9f32213fb3fbd04cc05b5',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(122,NULL,'7e67f08277ba72ce3f97754c24a468246473e44e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(123,NULL,'70eba0f28eb99577757b6f2e20f02cf5c3071748',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(124,NULL,'1d6c69cda87d9fce09deab5f889dc6afc9485274',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(125,NULL,'6bd257ac19f7085bca102592d9403864e5387174',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(126,NULL,'59ee139d16da0dbba4e87a678c4c41d863ca149d',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(127,NULL,'1ab526a2a0e0a7a08ffb5c9b79eff1034650f01c',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(128,NULL,'096a319ad97f0f6f28fd85bd08ea3b3b350b0a33',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(129,NULL,'0ccab01ee883246b3706d8a9cc52147636197a3a',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(130,NULL,'cd7ca968bf170e18d82423714e43bb4990e52e55',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(131,NULL,'3ff41ea249c8ea081eccad9cfc6768d22fe11a04',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(132,NULL,'8fc92f01d943ddb0e04619a057187344e9b66c78',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(133,NULL,'94da39042b664d7c94d0f8c216987f65b7c770be',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(134,NULL,'6682ea6563c1a17d406cb697f15ae672a7726e7e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(135,NULL,'1d3612c0c8400f690f991316a0c51b997d927b68',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(136,NULL,'d1c1646b661e240fcc9e40cbdfc3e6869339236e',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(137,NULL,'9c2f95ef680c6fcd575c27465236699940e7e87d',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(138,NULL,'d95abf934c1366d9170061619bb1f1b420a581b9',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(139,NULL,'0fd2a8edd2203c63efcb78816b32efa829f53daa',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(140,NULL,'42ed31b3fced6ea2275e0518b65bd491d7acdaa6',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(141,NULL,'78392bdb8bfc52c048bd70938215dbbc9b8fe5b5',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(142,NULL,'50a74fd888008170fbf6adff4bedac323423be06',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(143,NULL,'01640dba73eb45b4408a18e4a33bf95395d7defd',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(144,NULL,'3475ec7d2d6c32feeaeb069e653b6d60c34000e1',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(145,NULL,'fd837ac370f67ebb522b570496b32c47085f8fdb',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(146,NULL,'19f98c27427720f6e5dabb86676c725c5aa99635',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(147,NULL,'ec1dac20148c004e1cdca88101fbc67e670d0912',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(148,NULL,'752e3b918e0039dae7371a7da62d0c1410d0a688',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(149,NULL,'a1d8c799de0358a134485826b61303b060473b57',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(150,NULL,'3f0be68e4fa329e42ac08cdd9713a50c39f57010',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(151,NULL,'5f9cbeb6898321be21243904ea7613e93c082296',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(152,NULL,'3b06518ce5cda240932be9c4e341e5ed6015fb1c',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(153,NULL,'e2aa9edf4b2741c3dc918393bbeecd5b206ef686',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(154,NULL,'307aa2f66bed1059b6f00813dd222f000d10c3bc',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(155,NULL,'149d80fa939bef673159467573a5df978a6b9add',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(156,NULL,'539576772757751a382937ba9a729cbf44190ca2',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(157,NULL,'39f12eac6b7b2e2f08257a1b853242ce2431ca65',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(158,NULL,'50801b9b449b7ed667cec70df2c1b112a2f6646c',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(159,NULL,'63c05518161c14977369af17f9bcb851f00dde52',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(160,NULL,'621f0c42ea5d672f72724b359f5d6542dc3fe550',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(161,NULL,'be0cf0988d8affa46341b5a4fd6986a40e820e94',1,NULL,'302ea332-e488-40d6-aa0b-9b78c7f9546f'), +(169,NULL,'7c66bbed301d2ba3d16ce48c2d0fd94db67141ba',1,NULL,'54969a5b-8ad4-4135-8117-b4800c29bd04'), +(170,NULL,'f5aee4ed9b9847f0fa8ea7bf12b677bbbb3c4176',1,NULL,'0c134ec8-b564-42c9-a02f-cfb234a0c45a'), +(171,NULL,'79246567e152d6dfeb57013213d9ec24e0f3c7d7',1,NULL,'f8cff7ae-db88-47cd-9f9e-2877478a8856'), +(172,NULL,'29d587f8f217fbba34c5166c8541dc0ba8d8714a',1,NULL,'359ed82b-8c2d-4a00-b00e-19d4e38ceb8d'), +(173,NULL,'43fcf4f4725579dae0874e0f476ec53b6b4d0c0c',1,NULL,'2e333e60-9884-46a4-8661-ae777bc81c6c'), +(174,NULL,'71d8bcc04472dd7ed4ff2d4fc250602da710e99d',1,NULL,'fa408261-eb8d-4da1-9ed6-026e344ec756'), +(175,NULL,'a9c2d795208d4f97d1ca4ffc0ad6bcc536eb080d',1,NULL,'586d8321-a31d-4251-af76-aacd4feaa35a'), +(176,NULL,'84d020df89d77a24242a9b88880816c73eac3b11',1,NULL,'a7994338-93c3-4cb7-9eaa-e0ea57de0069'), +(177,NULL,'7856155c34adcfce84b93ab34f4ebd8f83e29fef',1,NULL,'a71ff821-afb8-4f0d-9cee-d33a155ac395'), +(178,NULL,'c8747d1dd190e9a1d1c4a8608cd9722e43789d34',1,NULL,'a2fcd44f-17be-4181-b7f0-2d60cd894ccd'), +(179,NULL,'11e5db367245434f548e9f0e15da304819554ab0',1,NULL,'df090b99-7e9b-43ba-a461-367778e17068'), +(180,NULL,'36535bb3c35d37e8b0202ed3fed35e72712682df',1,NULL,'d66682c8-1138-4734-be3d-3e7f491899b2'), +(181,NULL,'fc07e627b38b8c2fd03105f92703c5b0fcb79c0d',1,NULL,'29851bcb-1bef-4829-9f65-4260a2a68b48'), +(182,NULL,'7666694e3e23930e540634f25dee7a8988d39ec4',1,NULL,'9091b57d-818e-4968-abee-c80780de2a4e'), +(183,NULL,'6e61714f4fa2c8088b76865524262fec9336b0e3',1,NULL,'bdf470b8-c562-4e4d-a2f2-1f2680822aed'), +(184,NULL,'ee99db165c01dc522e988f0c084e6089cb0d2abf',1,NULL,'fbab0a14-0ffb-4385-84c7-9d003b932757'), +(184,NULL,'f94be48600923430aa2408e0f4e341f4e2b38baa',2,1,'297074fa-3d95-4ee9-bf08-26ff31b8e1fc'), +(185,NULL,'56c72df0008be43bd789d8630c3d5f28074e9753',1,NULL,'fbab0a14-0ffb-4385-84c7-9d003b932757'), +(185,NULL,'95cef335f17eebc4c868f77b8ddf62e2579988d5',2,1,'297074fa-3d95-4ee9-bf08-26ff31b8e1fc'), +(186,NULL,'4dacb27a51cd5b913042400d86978dd5aa9a0d63',1,NULL,'fbab0a14-0ffb-4385-84c7-9d003b932757'), +(186,NULL,'52228d33eccdfdece8f6a29180b464717a4100f9',2,1,'297074fa-3d95-4ee9-bf08-26ff31b8e1fc'), +(187,NULL,'09aba54cfa01dab9850c84c2bdb4ad1ed7e6f764',1,NULL,'15c567d7-18dd-4484-a9d5-29246cdd33ae'), +(187,NULL,'b4bc47329bd9683407b4fe544709fa73c7998842',2,1,'72507861-7c79-4d8a-b121-db00b523a7b9'), +(189,NULL,'70d31c14477a778727a7a7a55d8440921d5938ad',1,NULL,'b9286cef-88c4-4e3d-90e1-ee09a76feae8'), +(189,NULL,'bc10f1957f3f017902e6a2899cf8b22bbe9eb1d8',2,1,'98cd3884-f6e2-40cd-81e8-ac8738bab3e9'), +(190,NULL,'f80f827ca4fab832c568aa7ee4e24cd824fa0574',1,NULL,'b9286cef-88c4-4e3d-90e1-ee09a76feae8'), +(190,NULL,'15e6a6e20241f59707526eacc9613b4405a1b46c',2,1,'98cd3884-f6e2-40cd-81e8-ac8738bab3e9'), +(191,NULL,'e77ed867a660cd46c942c86a89f9694ae67db1d2',1,NULL,'b9286cef-88c4-4e3d-90e1-ee09a76feae8'), +(191,NULL,'9cbac64a1b5169b9fe781e5984c8a8336ad15fb3',2,1,'98cd3884-f6e2-40cd-81e8-ac8738bab3e9'), +(192,NULL,'f688184e176ecb67fd010c328b4c0f13bd4ebb4f',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'), +(192,NULL,'0829ac0314f70f5f4ce5013b790f541b0b2162d6',2,1,'427f72b6-f28e-4b3d-8bb9-0c668f68d788'), +(193,NULL,'55e82c1d319424b57d606ab70c361ed7e1ebf11e',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'), +(194,NULL,'e0bfa06daed665705769511cb10e1b7389db1a8b',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'), +(194,NULL,'b0f30ee789256476386c39e4c823fca93d1434c8',2,1,'427f72b6-f28e-4b3d-8bb9-0c668f68d788'), +(195,NULL,'868b2a29e9a490b86135d25a85a4d5847992eb11',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'), +(196,NULL,'38231ed65e00db42f444b3becff3293d88b06fdf',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'), +(196,NULL,'e18e832bbc9a3bd6c04d69157f5a07ecc6a8754e',2,1,'427f72b6-f28e-4b3d-8bb9-0c668f68d788'), +(197,NULL,'318f5b84e55e44d47b90830d3ed7b22f3a038a4d',1,NULL,'bb848008-57b3-427d-876a-c5ff774d3d88'); /*!40000 ALTER TABLE `entity_version` ENABLE KEYS */; UNLOCK TABLES; @@ -1453,7 +1429,7 @@ CREATE TABLE `enum_data` ( CONSTRAINT `enum_ov_forkey_dom` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `enum_ov_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `enum_ov_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1476,7 +1452,7 @@ CREATE TABLE `feature_config` ( `_key` varchar(255) NOT NULL, `_value` varchar(255) DEFAULT NULL, PRIMARY KEY (`_key`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1505,7 +1481,7 @@ CREATE TABLE `files` ( `checked_timestamp` bigint(20) NOT NULL DEFAULT 0, PRIMARY KEY (`file_id`), CONSTRAINT `fil_file_id_entity` FOREIGN KEY (`file_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1538,7 +1514,7 @@ CREATE TABLE `integer_data` ( CONSTRAINT `int_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `int_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `int_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1548,9 +1524,7 @@ CREATE TABLE `integer_data` ( LOCK TABLES `integer_data` WRITE; /*!40000 ALTER TABLE `integer_data` DISABLE KEYS */; INSERT INTO `integer_data` VALUES -(0,192,137,2,'FIX',0,NULL), -(0,194,137,2,'FIX',0,NULL), -(0,196,137,2,'FIX',0,NULL); +(0,187,162,4,'FIX',0,NULL); /*!40000 ALTER TABLE `integer_data` ENABLE KEYS */; UNLOCK TABLES; @@ -1569,7 +1543,7 @@ CREATE TABLE `isa_cache` ( KEY `isa_cache_parent_entity` (`parent`), CONSTRAINT `isa_cache_child_entity` FOREIGN KEY (`child`) REFERENCES `entities` (`id`), CONSTRAINT `isa_cache_parent_entity` FOREIGN KEY (`parent`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1581,81 +1555,64 @@ LOCK TABLES `isa_cache` WRITE; INSERT INTO `isa_cache` VALUES (101,100,'101'), (103,100,'101'), +(107,100,'107'), +(109,100,'109'), +(119,100,'119'), +(124,100,'124'), +(130,100,'130'), +(133,100,'133'), +(143,100,'143'), +(144,100,'144'), +(146,100,'146'), +(153,100,'153'), +(155,100,'155'), +(161,100,'161'), +(169,100,'155'), +(170,100,'153'), +(171,100,'146'), +(172,100,'144'), +(173,100,'143'), +(175,100,'130'), +(177,100,'124'), +(178,100,'119'), +(179,100,'109'), +(181,100,'107'), +(182,100,'103>101'), (103,101,'103'), -(105,100,'101'), -(105,101,'105'), -(106,100,'101'), -(106,101,'106'), -(110,100,'110'), -(120,100,'120'), -(121,100,'121'), -(135,100,'135'), -(138,100,'138'), -(148,100,'148'), -(153,150,'153'), -(159,100,'159'), -(160,100,'160'), -(161,143,'161'), -(162,143,'162'), -(165,100,'165'), -(167,100,'167'), -(168,100,'168'), -(169,100,'169'), -(170,100,'170'), -(171,100,'171'), -(172,100,'172'), -(173,100,'173'), -(174,100,'174'), -(175,100,'175'), -(176,100,'176'), -(186,100,'103>101'), -(186,101,'103'), -(186,103,'186'), -(187,100,'105>101'), -(187,101,'105'), -(187,105,'187'), -(188,100,'106>101'), -(188,101,'106'), -(188,106,'188'), -(189,100,'110'), -(189,110,'189'), -(190,100,'110'), -(190,110,'190'), -(191,100,'110'), -(191,110,'191'), -(192,136,'192'), -(193,111,'193'), -(194,136,'194'), -(195,111,'195'), -(196,136,'196'), -(197,111,'197'), -(198,118,'198'), -(199,100,'148'), -(199,148,'199'), -(200,100,'199>148'), -(200,148,'199'), -(200,199,'200'), -(201,100,'135'), -(201,135,'201'), -(202,119,'202'), -(203,100,'176'), -(203,176,'203'), -(204,100,'167'), -(204,167,'204'), -(205,100,'168'), -(205,168,'205'), -(206,100,'121'), -(206,121,'206'), -(207,100,'160'), -(207,160,'207'), -(208,100,'170'), -(208,170,'208'), -(209,100,'171'), -(209,171,'209'), -(210,150,'153'), -(210,153,'210'), -(211,143,'161'), -(211,161,'211'); +(182,101,'103'), +(182,103,'182'), +(181,107,'181'), +(180,108,'180'), +(179,109,'179'), +(189,111,'189'), +(190,111,'190'), +(191,111,'191'), +(184,118,'184'), +(185,118,'185'), +(186,118,'186'), +(178,119,'178'), +(177,124,'177'), +(192,126,'192'), +(194,126,'194'), +(196,126,'196'), +(176,127,'176'), +(175,130,'175'), +(187,131,'187'), +(174,135,'174'), +(157,139,'157'), +(158,139,'158'), +(193,139,'157'), +(195,139,'157'), +(197,139,'157'), +(173,143,'173'), +(172,144,'172'), +(171,146,'171'), +(170,153,'170'), +(169,155,'169'), +(193,157,'193'), +(195,157,'195'), +(197,157,'197'), +(183,159,'183'); /*!40000 ALTER TABLE `isa_cache` ENABLE KEYS */; UNLOCK TABLES; @@ -1681,7 +1638,7 @@ CREATE TABLE `name_data` ( CONSTRAINT `name_data_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `name_data_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `name_data_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1712,97 +1669,80 @@ INSERT INTO `name_data` VALUES (0,100,20,'ControlledRecordType','FIX',0), (0,101,20,'Responsible','FIX',0), (0,102,20,'Abbreviation','FIX',0), -(0,103,20,'ResearchDivision','FIX',0), -(0,104,20,'German name','FIX',0), -(0,105,20,'ResearchUnit','FIX',0), -(0,106,20,'Person','FIX',0), -(0,107,20,'First name','FIX',0), -(0,108,20,'Last name','FIX',0), -(0,109,20,'Email','FIX',0), -(0,110,20,'ContainerType','FIX',0), +(0,103,20,'Person','FIX',0), +(0,104,20,'Email','FIX',0), +(0,105,20,'first_name','FIX',0), +(0,106,20,'last_name','FIX',0), +(0,107,20,'ContainerType','FIX',0), +(0,108,20,'StorageCondition','FIX',0), +(0,109,20,'ConditionType','FIX',0), +(0,110,20,'condition_value','FIX',0), (0,111,20,'Container','FIX',0), -(0,112,20,'Child container','FIX',0), -(0,113,20,'Container size','FIX',0), -(0,114,20,'Custom label','FIX',0), -(0,115,20,'Container Contents','FIX',0), -(0,116,20,'PI','FIX',0), -(0,117,20,'BIS Label','FIX',0), +(0,112,20,'container_label','FIX',0), +(0,113,20,'Child_Container','FIX',0), +(0,114,20,'Container size','FIX',0), +(0,115,20,'custom_label','FIX',0), +(0,116,20,'Container Contents','FIX',0), +(0,117,20,'PI','FIX',0), (0,118,20,'Sample','FIX',0), -(0,119,20,'NagoyaCase','FIX',0), -(0,120,20,'FunctionalGroup','FIX',0), -(0,121,20,'SampleType','FIX',0), -(0,122,20,'Parent sample','FIX',0), -(0,123,20,'Sampling Person','FIX',0), -(0,124,20,'Main User','FIX',0), -(0,125,20,'Custom ID','FIX',0), -(0,126,20,'Publications','FIX',0), -(0,127,20,'NCBI BioProject','FIX',0), -(0,128,20,'NCBI BioSample','FIX',0), -(0,129,20,'NCBI Accession','FIX',0), -(0,130,20,'Sampling method','FIX',0), -(0,131,20,'Notes','FIX',0), -(0,132,20,'Time sampled start','FIX',0), -(0,133,20,'Time sampled stop','FIX',0), -(0,134,20,'AphiaID','FIX',0), -(0,135,20,'Collection','FIX',0), -(0,136,20,'LabelCounter','FIX',0), -(0,137,20,'Counter','FIX',0), -(0,138,20,'EEZ','FIX',0), -(0,139,20,'MRGID','FIX',0), -(0,140,20,'Nagoya documents','FIX',0), -(0,141,20,'Old case number','FIX',0), -(0,142,20,'Nagoya Case Number','FIX',0), -(0,143,20,'Position','FIX',0), -(0,144,20,'Water depth','FIX',0), -(0,145,20,'Sampling depth','FIX',0), -(0,146,20,'Latitude','FIX',0), -(0,147,20,'Longitude','FIX',0), -(0,148,20,'Gear','FIX',0), -(0,149,20,'Configuration','FIX',0), -(0,150,20,'Event','FIX',0), -(0,151,20,'Time start','FIX',0), -(0,152,20,'Time stop','FIX',0), -(0,153,20,'SourceEvent','FIX',0), -(0,154,20,'Hol','FIX',0), -(0,155,20,'Station Number','FIX',0), -(0,156,20,'IGSN','FIX',0), -(0,157,20,'DOI','FIX',0), -(0,158,20,'Station ID','FIX',0), -(0,159,20,'Platform','FIX',0), -(0,160,20,'SampleTypeSpecific','FIX',0), -(0,161,20,'StartPosition','FIX',0), -(0,162,20,'StopPosition','FIX',0), -(0,163,20,'RegisterSampleTemplate','FIX',0), -(0,164,20,'column_names','FIX',0), -(0,165,20,'MeshPoreSize','FIX',0), -(0,166,20,'enumValue','FIX',0), -(0,167,20,'Fixation','FIX',0), -(0,168,20,'StorageTemperature','FIX',0), -(0,169,20,'Campaign','FIX',0), -(0,170,20,'Sphere','FIX',0), -(0,171,20,'Sample context','FIX',0), -(0,172,20,'PDFReport','FIX',0), -(0,173,20,'commonGermanName','FIX',0), -(0,174,20,'commonEnglishName','FIX',0), -(0,175,20,'Taxonomic group/Species','FIX',0), -(0,176,20,'Sample container','FIX',0), -(0,186,20,'Fancy Division','FIX',0), -(0,187,20,'MyResearchUnit','FIX',0), -(0,188,20,'My Name','FIX',0), -(0,189,20,'Building','FIX',0), -(0,190,20,'Room','FIX',0), -(0,191,20,'Fridge','FIX',0), -(0,193,20,'fad_mru_myn_2025_bld_1','FIX',0), -(0,195,20,'fad_mru_myn_2025_rom_1','FIX',0), -(0,197,20,'fad_mru_myn_2025_fri_1','FIX',0), -(0,199,20,'Fishing rod','FIX',0), -(0,201,20,'Test Collection','FIX',0), -(0,202,20,'ABCD-1234','FIX',0), -(0,203,20,'Paper bag','FIX',0), -(0,206,20,'Fish','FIX',0), -(0,207,20,'Filet','FIX',0), -(0,208,20,'Water','FIX',0), -(0,209,20,'Field sampling','FIX',0); +(0,119,20,'SampleType','FIX',0), +(0,120,20,'Parent_sample','FIX',0), +(0,121,20,'Main User','FIX',0), +(0,122,20,'Collector','FIX',0), +(0,123,20,'Curator','FIX',0), +(0,124,20,'Device','FIX',0), +(0,125,20,'material_left','FIX',0), +(0,126,20,'Event','FIX',0), +(0,127,20,'SampleMethod','FIX',0), +(0,128,20,'nagoya_link','FIX',0), +(0,129,20,'nagoya_relevance','FIX',0), +(0,130,20,'Collection','FIX',0), +(0,131,20,'LabelCounter','FIX',0), +(0,132,20,'Counter','FIX',0), +(0,133,20,'EEZ','FIX',0), +(0,134,20,'MRGID','FIX',0), +(0,135,20,'NagoyaCase','FIX',0), +(0,136,20,'Nagoya documents','FIX',0), +(0,137,20,'Old case number','FIX',0), +(0,138,20,'Nagoya Case Number','FIX',0), +(0,139,20,'Position','FIX',0), +(0,140,20,'latitude','FIX',0), +(0,141,20,'longitude','FIX',0), +(0,142,20,'elevation','FIX',0), +(0,143,20,'EventType','FIX',0), +(0,144,20,'Biome','FIX',0), +(0,145,20,'Parent_Event','FIX',0), +(0,146,20,'Campaign','FIX',0), +(0,147,20,'start_date','FIX',0), +(0,148,20,'end_date','FIX',0), +(0,149,20,'igsn_doi','FIX',0), +(0,150,20,'level','FIX',0), +(0,151,20,'locality_description','FIX',0), +(0,152,20,'locality_name','FIX',0), +(0,153,20,'Sphere','FIX',0), +(0,154,20,'Responsible_Person','FIX',0), +(0,155,20,'Platform','FIX',0), +(0,156,20,'Parent_SampleMethod','FIX',0), +(0,157,20,'StartPosition','FIX',0), +(0,158,20,'StopPosition','FIX',0), +(0,159,20,'RegisterSampleTemplate','FIX',0), +(0,160,20,'column_names','FIX',0), +(0,161,20,'PDFReport','FIX',0), +(0,169,20,'Meteor','FIX',0), +(0,170,20,'Water','FIX',0), +(0,171,20,'Expo','FIX',0), +(0,172,20,'Ocean','FIX',0), +(0,173,20,'Fisching Trip','FIX',0), +(0,175,20,'wild life','FIX',0), +(0,176,20,'fisching','FIX',0), +(0,177,20,'net','FIX',0), +(0,178,20,'fish','FIX',0), +(0,179,20,'frozen','FIX',0), +(0,181,20,'Steelbox','FIX',0), +(0,183,20,'test-template','FIX',0), +(0,189,20,'MM_sb_1','FIX',0), +(0,190,20,'MM_sb_2','FIX',0), +(0,191,20,'MM_sb_3','FIX',0); /*!40000 ALTER TABLE `name_data` ENABLE KEYS */; UNLOCK TABLES; @@ -1825,7 +1765,7 @@ CREATE TABLE `name_overrides` ( CONSTRAINT `name_ov_forkey_dom` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `name_ov_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `name_ov_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1835,10 +1775,7 @@ CREATE TABLE `name_overrides` ( LOCK TABLES `name_overrides` WRITE; /*!40000 ALTER TABLE `name_overrides` DISABLE KEYS */; INSERT INTO `name_overrides` VALUES -(0,193,177,'BIS label'), -(0,197,177,'BIS label'), -(0,195,177,'BIS label'), -(0,198,180,'Sample Context'); +(0,187,162,'counter'); /*!40000 ALTER TABLE `name_overrides` ENABLE KEYS */; UNLOCK TABLES; @@ -1861,7 +1798,7 @@ CREATE TABLE `null_data` ( CONSTRAINT `null_forkey_dom` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `null_forkey_ent` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `null_forkey_pro` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1871,79 +1808,74 @@ CREATE TABLE `null_data` ( LOCK TABLES `null_data` WRITE; /*!40000 ALTER TABLE `null_data` DISABLE KEYS */; INSERT INTO `null_data` VALUES -(0,101,102,'OBLIGATORY',0), -(0,103,104,'OBLIGATORY',0), -(0,103,102,'OBLIGATORY',0), -(0,105,104,'OBLIGATORY',0), -(0,105,177,'OBLIGATORY',0), -(0,105,102,'OBLIGATORY',0), -(0,106,107,'OBLIGATORY',0), -(0,106,108,'OBLIGATORY',1), -(0,106,109,'OBLIGATORY',2), -(0,106,177,'OBLIGATORY',0), -(0,106,102,'OBLIGATORY',0), -(0,110,102,'OBLIGATORY',0), -(0,111,177,'OBLIGATORY',0), -(0,111,178,'OBLIGATORY',0), -(0,111,117,'OBLIGATORY',2), -(0,111,179,'SUGGESTED',0), -(0,111,113,'SUGGESTED',4), -(0,111,114,'SUGGESTED',5), -(0,111,115,'SUGGESTED',6), -(0,111,116,'SUGGESTED',7), -(0,118,177,'RECOMMENDED',0), -(0,118,178,'RECOMMENDED',0), -(0,118,179,'RECOMMENDED',0), -(0,118,180,'RECOMMENDED',0), -(0,118,122,'RECOMMENDED',4), -(0,118,123,'RECOMMENDED',5), -(0,118,124,'RECOMMENDED',6), -(0,118,181,'SUGGESTED',0), -(0,118,182,'SUGGESTED',0), -(0,118,183,'SUGGESTED',0), -(0,118,184,'SUGGESTED',0), -(0,118,185,'SUGGESTED',0), -(0,118,130,'SUGGESTED',12), -(0,118,131,'SUGGESTED',13), -(0,118,132,'SUGGESTED',14), -(0,118,133,'SUGGESTED',15), -(0,118,134,'SUGGESTED',16), -(0,119,142,'OBLIGATORY',0), -(0,119,140,'RECOMMENDED',1), -(0,119,177,'SUGGESTED',0), -(0,119,178,'SUGGESTED',0), -(0,136,137,'OBLIGATORY',0), -(0,136,177,'OBLIGATORY',0), -(0,138,139,'OBLIGATORY',0), -(0,143,146,'OBLIGATORY',0), -(0,143,147,'OBLIGATORY',1), -(0,143,144,'RECOMMENDED',2), -(0,143,145,'RECOMMENDED',3), -(0,148,149,'RECOMMENDED',0), -(0,150,177,'OBLIGATORY',0), -(0,150,151,'RECOMMENDED',1), -(0,150,152,'SUGGESTED',2), -(0,153,177,'OBLIGATORY',0), -(0,153,178,'OBLIGATORY',0), -(0,153,154,'RECOMMENDED',2), -(0,153,155,'RECOMMENDED',3), -(0,153,156,'RECOMMENDED',4), -(0,153,157,'RECOMMENDED',5), -(0,153,158,'SUGGESTED',6), -(0,153,151,'RECOMMENDED',0), -(0,153,152,'SUGGESTED',0), -(0,161,146,'OBLIGATORY',0), -(0,161,147,'OBLIGATORY',0), -(0,162,146,'OBLIGATORY',0), -(0,162,147,'OBLIGATORY',0), -(0,163,177,'RECOMMENDED',0), -(0,165,166,'RECOMMENDED',0), -(0,167,166,'RECOMMENDED',0), -(0,168,166,'RECOMMENDED',0), -(0,197,178,'FIX',0), -(0,199,149,'RECOMMENDED',0), -(0,211,145,'FIX',2), -(0,211,144,'FIX',3); +(0,101,102,'RECOMMENDED',0), +(0,103,105,'OBLIGATORY',0), +(0,103,106,'OBLIGATORY',1), +(0,103,104,'RECOMMENDED',2), +(0,103,102,'RECOMMENDED',0), +(0,107,102,'OBLIGATORY',0), +(0,108,162,'RECOMMENDED',0), +(0,108,110,'RECOMMENDED',1), +(0,111,162,'OBLIGATORY',0), +(0,111,163,'OBLIGATORY',0), +(0,111,112,'SUGGESTED',2), +(0,111,164,'SUGGESTED',0), +(0,111,114,'SUGGESTED',4), +(0,111,115,'SUGGESTED',5), +(0,111,116,'SUGGESTED',6), +(0,111,117,'SUGGESTED',7), +(0,111,165,'SUGGESTED',0), +(0,118,162,'RECOMMENDED',0), +(0,118,163,'RECOMMENDED',0), +(0,118,120,'RECOMMENDED',2), +(0,118,121,'RECOMMENDED',3), +(0,118,164,'RECOMMENDED',0), +(0,118,165,'RECOMMENDED',0), +(0,118,166,'RECOMMENDED',0), +(0,118,125,'RECOMMENDED',7), +(0,118,167,'RECOMMENDED',0), +(0,118,168,'RECOMMENDED',0), +(0,118,128,'RECOMMENDED',10), +(0,118,129,'RECOMMENDED',11), +(0,126,145,'RECOMMENDED',0), +(0,126,162,'RECOMMENDED',0), +(0,126,163,'RECOMMENDED',0), +(0,126,164,'RECOMMENDED',0), +(0,126,165,'RECOMMENDED',0), +(0,126,147,'RECOMMENDED',5), +(0,126,148,'RECOMMENDED',6), +(0,126,166,'RECOMMENDED',0), +(0,126,149,'RECOMMENDED',8), +(0,126,150,'RECOMMENDED',9), +(0,126,151,'RECOMMENDED',10), +(0,126,152,'RECOMMENDED',11), +(0,126,167,'RECOMMENDED',0), +(0,126,168,'RECOMMENDED',0), +(0,127,156,'RECOMMENDED',0), +(0,131,132,'OBLIGATORY',0), +(0,131,162,'OBLIGATORY',0), +(0,133,134,'OBLIGATORY',0), +(0,135,138,'OBLIGATORY',0), +(0,135,136,'RECOMMENDED',1), +(0,135,162,'SUGGESTED',0), +(0,135,163,'SUGGESTED',0), +(0,139,140,'OBLIGATORY',0), +(0,139,141,'OBLIGATORY',1), +(0,139,142,'OBLIGATORY',2), +(0,157,140,'OBLIGATORY',0), +(0,157,141,'OBLIGATORY',0), +(0,157,142,'OBLIGATORY',0), +(0,158,140,'OBLIGATORY',0), +(0,158,141,'OBLIGATORY',0), +(0,158,142,'OBLIGATORY',0), +(0,159,162,'RECOMMENDED',0), +(0,174,162,'FIX',0), +(0,174,163,'FIX',0), +(0,176,162,'FIX',0), +(0,191,162,'FIX',0), +(0,190,162,'FIX',0), +(0,193,142,'FIX',2), +(0,197,142,'FIX',2); /*!40000 ALTER TABLE `null_data` ENABLE KEYS */; UNLOCK TABLES; @@ -1961,7 +1893,7 @@ CREATE TABLE `passwd` ( `it` int(10) unsigned DEFAULT 5000, `salt` varbinary(255) NOT NULL, PRIMARY KEY (`principal`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -1985,7 +1917,7 @@ CREATE TABLE `permissions` ( `permissions` mediumtext NOT NULL, PRIMARY KEY (`role`), CONSTRAINT `perm_name_roles` FOREIGN KEY (`role`) REFERENCES `roles` (`name`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2011,7 +1943,7 @@ CREATE TABLE `query_template_def` ( `definition` mediumtext NOT NULL, PRIMARY KEY (`id`), CONSTRAINT `query_template_def_ibfk_1` FOREIGN KEY (`id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2048,7 +1980,7 @@ CREATE TABLE `reference_data` ( CONSTRAINT `ref_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`), CONSTRAINT `ref_value_entity` FOREIGN KEY (`value`) REFERENCES `entities` (`id`), CONSTRAINT `reference_data_ibfk_1` FOREIGN KEY (`value`, `value_iversion`) REFERENCES `entity_version` (`entity_id`, `_iversion`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2058,67 +1990,85 @@ CREATE TABLE `reference_data` ( LOCK TABLES `reference_data` WRITE; /*!40000 ALTER TABLE `reference_data` DISABLE KEYS */; INSERT INTO `reference_data` VALUES -(0,105,103,177,'REPLACEMENT',1,NULL), -(0,106,105,177,'REPLACEMENT',3,NULL), -(0,111,101,177,'REPLACEMENT',0,NULL), -(0,111,110,178,'REPLACEMENT',1,NULL), -(0,111,112,179,'REPLACEMENT',3,NULL), -(0,118,119,177,'REPLACEMENT',0,NULL), -(0,118,111,178,'REPLACEMENT',1,NULL), -(0,118,120,179,'REPLACEMENT',2,NULL), -(0,118,121,180,'REPLACEMENT',3,NULL), -(0,118,125,181,'REPLACEMENT',7,NULL), -(0,118,126,182,'REPLACEMENT',8,NULL), -(0,118,127,183,'REPLACEMENT',9,NULL), -(0,118,128,184,'REPLACEMENT',10,NULL), -(0,118,129,185,'REPLACEMENT',11,NULL), -(0,119,141,177,'REPLACEMENT',2,NULL), -(0,119,138,178,'REPLACEMENT',3,NULL), -(0,136,110,177,'REPLACEMENT',1,NULL), -(0,150,148,177,'REPLACEMENT',0,NULL), -(0,153,143,177,'REPLACEMENT',0,NULL), -(0,153,148,178,'REPLACEMENT',1,NULL), -(0,163,164,177,'REPLACEMENT',0,NULL), -(0,187,177,186,'FIX',0,NULL), -(0,187,103,177,'REPLACEMENT',1,NULL), -(0,188,177,187,'FIX',0,NULL), -(0,188,105,177,'REPLACEMENT',4,NULL), -(0,192,110,189,'FIX',1,NULL), -(0,193,101,188,'FIX',0,NULL), -(0,193,110,189,'FIX',1,NULL), -(0,193,117,177,'REPLACEMENT',3,NULL), -(0,193,178,195,'FIX',0,NULL), -(0,193,112,178,'REPLACEMENT',4,NULL), -(0,194,110,190,'FIX',1,NULL), -(0,197,101,188,'FIX',0,NULL), -(0,197,110,191,'FIX',1,NULL), -(0,197,117,177,'REPLACEMENT',3,NULL), -(0,197,112,178,'REPLACEMENT',4,NULL), -(0,195,101,188,'FIX',0,NULL), -(0,195,110,190,'FIX',1,NULL), -(0,195,117,177,'REPLACEMENT',3,NULL), -(0,195,178,197,'FIX',0,NULL), -(0,195,112,178,'REPLACEMENT',4,NULL), -(0,196,110,191,'FIX',1,NULL), -(0,210,177,200,'FIX',0,NULL), -(0,210,148,177,'REPLACEMENT',1,NULL), -(0,210,178,211,'FIX',0,NULL), -(0,210,143,178,'REPLACEMENT',2,NULL), -(0,198,177,188,'FIX',0,NULL), -(0,198,124,177,'REPLACEMENT',0,NULL), -(0,198,178,197,'FIX',0,NULL), -(0,198,111,178,'REPLACEMENT',1,NULL), -(0,198,179,202,'FIX',0,NULL), -(0,198,119,179,'REPLACEMENT',2,NULL), -(0,198,116,188,'FIX',3,NULL), -(0,198,153,210,'FIX',4,NULL), -(0,198,176,203,'FIX',5,NULL), -(0,198,167,204,'FIX',6,NULL), -(0,198,168,205,'FIX',7,NULL), -(0,198,121,206,'FIX',8,NULL), -(0,198,160,207,'FIX',9,NULL), -(0,198,180,209,'FIX',0,NULL), -(0,198,171,180,'REPLACEMENT',10,NULL); +(0,108,109,162,'REPLACEMENT',0,NULL), +(0,111,101,162,'REPLACEMENT',0,NULL), +(0,111,107,163,'REPLACEMENT',1,NULL), +(0,111,113,164,'REPLACEMENT',3,NULL), +(0,111,108,165,'REPLACEMENT',8,NULL), +(0,118,111,162,'REPLACEMENT',0,NULL), +(0,118,119,163,'REPLACEMENT',1,NULL), +(0,118,122,164,'REPLACEMENT',4,NULL), +(0,118,123,165,'REPLACEMENT',5,NULL), +(0,118,124,166,'REPLACEMENT',6,NULL), +(0,118,126,167,'REPLACEMENT',8,NULL), +(0,118,127,168,'REPLACEMENT',9,NULL), +(0,126,144,162,'REPLACEMENT',1,NULL), +(0,126,146,163,'REPLACEMENT',2,NULL), +(0,126,124,164,'REPLACEMENT',3,NULL), +(0,126,139,165,'REPLACEMENT',4,NULL), +(0,126,143,166,'REPLACEMENT',7,NULL), +(0,126,153,167,'REPLACEMENT',12,NULL), +(0,126,154,168,'REPLACEMENT',13,NULL), +(0,131,107,162,'REPLACEMENT',1,NULL), +(0,135,137,162,'REPLACEMENT',2,NULL), +(0,135,133,163,'REPLACEMENT',3,NULL), +(0,159,160,162,'REPLACEMENT',0,NULL), +(0,174,137,162,'REPLACEMENT',2,NULL), +(0,174,133,163,'REPLACEMENT',3,NULL), +(0,176,156,162,'REPLACEMENT',0,NULL), +(0,180,162,179,'FIX',0,NULL), +(0,180,109,162,'REPLACEMENT',0,NULL), +(0,183,160,162,'REPLACEMENT',0,NULL), +(0,183,121,182,'FIX',1,NULL), +(0,187,132,162,'REPLACEMENT',0,NULL), +(0,187,107,181,'FIX',1,NULL), +(0,191,101,182,'FIX',0,NULL), +(0,191,107,181,'FIX',1,NULL), +(0,191,113,162,'REPLACEMENT',3,NULL), +(0,191,117,182,'FIX',4,NULL), +(0,191,163,175,'FIX',0,NULL), +(0,191,130,163,'REPLACEMENT',5,NULL), +(0,190,101,182,'FIX',0,NULL), +(0,190,107,181,'FIX',1,NULL), +(0,190,113,162,'REPLACEMENT',3,NULL), +(0,190,117,182,'FIX',4,NULL), +(0,190,163,175,'FIX',0,NULL), +(0,190,130,163,'REPLACEMENT',5,NULL), +(0,189,101,182,'FIX',0,NULL), +(0,189,107,181,'FIX',1,NULL), +(0,189,162,191,'FIX',1,NULL), +(0,189,162,190,'FIX',0,NULL), +(0,189,113,162,'REPLACEMENT',3,NULL), +(0,189,117,182,'FIX',4,NULL), +(0,189,163,175,'FIX',0,NULL), +(0,189,130,163,'REPLACEMENT',5,NULL), +(0,186,162,182,'FIX',0,NULL), +(0,186,121,162,'REPLACEMENT',0,NULL), +(0,186,111,190,'FIX',1,NULL), +(0,186,124,177,'FIX',2,NULL), +(0,186,126,196,'FIX',3,NULL), +(0,185,162,182,'FIX',0,NULL), +(0,185,121,162,'REPLACEMENT',0,NULL), +(0,185,111,190,'FIX',1,NULL), +(0,185,124,177,'FIX',2,NULL), +(0,185,126,194,'FIX',3,NULL), +(0,184,162,182,'FIX',0,NULL), +(0,184,121,162,'REPLACEMENT',0,NULL), +(0,184,111,189,'FIX',1,NULL), +(0,184,124,177,'FIX',2,NULL), +(0,184,126,192,'FIX',3,NULL), +(0,196,162,197,'FIX',0,NULL), +(0,196,139,162,'REPLACEMENT',0,NULL), +(0,196,163,177,'FIX',0,NULL), +(0,196,124,163,'REPLACEMENT',1,NULL), +(0,194,162,195,'FIX',0,NULL), +(0,194,139,162,'REPLACEMENT',0,NULL), +(0,194,163,177,'FIX',0,NULL), +(0,194,124,163,'REPLACEMENT',1,NULL), +(0,192,162,193,'FIX',0,NULL), +(0,192,139,162,'REPLACEMENT',0,NULL), +(0,192,163,177,'FIX',0,NULL), +(0,192,124,163,'REPLACEMENT',1,NULL); /*!40000 ALTER TABLE `reference_data` ENABLE KEYS */; UNLOCK TABLES; @@ -2133,7 +2083,7 @@ CREATE TABLE `roles` ( `name` varbinary(255) NOT NULL, `description` mediumtext DEFAULT NULL, PRIMARY KEY (`name`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2159,7 +2109,7 @@ CREATE TABLE `stats` ( `name` varchar(255) NOT NULL, `value` blob DEFAULT NULL, PRIMARY KEY (`name`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2169,7 +2119,7 @@ CREATE TABLE `stats` ( LOCK TABLES `stats` WRITE; /*!40000 ALTER TABLE `stats` DISABLE KEYS */; INSERT INTO `stats` VALUES -('RootBenchmark','¬í\0sr\0-org.caosdb.server.database.misc.RootBenchmarkþ¼ƒQk]\0\0xr\04org.caosdb.server.database.misc.TransactionBenchmarkþ¼ƒQk]\0J\0sinceL\0measurementst\0Ljava/util/Map;[\0stackTraceElementst\0[Ljava/lang/StackTraceElement;L\0\rsubBenchmarksq\0~\0xp\0\0”Av§«sr\0java.util.HashMapÚÁÃ`Ñ\0F\0\nloadFactorI\0 thresholdxp?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xur\0[Ljava.lang.StackTraceElement;F*<<ý\"9\0\0xp\0\0\0sr\0java.lang.StackTraceElementa Åš&6Ý…\0B\0formatI\0\nlineNumberL\0classLoaderNamet\0Ljava/lang/String;L\0declaringClassq\0~\0\nL\0fileNameq\0~\0\nL\0\nmethodNameq\0~\0\nL\0\nmoduleNameq\0~\0\nL\0\rmoduleVersionq\0~\0\nxp\0\0Spt\0java.lang.Threadt\0Thread.javat\0\rgetStackTracet\0 java.baset\017.0.13sq\0~\0 \0\0!t\0appt\04org.caosdb.server.database.misc.TransactionBenchmarkt\0TransactionBenchmark.javat\0<init>ppsq\0~\0 \0\0\0ƒq\0~\0t\0-org.caosdb.server.database.misc.RootBenchmarkq\0~\0q\0~\0ppsq\0~\0 \0\0q\0~\0q\0~\0q\0~\0t\0<clinit>ppsq\0~\0 \0\0fq\0~\0t\0org.caosdb.server.CaosDBServert\0CaosDBServer.javat\0initBackendppsq\0~\0 \0\0\0 q\0~\0q\0~\0q\0~\0t\0mainppsq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0Infosr\0,org.caosdb.server.database.misc.SubBenchmarkþ¼ƒQk]\0L\0nameq\0~\0\nxq\0~\0\0\0”Av§®sq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0\nsq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0t\0,org.caosdb.server.database.misc.SubBenchmarkq\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0t\0getBenchmarkppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0\0$q\0~\0t\02org.caosdb.server.transaction.TransactionInterfacet\0TransactionInterface.javat\0getTransactionBenchmarkppsq\0~\0 \0\0\00q\0~\0q\0~\0.q\0~\0/t\0executeppsq\0~\0 \0\0\0õq\0~\0t\0org.caosdb.server.utils.Infot\0 Info.javat\0syncDatabaseppsq\0~\0 \0\0\0–q\0~\0t\0/org.caosdb.server.database.misc.RootBenchmark$1q\0~\0t\0runppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0 SyncStatssq\0~\0\"\0\0”Av§®sq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0 sq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0q\0~\0)q\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0\00q\0~\0q\0~\0.q\0~\0/q\0~\02ppsq\0~\0 \0\0\0õq\0~\0q\0~\04q\0~\05q\0~\06ppsq\0~\0 \0\0\0–q\0~\0q\0~\08q\0~\0q\0~\09ppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0MySQLSyncStatssq\0~\0\"\0\0”Av§¯sq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0sq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0q\0~\0)q\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0q\0~\0t\0-org.caosdb.server.database.BackendTransactiont\0BackendTransaction.javat\0getImplementationppsq\0~\0 \0\0\0+q\0~\0t\08org.caosdb.server.database.backend.transaction.SyncStatst\0SyncStats.javaq\0~\02ppsq\0~\0 \0\0\0žq\0~\0q\0~\0Tq\0~\0Ut\0executeTransactionppsq\0~\0 \0\0\01q\0~\0q\0~\0.q\0~\0/q\0~\02ppsq\0~\0 \0\0\0õq\0~\0q\0~\04q\0~\05q\0~\06ppsq\0~\0 \0\0\0–q\0~\0q\0~\08q\0~\0q\0~\09ppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xq\0~\0Jxq\0~\0<xq\0~\0!x'), +('RootBenchmark','¬í\0sr\0-org.caosdb.server.database.misc.RootBenchmarkþ¼ƒQk]\0\0xr\04org.caosdb.server.database.misc.TransactionBenchmarkþ¼ƒQk]\0J\0sinceL\0measurementst\0Ljava/util/Map;[\0stackTraceElementst\0[Ljava/lang/StackTraceElement;L\0\rsubBenchmarksq\0~\0xp\0\0•…‚\0ásr\0java.util.HashMapÚÁÃ`Ñ\0F\0\nloadFactorI\0 thresholdxp?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xur\0[Ljava.lang.StackTraceElement;F*<<ý\"9\0\0xp\0\0\0sr\0java.lang.StackTraceElementa Åš&6Ý…\0B\0formatI\0\nlineNumberL\0classLoaderNamet\0Ljava/lang/String;L\0declaringClassq\0~\0\nL\0fileNameq\0~\0\nL\0\nmethodNameq\0~\0\nL\0\nmoduleNameq\0~\0\nL\0\rmoduleVersionq\0~\0\nxp\0\0Spt\0java.lang.Threadt\0Thread.javat\0\rgetStackTracet\0 java.baset\017.0.14sq\0~\0 \0\0!t\0appt\04org.caosdb.server.database.misc.TransactionBenchmarkt\0TransactionBenchmark.javat\0<init>ppsq\0~\0 \0\0\0ƒq\0~\0t\0-org.caosdb.server.database.misc.RootBenchmarkq\0~\0q\0~\0ppsq\0~\0 \0\0q\0~\0q\0~\0q\0~\0t\0<clinit>ppsq\0~\0 \0\0fq\0~\0t\0org.caosdb.server.CaosDBServert\0CaosDBServer.javat\0initBackendppsq\0~\0 \0\0\0 q\0~\0q\0~\0q\0~\0t\0mainppsq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0Infosr\0,org.caosdb.server.database.misc.SubBenchmarkþ¼ƒQk]\0L\0nameq\0~\0\nxq\0~\0\0\0•…‚\0ãsq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0\nsq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0t\0,org.caosdb.server.database.misc.SubBenchmarkq\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0t\0getBenchmarkppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0\0$q\0~\0t\02org.caosdb.server.transaction.TransactionInterfacet\0TransactionInterface.javat\0getTransactionBenchmarkppsq\0~\0 \0\0\00q\0~\0q\0~\0.q\0~\0/t\0executeppsq\0~\0 \0\0\0õq\0~\0t\0org.caosdb.server.utils.Infot\0 Info.javat\0syncDatabaseppsq\0~\0 \0\0\0–q\0~\0t\0/org.caosdb.server.database.misc.RootBenchmark$1q\0~\0t\0runppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0 SyncStatssq\0~\0\"\0\0•…‚\0ãsq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0 sq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0q\0~\0)q\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0\00q\0~\0q\0~\0.q\0~\0/q\0~\02ppsq\0~\0 \0\0\0õq\0~\0q\0~\04q\0~\05q\0~\06ppsq\0~\0 \0\0\0–q\0~\0q\0~\08q\0~\0q\0~\09ppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0t\0MySQLSyncStatssq\0~\0\"\0\0•…‚\0ãsq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xuq\0~\0\0\0\0sq\0~\0 \0\0Spq\0~\0q\0~\0\rq\0~\0q\0~\0q\0~\0sq\0~\0 \0\0!q\0~\0q\0~\0q\0~\0q\0~\0ppsq\0~\0 \0\0\0×q\0~\0q\0~\0)q\0~\0q\0~\0ppsq\0~\0 \0\0šq\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0£q\0~\0q\0~\0q\0~\0q\0~\0+ppsq\0~\0 \0\0q\0~\0t\0-org.caosdb.server.database.BackendTransactiont\0BackendTransaction.javat\0getImplementationppsq\0~\0 \0\0\0+q\0~\0t\08org.caosdb.server.database.backend.transaction.SyncStatst\0SyncStats.javaq\0~\02ppsq\0~\0 \0\0\0žq\0~\0q\0~\0Tq\0~\0Ut\0executeTransactionppsq\0~\0 \0\0\01q\0~\0q\0~\0.q\0~\0/q\0~\02ppsq\0~\0 \0\0\0õq\0~\0q\0~\04q\0~\05q\0~\06ppsq\0~\0 \0\0\0–q\0~\0q\0~\08q\0~\0q\0~\09ppsq\0~\0 \0\0Hpq\0~\0q\0~\0\rq\0~\09q\0~\0q\0~\0sq\0~\0?@\0\0\0\0\0\0w\0\0\0\0\0\0\0xq\0~\0Jxq\0~\0<xq\0~\0!x'), ('TransactionBenchmark','¬í\0sr\00caosdb.server.database.misc.TransactionBenchmark„Cl=˜ÕãE\0J\0sinceL\0acct\0Ljava/util/HashMap;L\0countsq\0~\0xp\0\0lžèòWsr\0java.util.HashMapÚÁÃ`Ñ\0F\0\nloadFactorI\0 thresholdxp?@\0\0\0\0\0w\0\0\0\0\0\0t\0 SyncStatssr\0java.lang.Long;‹äÌ#ß\0J\0valuexr\0java.lang.Number†¬•”à‹\0\0xp\0\0\0\0\0\0\0t\0GetInfosq\0~\0\0\0\0\0\0\0 xsq\0~\0?@\0\0\0\0\0w\0\0\0\0\0\0q\0~\0sr\0java.lang.Integerâ ¤÷‡8\0I\0valuexq\0~\0\0\0\0q\0~\0 sq\0~\0\0\0\0x'); /*!40000 ALTER TABLE `stats` ENABLE KEYS */; UNLOCK TABLES; @@ -2194,7 +2144,7 @@ CREATE TABLE `text_data` ( CONSTRAINT `str_domain_id_entity` FOREIGN KEY (`domain_id`) REFERENCES `entities` (`id`), CONSTRAINT `str_entity_id_entity` FOREIGN KEY (`entity_id`) REFERENCES `entities` (`id`), CONSTRAINT `str_property_id_entity` FOREIGN KEY (`property_id`) REFERENCES `entities` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2204,43 +2154,35 @@ CREATE TABLE `text_data` ( LOCK TABLES `text_data` WRITE; /*!40000 ALTER TABLE `text_data` DISABLE KEYS */; INSERT INTO `text_data` VALUES -(143,146,21,'°','FIX',0), -(143,147,21,'°','FIX',0), -(143,144,21,'m','FIX',0), -(143,145,21,'m','FIX',0), -(0,144,21,'m','FIX',0), -(0,145,21,'m','FIX',0), -(0,146,21,'°','FIX',0), -(0,147,21,'°','FIX',0), -(161,146,21,'°','FIX',0), -(161,147,21,'°','FIX',0), -(162,146,21,'°','FIX',0), -(162,147,21,'°','FIX',0), -(0,186,104,'Spannende Abteilung','FIX',0), -(0,186,102,'fad','FIX',1), -(0,187,102,'mru','FIX',0), -(0,187,104,'Meine Forschungsgruppe','FIX',2), -(0,189,102,'bld','FIX',0), -(0,190,102,'rom','FIX',0), -(0,191,102,'fri','FIX',0), -(0,188,107,'My','FIX',0), -(0,188,102,'myn','FIX',1), -(0,188,108,'Name','FIX',2), -(0,188,109,'my.name@example.com','FIX',3), -(0,193,113,'20 rooms','FIX',2), -(0,193,177,'fad_mru_myn_2025_bld_1','FIX',0), -(0,197,113,'300 l','FIX',2), -(0,197,177,'fad_mru_myn_2025_fri_1','FIX',0), -(0,195,113,'30 m²','FIX',2), -(0,195,177,'fad_mru_myn_2025_rom_1','FIX',0), -(0,200,149,'2m','FIX',0), -(0,202,142,'ABCD-1234','FIX',0), -(0,204,166,'Nails','FIX',0), -(0,205,166,'6C','FIX',0), -(211,146,21,'°','FIX',0), -(211,147,21,'°','FIX',0), -(211,145,21,'m','FIX',0), -(211,144,21,'m','FIX',0); +(139,140,21,'°','FIX',0), +(139,141,21,'°','FIX',0), +(0,140,21,'°','FIX',0), +(0,141,21,'°','FIX',0), +(157,140,21,'°','FIX',0), +(157,141,21,'°','FIX',0), +(158,140,21,'°','FIX',0), +(158,141,21,'°','FIX',0), +(0,174,138,'abc','FIX',0), +(0,174,136,'siehe bla','FIX',1), +(0,180,110,'-6C','FIX',1), +(0,181,102,'sb','FIX',0), +(0,182,105,'Martha','FIX',0), +(0,182,102,'MM','FIX',1), +(0,182,106,'Musterfrau','FIX',2), +(0,182,104,'mm@example.com','FIX',3), +(0,183,162,'End date','FIX',1), +(0,183,162,'IGSN DOI','FIX',2), +(0,183,162,'Curator','FIX',0), +(0,191,114,'32','FIX',2), +(0,191,116,'somethign','FIX',6), +(0,190,114,'32','FIX',2), +(0,189,114,'32','FIX',2), +(193,140,21,'°','FIX',0), +(193,141,21,'°','FIX',0), +(195,140,21,'°','FIX',0), +(195,141,21,'°','FIX',0), +(197,140,21,'°','FIX',0), +(197,141,21,'°','FIX',0); /*!40000 ALTER TABLE `text_data` ENABLE KEYS */; UNLOCK TABLES; @@ -2259,7 +2201,7 @@ CREATE TABLE `transaction_log` ( `nanos` int(10) unsigned NOT NULL DEFAULT 0, `realm` varbinary(255) NOT NULL, KEY `entity_id` (`entity_id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2269,119 +2211,110 @@ CREATE TABLE `transaction_log` ( LOCK TABLES `transaction_log` WRITE; /*!40000 ALTER TABLE `transaction_log` DISABLE KEYS */; INSERT INTO `transaction_log` VALUES -('Insert','102','admin',1736264522,437000000,'PAM'), -('Insert','103','admin',1736264522,437000000,'PAM'), -('Insert','104','admin',1736264522,437000000,'PAM'), -('Insert','105','admin',1736264522,437000000,'PAM'), -('Insert','106','admin',1736264522,437000000,'PAM'), -('Insert','107','admin',1736264522,437000000,'PAM'), -('Insert','108','admin',1736264522,437000000,'PAM'), -('Insert','109','admin',1736264522,437000000,'PAM'), -('Insert','110','admin',1736264522,437000000,'PAM'), -('Insert','111','admin',1736264522,437000000,'PAM'), -('Insert','112','admin',1736264522,437000000,'PAM'), -('Insert','113','admin',1736264522,437000000,'PAM'), -('Insert','114','admin',1736264522,437000000,'PAM'), -('Insert','115','admin',1736264522,437000000,'PAM'), -('Insert','116','admin',1736264522,437000000,'PAM'), -('Insert','117','admin',1736264522,437000000,'PAM'), -('Insert','118','admin',1736264522,437000000,'PAM'), -('Insert','119','admin',1736264522,437000000,'PAM'), -('Insert','120','admin',1736264522,437000000,'PAM'), -('Insert','121','admin',1736264522,437000000,'PAM'), -('Insert','122','admin',1736264522,437000000,'PAM'), -('Insert','123','admin',1736264522,437000000,'PAM'), -('Insert','124','admin',1736264522,437000000,'PAM'), -('Insert','125','admin',1736264522,437000000,'PAM'), -('Insert','126','admin',1736264522,437000000,'PAM'), -('Insert','127','admin',1736264522,437000000,'PAM'), -('Insert','128','admin',1736264522,437000000,'PAM'), -('Insert','129','admin',1736264522,437000000,'PAM'), -('Insert','130','admin',1736264522,437000000,'PAM'), -('Insert','131','admin',1736264522,437000000,'PAM'), -('Insert','132','admin',1736264522,437000000,'PAM'), -('Insert','133','admin',1736264522,437000000,'PAM'), -('Insert','134','admin',1736264522,437000000,'PAM'), -('Insert','135','admin',1736264522,437000000,'PAM'), -('Insert','136','admin',1736264522,437000000,'PAM'), -('Insert','137','admin',1736264522,437000000,'PAM'), -('Insert','138','admin',1736264522,437000000,'PAM'), -('Insert','139','admin',1736264522,437000000,'PAM'), -('Insert','140','admin',1736264522,437000000,'PAM'), -('Insert','141','admin',1736264522,437000000,'PAM'), -('Insert','142','admin',1736264522,437000000,'PAM'), -('Insert','143','admin',1736264522,437000000,'PAM'), -('Insert','144','admin',1736264522,437000000,'PAM'), -('Insert','145','admin',1736264522,437000000,'PAM'), -('Insert','146','admin',1736264522,437000000,'PAM'), -('Insert','147','admin',1736264522,437000000,'PAM'), -('Insert','148','admin',1736264522,437000000,'PAM'), -('Insert','149','admin',1736264522,437000000,'PAM'), -('Insert','150','admin',1736264522,437000000,'PAM'), -('Insert','151','admin',1736264522,437000000,'PAM'), -('Insert','152','admin',1736264522,437000000,'PAM'), -('Insert','153','admin',1736264522,437000000,'PAM'), -('Insert','154','admin',1736264522,437000000,'PAM'), -('Insert','155','admin',1736264522,437000000,'PAM'), -('Insert','156','admin',1736264522,437000000,'PAM'), -('Insert','157','admin',1736264522,437000000,'PAM'), -('Insert','158','admin',1736264522,437000000,'PAM'), -('Insert','159','admin',1736264522,437000000,'PAM'), -('Insert','160','admin',1736264522,437000000,'PAM'), -('Insert','161','admin',1736264522,437000000,'PAM'), -('Insert','162','admin',1736264522,437000000,'PAM'), -('Insert','163','admin',1736264522,437000000,'PAM'), -('Insert','164','admin',1736264522,437000000,'PAM'), -('Insert','165','admin',1736264522,437000000,'PAM'), -('Insert','166','admin',1736264522,437000000,'PAM'), -('Insert','167','admin',1736264522,437000000,'PAM'), -('Insert','168','admin',1736264522,437000000,'PAM'), -('Insert','169','admin',1736264522,437000000,'PAM'), -('Insert','170','admin',1736264522,437000000,'PAM'), -('Insert','171','admin',1736264522,437000000,'PAM'), -('Insert','172','admin',1736264522,437000000,'PAM'), -('Insert','173','admin',1736264522,437000000,'PAM'), -('Insert','174','admin',1736264522,437000000,'PAM'), -('Insert','175','admin',1736264522,437000000,'PAM'), -('Insert','176','admin',1736264522,437000000,'PAM'), -('Insert','177','admin',1736264522,437000000,'PAM'), -('Insert','178','admin',1736264522,437000000,'PAM'), -('Insert','179','admin',1736264603,389000000,'PAM'), -('Insert','180','admin',1736264663,736000000,'PAM'), -('Insert','181','admin',1736264700,733000000,'PAM'), -('Insert','182','admin',1736264727,998000000,'PAM'), -('Insert','183','admin',1736264740,329000000,'PAM'), -('Insert','184','admin',1736264752,282000000,'PAM'), -('Update','181','admin',1736264805,952000000,'PAM'), -('Insert','185','admin',1736265204,451000000,'PAM'), -('Insert','186','admin',1736265204,580000000,'PAM'), -('Update','186','admin',1736265204,802000000,'PAM'), -('Update','185','admin',1736265204,999000000,'PAM'), -('Insert','187','admin',1736265392,321000000,'PAM'), -('Insert','188','admin',1736265392,368000000,'PAM'), -('Update','188','admin',1736265392,478000000,'PAM'), -('Update','186','admin',1736265392,576000000,'PAM'), -('Update','187','admin',1736265392,704000000,'PAM'), -('Insert','189','admin',1736265441,647000000,'PAM'), -('Insert','190','admin',1736265441,688000000,'PAM'), -('Update','190','admin',1736265441,780000000,'PAM'), -('Update','188','admin',1736265441,871000000,'PAM'), -('Update','189','admin',1736265441,988000000,'PAM'), -('Insert','191','admin',1736265488,391000000,'PAM'), -('Insert','192','admin',1736265602,369000000,'PAM'), -('Insert','193','admin',1736265619,737000000,'PAM'), -('Insert','194','admin',1736265656,336000000,'PAM'), -('Insert','195','admin',1736265702,487000000,'PAM'), -('Insert','196','admin',1736265742,101000000,'PAM'), -('Insert','197','admin',1736265770,375000000,'PAM'), -('Insert','198','admin',1736265798,309000000,'PAM'), -('Insert','199','admin',1736265822,771000000,'PAM'), -('Insert','200','admin',1736265841,217000000,'PAM'), -('Insert','201','admin',1736265865,579000000,'PAM'), -('Insert','202','admin',1736265896,593000000,'PAM'), -('Insert','203','admin',1736265919,980000000,'PAM'), -('Insert','204','admin',1736265919,980000000,'PAM'), -('Update','191','admin',1736265920,63000000,'PAM'); +('Insert','102','admin',1741701919,236000000,'PAM'), +('Insert','103','admin',1741701919,236000000,'PAM'), +('Insert','104','admin',1741701919,236000000,'PAM'), +('Insert','105','admin',1741701919,236000000,'PAM'), +('Insert','106','admin',1741701919,236000000,'PAM'), +('Insert','107','admin',1741701919,236000000,'PAM'), +('Insert','108','admin',1741701919,236000000,'PAM'), +('Insert','109','admin',1741701919,236000000,'PAM'), +('Insert','110','admin',1741701919,236000000,'PAM'), +('Insert','111','admin',1741701919,236000000,'PAM'), +('Insert','112','admin',1741701919,236000000,'PAM'), +('Insert','113','admin',1741701919,236000000,'PAM'), +('Insert','114','admin',1741701919,236000000,'PAM'), +('Insert','115','admin',1741701919,236000000,'PAM'), +('Insert','116','admin',1741701919,236000000,'PAM'), +('Insert','117','admin',1741701919,236000000,'PAM'), +('Insert','118','admin',1741701919,236000000,'PAM'), +('Insert','119','admin',1741701919,236000000,'PAM'), +('Insert','120','admin',1741701919,236000000,'PAM'), +('Insert','121','admin',1741701919,236000000,'PAM'), +('Insert','122','admin',1741701919,236000000,'PAM'), +('Insert','123','admin',1741701919,236000000,'PAM'), +('Insert','124','admin',1741701919,236000000,'PAM'), +('Insert','125','admin',1741701919,236000000,'PAM'), +('Insert','126','admin',1741701919,236000000,'PAM'), +('Insert','127','admin',1741701919,236000000,'PAM'), +('Insert','128','admin',1741701919,236000000,'PAM'), +('Insert','129','admin',1741701919,236000000,'PAM'), +('Insert','130','admin',1741701919,236000000,'PAM'), +('Insert','131','admin',1741701919,236000000,'PAM'), +('Insert','132','admin',1741701919,236000000,'PAM'), +('Insert','133','admin',1741701919,236000000,'PAM'), +('Insert','134','admin',1741701919,236000000,'PAM'), +('Insert','135','admin',1741701919,236000000,'PAM'), +('Insert','136','admin',1741701919,236000000,'PAM'), +('Insert','137','admin',1741701919,236000000,'PAM'), +('Insert','138','admin',1741701919,236000000,'PAM'), +('Insert','139','admin',1741701919,236000000,'PAM'), +('Insert','140','admin',1741701919,236000000,'PAM'), +('Insert','141','admin',1741701919,236000000,'PAM'), +('Insert','142','admin',1741701919,236000000,'PAM'), +('Insert','143','admin',1741701919,236000000,'PAM'), +('Insert','144','admin',1741701919,236000000,'PAM'), +('Insert','145','admin',1741701919,236000000,'PAM'), +('Insert','146','admin',1741701919,236000000,'PAM'), +('Insert','147','admin',1741701919,236000000,'PAM'), +('Insert','148','admin',1741701919,236000000,'PAM'), +('Insert','149','admin',1741701919,236000000,'PAM'), +('Insert','150','admin',1741701919,236000000,'PAM'), +('Insert','151','admin',1741701919,236000000,'PAM'), +('Insert','152','admin',1741701919,236000000,'PAM'), +('Insert','153','admin',1741701919,236000000,'PAM'), +('Insert','154','admin',1741701919,236000000,'PAM'), +('Insert','155','admin',1741701919,236000000,'PAM'), +('Insert','156','admin',1741701919,236000000,'PAM'), +('Insert','157','admin',1741701919,236000000,'PAM'), +('Insert','158','admin',1741701919,236000000,'PAM'), +('Insert','159','admin',1741701919,236000000,'PAM'), +('Insert','160','admin',1741701919,236000000,'PAM'), +('Insert','161','admin',1741701919,236000000,'PAM'), +('Insert','162','admin',1741701919,236000000,'PAM'), +('Insert','163','admin',1741701919,236000000,'PAM'), +('Insert','164','admin',1741702140,85000000,'PAM'), +('Insert','165','admin',1741702165,542000000,'PAM'), +('Insert','166','admin',1741702183,776000000,'PAM'), +('Insert','167','admin',1741702209,101000000,'PAM'), +('Insert','168','admin',1741702225,299000000,'PAM'), +('Insert','169','admin',1741702250,689000000,'PAM'), +('Insert','170','admin',1741702276,591000000,'PAM'), +('Insert','171','admin',1741702289,139000000,'PAM'), +('Insert','172','admin',1741702333,33000000,'PAM'), +('Insert','173','admin',1741702347,74000000,'PAM'), +('Insert','174','admin',1741702362,280000000,'PAM'), +('Insert','175','admin',1741702380,899000000,'PAM'), +('Insert','176','admin',1741702402,373000000,'PAM'), +('Insert','177','admin',1741702453,376000000,'PAM'), +('Insert','178','admin',1741702512,658000000,'PAM'), +('Insert','179','admin',1741702528,907000000,'PAM'), +('Insert','180','admin',1741702528,907000000,'PAM'), +('Insert','181','admin',1741702528,907000000,'PAM'), +('Insert','182','admin',1741702565,365000000,'PAM'), +('Insert','183','admin',1741702792,705000000,'PAM'), +('Delete','183','admin',1741702796,807000000,'PAM'), +('Update','115','admin',1741702929,507000000,'PAM'), +('Insert','184','admin',1741702954,287000000,'PAM'), +('Insert','185','admin',1741702954,287000000,'PAM'), +('Insert','186','admin',1741702954,287000000,'PAM'), +('Update','182','admin',1741702954,341000000,'PAM'), +('Update','186','admin',1741703106,793000000,'PAM'), +('Update','185','admin',1741703106,793000000,'PAM'), +('Update','184','admin',1741703106,793000000,'PAM'), +('Update','122','admin',1741704265,193000000,'PAM'), +('Insert','187','admin',1741704540,640000000,'PAM'), +('Insert','188','admin',1741704540,640000000,'PAM'), +('Insert','189','admin',1741704540,640000000,'PAM'), +('Insert','190','admin',1741704540,640000000,'PAM'), +('Insert','191','admin',1741704540,640000000,'PAM'), +('Insert','192','admin',1741704540,640000000,'PAM'), +('Update','181','admin',1741704540,693000000,'PAM'), +('Update','180','admin',1741704540,693000000,'PAM'), +('Update','179','admin',1741704540,693000000,'PAM'), +('Update','191','admin',1741705215,273000000,'PAM'), +('Update','189','admin',1741705215,273000000,'PAM'), +('Update','187','admin',1741705215,273000000,'PAM'); /*!40000 ALTER TABLE `transaction_log` ENABLE KEYS */; UNLOCK TABLES; @@ -2399,7 +2332,7 @@ CREATE TABLE `transactions` ( `seconds` bigint(20) unsigned NOT NULL, `nanos` int(10) unsigned NOT NULL, PRIMARY KEY (`srid`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2409,44 +2342,39 @@ CREATE TABLE `transactions` ( LOCK TABLES `transactions` WRITE; /*!40000 ALTER TABLE `transactions` DISABLE KEYS */; INSERT INTO `transactions` VALUES -('032bc38a-28d3-4c59-9d31-1494b62a2ba4','admin','PAM',1736265920,63000000), -('06147cec-8ac2-4c72-9744-cab44e960e09','admin','PAM',1736265865,579000000), -('081a60fb-04b9-4d71-b025-d865b503fc7e','admin','PAM',1736265822,771000000), -('20f056fc-7deb-4c52-a54f-a80a92236315','admin','PAM',1736264636,79000000), -('22245d86-e095-4514-917a-7677616c544b','admin','PAM',1736265602,369000000), -('27f2534b-49b9-4e44-9232-53eadf8ba072','admin','PAM',1736264522,437000000), -('2d5a4ba7-517f-4b85-9727-b3d96988c485','admin','PAM',1736265392,478000000), -('2df660f0-e3a6-447e-837f-4778e5323f5d','admin','PAM',1736265392,576000000), -('34fade63-2811-4725-becf-0f2b1f84ad90','admin','PAM',1736265204,999000000), -('3c931e13-7e9f-4835-99df-d325020b887d','admin','PAM',1736265488,391000000), -('488967a3-a042-43d4-b74d-485f4bf622a2','admin','PAM',1736265441,647000000), -('55dcaf7b-7372-4778-995a-a6a8bde1b471','admin','PAM',1736264805,952000000), -('7663f26d-982f-4710-aeab-073a44290f3a','admin','PAM',1736265841,217000000), -('8205d330-42ef-4519-9e5e-ade1e2465bed','admin','PAM',1736265392,368000000), -('84bd4faf-510c-4950-b827-4bab81546887','admin','PAM',1736264663,736000000), -('89a537e0-496a-4e50-b078-74f34cd21a07','admin','PAM',1736265441,688000000), -('975ea8ec-ba79-452e-bc44-2d9432ce4993','admin','PAM',1736265742,101000000), -('9d2c4381-148f-4559-9d30-95a328169bbf','admin','PAM',1736265204,451000000), -('a41c5d15-6c9e-4e5c-a8fb-2fa7d38da140','admin','PAM',1736264752,282000000), -('aa019373-730c-4cec-8f83-3eb2f2ee9629','admin','PAM',1736265619,737000000), -('ada94a1f-e942-46e2-a5de-626a18cd2af9','admin','PAM',1736265702,487000000), -('b29fcae5-7964-4beb-b73a-2f7ce0a4809f','admin','PAM',1736265441,871000000), -('b51ffa82-bcf0-42a1-922c-9f4a845e6f42','admin','PAM',1736265919,980000000), -('c03fb8ed-a263-4997-a02b-37bf97af9563','admin','PAM',1736265441,780000000), -('c6549725-13bb-4076-9d49-219c6b8192f3','admin','PAM',1736264700,733000000), -('ca9db47b-d2d5-4980-84da-7268869f9e3c','admin','PAM',1736264603,389000000), -('cad02c5b-f559-4400-9897-80e56c1035ba','admin','PAM',1736264740,329000000), -('cbb111a6-b62c-4705-b1c4-36f055b3c815','admin','PAM',1736265656,336000000), -('cd4b5be6-82c4-4444-bb3e-143ea3c1fcfe','admin','PAM',1736265896,593000000), -('ce298544-326b-4b86-96fd-03de11548c9b','admin','PAM',1736264727,998000000), +('03f50600-7e2b-47b9-bc05-bfd5a7933304','admin','PAM',1741702792,705000000), +('0c134ec8-b564-42c9-a02f-cfb234a0c45a','admin','PAM',1741702165,542000000), +('0c173ae6-d2cd-4b30-a759-2d91919b0a1e','admin','PAM',1741702929,507000000), +('15c567d7-18dd-4484-a9d5-29246cdd33ae','admin','PAM',1741702565,365000000), +('297074fa-3d95-4ee9-bf08-26ff31b8e1fc','admin','PAM',1741704540,693000000), +('29851bcb-1bef-4829-9f65-4260a2a68b48','admin','PAM',1741702402,373000000), +('2e333e60-9884-46a4-8661-ae777bc81c6c','admin','PAM',1741702225,299000000), +('302ea332-e488-40d6-aa0b-9b78c7f9546f','admin','PAM',1741701919,236000000), +('359ed82b-8c2d-4a00-b00e-19d4e38ceb8d','admin','PAM',1741702209,101000000), +('37756bb9-6856-419d-b56a-2c1e8b9941ea','admin','PAM',1741701851,46000000), +('3b35d558-9029-40f5-8380-462d978c8ce3','admin','PAM',1741701887,789000000), +('427f72b6-f28e-4b3d-8bb9-0c668f68d788','admin','PAM',1741705215,273000000), +('54969a5b-8ad4-4135-8117-b4800c29bd04','admin','PAM',1741702140,85000000), +('586d8321-a31d-4251-af76-aacd4feaa35a','admin','PAM',1741702276,591000000), +('69a23164-c576-440c-bba0-d3bcc06adadd','admin','PAM',1741701752,841000000), +('72507861-7c79-4d8a-b121-db00b523a7b9','admin','PAM',1741702954,341000000), +('9091b57d-818e-4968-abee-c80780de2a4e','admin','PAM',1741702453,376000000), +('98cd3884-f6e2-40cd-81e8-ac8738bab3e9','admin','PAM',1741703106,793000000), +('a2fcd44f-17be-4181-b7f0-2d60cd894ccd','admin','PAM',1741702347,74000000), +('a71ff821-afb8-4f0d-9cee-d33a155ac395','admin','PAM',1741702333,33000000), +('a7994338-93c3-4cb7-9eaa-e0ea57de0069','admin','PAM',1741702289,139000000), +('b9286cef-88c4-4e3d-90e1-ee09a76feae8','admin','PAM',1741702954,287000000), +('bb848008-57b3-427d-876a-c5ff774d3d88','admin','PAM',1741704540,640000000), +('bdf470b8-c562-4e4d-a2f2-1f2680822aed','admin','PAM',1741702512,658000000), ('cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e','administration','CaosDB',0,0), -('d0af8a57-7550-42b2-aba9-4b254897976b','admin','PAM',1736265392,704000000), -('da56170f-ae54-4ecf-9aa4-342d7512384f','admin','PAM',1736265798,309000000), -('daa492ef-1495-4123-9966-9951328211b3','admin','PAM',1736265204,580000000), -('dbf1dd0f-405c-4952-b8b6-62e169c79397','admin','PAM',1736265204,802000000), -('df3a0e97-6feb-42aa-8800-074ee5520428','admin','PAM',1736265392,321000000), -('f284f0d8-aba2-44e9-bfc7-ae68fab970b9','admin','PAM',1736265770,375000000), -('fe535f2d-6865-41b7-bab1-781f412454f6','admin','PAM',1736265441,988000000); +('d66682c8-1138-4734-be3d-3e7f491899b2','admin','PAM',1741702380,899000000), +('dcd7eccb-3e38-4c04-9a98-0159bda1e2c6','admin','PAM',1741704265,193000000), +('df090b99-7e9b-43ba-a461-367778e17068','admin','PAM',1741702362,280000000), +('f2003a25-cbfa-4513-ac3d-2cfc5ca7f2f4','admin','PAM',1741702565,386000000), +('f4a55cb7-a8f1-407e-9254-9de3402cd960','admin','PAM',1741702796,807000000), +('f8cff7ae-db88-47cd-9f9e-2877478a8856','admin','PAM',1741702183,776000000), +('fa408261-eb8d-4da1-9ed6-026e344ec756','admin','PAM',1741702250,689000000), +('fbab0a14-0ffb-4385-84c7-9d003b932757','admin','PAM',1741702528,907000000); /*!40000 ALTER TABLE `transactions` ENABLE KEYS */; UNLOCK TABLES; @@ -2465,7 +2393,7 @@ CREATE TABLE `units_lin_con` ( `b_divisor` int(11) NOT NULL, `c` decimal(65,30) NOT NULL, PRIMARY KEY (`signature_from`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2493,7 +2421,7 @@ CREATE TABLE `user_info` ( PRIMARY KEY (`realm`,`name`), KEY `subject_entity` (`entity`), CONSTRAINT `subjects_ibfk_2` FOREIGN KEY (`entity`) REFERENCES `entity_ids` (`id`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2521,7 +2449,7 @@ CREATE TABLE `user_roles` ( PRIMARY KEY (`realm`,`user`,`role`), KEY `user_roles_ibfk_1` (`role`), CONSTRAINT `user_roles_ibfk_1` FOREIGN KEY (`role`) REFERENCES `roles` (`name`) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci; +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_as_ci; /*!40101 SET character_set_client = @saved_cs_client */; -- @@ -2544,13 +2472,13 @@ UNLOCK TABLES; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `CaosDBVersion`() RETURNS varchar(255) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `CaosDBVersion`() RETURNS varchar(255) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC -RETURN 'v8.0.0' ;; +RETURN 'v8.1.0' ;; DELIMITER ; /*!50003 SET sql_mode = @saved_sql_mode */ ; /*!50003 SET character_set_client = @saved_cs_client */ ; @@ -2562,11 +2490,11 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `constructDateTimeWhereClauseForColumn`(seconds_col VARCHAR(255), nanos_col VARCHAR(255), vDateTimeSecLow VARCHAR(255), vDateTimeNSLow VARCHAR(255), vDateTimeSecUpp VARCHAR(255), vDateTimeNSUpp VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `constructDateTimeWhereClauseForColumn`(seconds_col VARCHAR(255), nanos_col VARCHAR(255), vDateTimeSecLow VARCHAR(255), vDateTimeNSLow VARCHAR(255), vDateTimeSecUpp VARCHAR(255), vDateTimeNSUpp VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC BEGIN @@ -2631,9 +2559,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `convert_unit`(unit_sig BIGINT, value DECIMAL(65,30)) RETURNS decimal(65,30) DETERMINISTIC @@ -2654,11 +2582,11 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `getAggValueWhereClause`(entities VARCHAR(255), properties VARCHAR(255)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `getAggValueWhereClause`(entities VARCHAR(255), properties VARCHAR(255)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC BEGIN RETURN CONCAT(" EXISTS (SELECT 1 FROM `", entities, "` AS ent WHERE ent.id = subdata.entity_id LIMIT 1)", IF(properties IS NOT NULL AND properties != '', CONCAT(" AND EXISTS (SELECT 1 FROM `", properties, "` as props WHERE props.id = subdata.property_id LIMIT 1)"),'')); @@ -2674,11 +2602,11 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `getDateTimeWhereClause`(vDateTime VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `getDateTimeWhereClause`(vDateTime VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC BEGIN DECLARE sep_loc INTEGER DEFAULT LOCATE('--',vDateTime); @@ -2705,11 +2633,11 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `getDateWhereClause`(vDateTimeDotNotation VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `getDateWhereClause`(vDateTimeDotNotation VARCHAR(255), operator CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC BEGIN DECLARE isInterval INTEGER DEFAULT LOCATE('--',vDateTimeDotNotation); @@ -2814,11 +2742,11 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; -CREATE DEFINER=`root`@`%` FUNCTION `getDoubleWhereClause`(value DOUBLE, unit_sig BIGINT, valueStdUnit DECIMAL(65,30), stdUnit_sig BIGINT, o CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci +CREATE DEFINER=`root`@`%` FUNCTION `getDoubleWhereClause`(value DOUBLE, unit_sig BIGINT, valueStdUnit DECIMAL(65,30), stdUnit_sig BIGINT, o CHAR(4)) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci DETERMINISTIC BEGIN RETURN IF(unit_sig IS NULL AND value IS NOT NULL, @@ -2839,9 +2767,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `get_head_relative`(EntityID VARCHAR(255), HeadOffset INT UNSIGNED) RETURNS varbinary(255) @@ -2874,9 +2802,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `get_head_version`(EntityID VARCHAR(255)) RETURNS varbinary(255) READS SQL DATA @@ -2894,9 +2822,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `get_iversion`(InternalEntityID INT UNSIGNED, Version VARBINARY(255)) RETURNS int(10) unsigned @@ -2920,9 +2848,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `get_primary_parent_version`(EntityID VARCHAR(255), Version VARBINARY(255)) RETURNS varbinary(255) @@ -2952,12 +2880,12 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `get_version_timestamp`(EntityID VARCHAR(255), - Version VARBINARY(255)) RETURNS varchar(255) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci + Version VARBINARY(255)) RETURNS varchar(255) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci READS SQL DATA BEGIN DECLARE InternalEntityID INT UNSIGNED DEFAULT NULL; @@ -2983,9 +2911,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `is_feature_config`(_Key VARCHAR(255), Expected VARCHAR(255)) RETURNS tinyint(1) @@ -3006,12 +2934,12 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `makeStmt`(sourceSet VARCHAR(255), targetSet VARCHAR(255), data VARCHAR(20000), - properties VARCHAR(20000), versioned BOOLEAN) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_unicode_ci + properties VARCHAR(20000), versioned BOOLEAN) RETURNS varchar(20000) CHARSET utf8mb3 COLLATE utf8mb3_uca1400_as_ci NO SQL BEGIN IF sourceSet = "entities" AND versioned THEN @@ -3064,9 +2992,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `standard_unit`(unit_sig BIGINT) RETURNS bigint(20) DETERMINISTIC @@ -3087,9 +3015,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `_get_head_iversion`(InternalEntityID INT UNSIGNED) RETURNS int(10) unsigned READS SQL DATA @@ -3117,9 +3045,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` FUNCTION `_get_version`(InternalEntityID INT UNSIGNED, IVersion INT UNSIGNED) RETURNS varbinary(255) @@ -3142,9 +3070,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applyBackReference`(in sourceSet VARCHAR(255), targetSet VARCHAR(255), in propertiesTable VARCHAR(255), in entitiesTable VARCHAR(255), in subQuery BOOLEAN, @@ -3323,9 +3251,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applyIDFilter`(in sourceSet VARCHAR(255), in targetSet VARCHAR(255), in o CHAR(2), in EntityID VARCHAR(255), in agg CHAR(3), in versioned BOOLEAN) @@ -3460,9 +3388,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applyPOV`(in sourceSet VARCHAR(255), in targetSet VARCHAR(255), @@ -3765,9 +3693,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applyRefPOV`(in sourceSet VARCHAR(255), in targetSet VARCHAR(255), in properties VARCHAR(255), in refs VARCHAR(255), @@ -3805,9 +3733,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applySAT`(in sourceSet VARCHAR(255), in targetSet VARCHAR(255), in loc MEDIUMTEXT, in op CHAR(5)) BEGIN @@ -3834,9 +3762,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `applyTransactionFilter`(in sourceSet VARCHAR(255), targetSet VARCHAR(255), in transaction VARCHAR(255), in operator_u CHAR(2), in realm VARCHAR(255), in userName VARCHAR(255), in ilb BIGINT, in ilb_nanos INT UNSIGNED, in eub BIGINT, in eub_nanos INT UNSIGNED, in operator_t CHAR(2)) BEGIN @@ -3876,9 +3804,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `calcComplementUnion`(in targetSet VARCHAR(255), in subResultSet VARCHAR(255), in universe VARCHAR(255), in versioned BOOLEAN) BEGIN @@ -3916,9 +3844,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `calcDifference`(in resultSetTable VARCHAR(255), in diff VARCHAR(255), in versioned BOOLEAN) BEGIN @@ -3943,9 +3871,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `calcIntersection`(in resultSetTable VARCHAR(255), in intersectWith VARCHAR(255), in versioned BOOLEAN) BEGIN @@ -3978,9 +3906,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `calcUnion`(in targetSet VARCHAR(255), in sourceSet VARCHAR(255)) BEGIN @@ -4001,9 +3929,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `cleanUpLinCon`() BEGIN @@ -4022,9 +3950,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `cleanUpQuery`() BEGIN @@ -4049,9 +3977,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `copyTable`(in fromTable VARCHAR(255), in toTable VARCHAR(255)) BEGIN @@ -4072,9 +4000,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `createTmpTable`(out newTableName VARCHAR(255), in versioned BOOLEAN) BEGIN @@ -4102,9 +4030,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `createTmpTable2`(out newTableName VARCHAR(255)) BEGIN @@ -4128,9 +4056,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `deleteEntity`(in EntityID VARCHAR(255)) BEGIN @@ -4184,9 +4112,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `deleteEntityProperties`(in EntityID VARCHAR(255)) BEGIN @@ -4357,9 +4285,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `deleteIsa`(IN InternalEntityID INT UNSIGNED) BEGIN @@ -4395,9 +4323,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `deleteLinCon`(in sig BIGINT) BEGIN @@ -4416,9 +4344,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `entityACL`(out ACLID INT UNSIGNED, in ACLSTR VARBINARY(65525)) BEGIN @@ -4439,9 +4367,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `finishNegationFilter`(in resultSetTable VARCHAR(255), in diff VARCHAR(255)) BEGIN @@ -4458,9 +4386,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `finishSubProperty`(in sourceSet VARCHAR(255),in targetSet VARCHAR(255), in list VARCHAR(255), in versioned BOOLEAN) @@ -4486,9 +4414,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `getChildren`(in tableName varchar(255), in versioned BOOLEAN) BEGIN @@ -4555,9 +4483,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `getDependentEntities`(in EntityID VARCHAR(255)) BEGIN @@ -4659,9 +4587,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `getFileIdByPath`(in FilePath TEXT) BEGIN @@ -4680,9 +4608,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `getIdByName`(in Name VARCHAR(255), in Role VARCHAR(255), in Lmt INT UNSIGNED) BEGIN @@ -4743,9 +4671,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `get_version_history`( in EntityID VARCHAR(255)) @@ -4796,9 +4724,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initBackReference`(in PropertyID VARCHAR(255), in PropertyName VARCHAR(255), in EntityID VARCHAR(255), in EntityName VARCHAR(255)) BEGIN @@ -4854,9 +4782,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initDisjunctionFilter`(in versioned BOOLEAN) BEGIN @@ -4873,9 +4801,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initEmptyTargetSet`(in targetSet VARCHAR(255), in versioned BOOLEAN) BEGIN @@ -4905,9 +4833,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initEntity`(in eid VARCHAR(255), in ename VARCHAR(255), in enameLike VARCHAR(255), in enameRegexp VARCHAR(255), @@ -5041,9 +4969,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initPOVPropertiesTable`(in PropertyID VARCHAR(255), in PropertyName VARCHAR(255), in sourceSet VARCHAR(255)) BEGIN @@ -5123,9 +5051,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initPOVRefidsTable`(in PropertyID VARCHAR(255), in PropertyName VARCHAR(255)) BEGIN @@ -5152,9 +5080,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initQuery`(in versioned BOOLEAN) BEGIN @@ -5175,9 +5103,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initSubEntity`(in EntityID VARCHAR(255), in ename VARCHAR(255), in tableName VARCHAR(255)) BEGIN @@ -5227,9 +5155,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `initSubProperty`(in sourceSet VARCHAR(255), in propertiesTable VARCHAR(255), in refIdsTable VARCHAR(255)) BEGIN @@ -5278,9 +5206,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertEntity`(in EntityID VARCHAR(255), in EntityName VARCHAR(255), in EntityDesc TEXT, in EntityRole VARCHAR(255), in ACL VARBINARY(65525)) BEGIN @@ -5332,9 +5260,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertEntityCollection`(in PropertyID VARCHAR(255), in Collection VARCHAR(255)) BEGIN @@ -5356,9 +5284,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertEntityDataType`(in PropertyID VARCHAR(255), in DataTypeID VARCHAR(255)) BEGIN @@ -5381,9 +5309,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertEntityProperty`( in DomainID VARCHAR(255), @@ -5523,9 +5451,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertIsa`(IN ChildID VARCHAR(255), IN ParentID VARCHAR(255)) insert_is_a_proc: BEGIN @@ -5588,9 +5516,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insertLinCon`(in signature_from BIGINT, in signature_to BIGINT, in a DECIMAL(65,30), in b_dividend BIGINT, in b_divisor BIGINT, in c DECIMAL(65,30)) BEGIN @@ -5609,9 +5537,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `insert_single_child_version`( in InternalEntityID INT UNSIGNED, @@ -5664,9 +5592,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `intersectTable`(in resultSetTable VARCHAR(255), in diff VARCHAR(255)) BEGIN @@ -5687,9 +5615,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `isSubtype`(in ChildID VARCHAR(255), in ParentID VARCHAR(255)) BEGIN @@ -5714,9 +5642,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `overrideDesc`(in InternalDomainID INT UNSIGNED, in InternalEntityID INT UNSIGNED, in InternalPropertyID INT UNSIGNED, in Description TEXT) BEGIN @@ -5733,9 +5661,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `overrideName`(in InternalDomainID INT UNSIGNED, in InternalEntityID INT UNSIGNED, in InternalPropertyID INT UNSIGNED, in Name VARCHAR(255)) BEGIN @@ -5752,9 +5680,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `overrideType`(in InternalDomainID INT UNSIGNED, in InternalEntityID INT UNSIGNED, in InternalPropertyID INT UNSIGNED, in InternalDataTypeID INT UNSIGNED) BEGIN @@ -5771,9 +5699,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `raiseWarning`(in str VARCHAR(20000)) BEGIN @@ -5790,9 +5718,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `registerReplacementIds`(in amount INT UNSIGNED) BEGIN @@ -5821,9 +5749,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `registerTempTableName`(out newTableName VARCHAR(255)) BEGIN @@ -5869,9 +5797,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `retrieveEntity`( in EntityID VARCHAR(255), @@ -6027,9 +5955,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `retrieveEntityParents`( in EntityID VARCHAR(255), @@ -6108,9 +6036,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `retrieveEntityProperties`( in DomainID VARCHAR(255), @@ -6430,9 +6358,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `retrieveOverrides`( in DomainID VARCHAR(255), @@ -6625,9 +6553,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `retrieveQueryTemplateDef`( in EntityID VARCHAR(255), @@ -6680,9 +6608,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `setFileProperties`( in EntityID VARCHAR(255), @@ -6727,9 +6655,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `set_transaction`( srid VARBINARY(255), @@ -6754,9 +6682,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `showEntityAutoIncr`() BEGIN @@ -6778,9 +6706,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `updateEntity`( in EntityID VARCHAR(255), @@ -6892,9 +6820,9 @@ DELIMITER ; /*!50003 SET @saved_cs_client = @@character_set_client */ ; /*!50003 SET @saved_cs_results = @@character_set_results */ ; /*!50003 SET @saved_col_connection = @@collation_connection */ ; -/*!50003 SET character_set_client = utf8mb3 */ ; -/*!50003 SET character_set_results = utf8mb3 */ ; -/*!50003 SET collation_connection = utf8mb3_general_ci */ ; +/*!50003 SET character_set_client = latin1 */ ; +/*!50003 SET character_set_results = latin1 */ ; +/*!50003 SET collation_connection = latin1_swedish_ci */ ; DELIMITER ;; CREATE DEFINER=`root`@`%` PROCEDURE `updateLinCon`(in sig_from BIGINT, in sig_to BIGINT, in new_a DECIMAL(65,30), in new_b_dividend BIGINT, in new_b_divisor BIGINT, in new_c DECIMAL(65,30)) BEGIN @@ -6916,4 +6844,4 @@ DELIMITER ; /*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */; /*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */; --- Dump completed on 2025-01-07 16:05:15 +-- Dump completed on 2025-03-11 15:15:22 diff --git a/test-profile/custom/other/restore/caosroot.2025-03-11T16151741706122.tar.gz b/test-profile/custom/other/restore/caosroot.2025-03-11T16151741706122.tar.gz new file mode 100644 index 0000000000000000000000000000000000000000..3318963c255112ead04d39119581dc5972ab58a6 Binary files /dev/null and b/test-profile/custom/other/restore/caosroot.2025-03-11T16151741706122.tar.gz differ diff --git a/test-profile/custom/other/restore/caosroot.tar.gz b/test-profile/custom/other/restore/caosroot.tar.gz deleted file mode 100644 index 63e35ca3beaa34a5406928fc902d002cac087289..0000000000000000000000000000000000000000 Binary files a/test-profile/custom/other/restore/caosroot.tar.gz and /dev/null differ