DOC: added extensive documentation

src/doc/conf.py

@@ -182,10 +182,16 @@ epub_exclude_files = ['search.html']
 
 # -- Extension configuration -------------------------------------------------
 
-# -- Options for intersphinx extension ---------------------------------------
+# -- Options for intersphinx -------------------------------------------------
+
+# https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#confval-intersphinx_mapping
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/", None),
+    "caosdb-mysqlbackend": ("https://caosdb.gitlab.io/caosdb-mysqlbackend/",
+                            None),
+    "caosdb-server": ("https://caosdb.gitlab.io/caosdb-server/", None),
+}
 
-# Example configuration for intersphinx: refer to the Python standard library.
-intersphinx_mapping = {'https://docs.python.org/': None}
 
 # TODO Which options do we want?
 autodoc_default_options = {

src/doc/tutorials/Data-Insertion.rst

+Data Insertion
+==============
+
+Data Models
+~~~~~~~~~~~
+
+Data is stored and structured in CaosDB using a concept of RecordTypes,
+Properties, Records etc. If you do not know what these are, please look
+at the chapter :any:`caosdb-server:Data Model` .
+
+In order to insert some actual data, we need to create a data model
+using RecordTypes and Properties (You may skip this if you use a CaosDB
+instance that already has the required types). So, let’s create a simple
+Property called “a” of datatype double. This is very easy in pylib:
+
+.. code:: python
+
+   a = db.Property(name="a", datatype=db.DOUBLE)
+
+There are a few basic datatypes: db.INTEGER, db.TEXT. See `data
+type <Specification/Datatype>`__ for a full list.
+
+We can create our own small data model for e.g. a simulation by adding
+two more Properties and a RecordType:
+
+.. code:: python
+
+   b = db.Property(name="b", datatype=db.DOUBLE)
+   epsilon = db.Property(name="epsilon", datatype=db.DOUBLE)
+   recordtype = db.RecordType(name="BarkleySimulation")
+   recordtype.add_property(a)
+   recordtype.add_property(b)
+   recordtype.add_property(epsilon)
+   container = db.Container()
+   container.extend([a, b, epsilon, recordtype])
+   container.insert()
+
+Insert Actual Data
+~~~~~~~~~~~~~~~~~~
+
+Suppose the RecordType “Experiment” and the Property “date” exist in the
+database. You can then create single data Records by using the
+corresponding python class:
+
+.. code:: python
+
+   rec = db.Record()
+   rec.add_parent(name="Experiment")
+   rec.add_property(name="date", value="2020-01-07")
+   rec.insert()
+
+Here, the record has a parent: The RecordType “Experiment”. And a
+Property: date.
+
+Note, that if you want to use a property that is not a primitive
+datatype like db.INTEGER and so on, you need to use the ID of the Entity
+that you are referencing.
+
+.. code:: python
+
+   rec = db.Record()
+   rec.add_parent(name="Experiment")
+   rec.add_property(name="report", value=235507)
+   rec.add_property(name="Analysis", value=230007)
+   rec.insert()
+
+Of course, the IDs 235507 and 230007 need to exist in CaosDB. The first
+example shows how to use a db.REFERENCE Property (report) and the second
+shows that you can use any RecordType as Property to reference a Record
+that has such a parent.
+
+Most Records do not have name however it can absolutely make sense. In
+that case use the name argument when creating it. Another useful feature
+is the fact that properties can have units:
+
+.. code:: python
+
+   rec = db.Record("DeviceNo-AB110")
+   rec.add_parent(name="SlicingMachine")
+   rec.add_property(name="weight", value="1749", unit="kg")
+   rec.insert()
+
+If you are in some kind of analysis you can do this in batch mode with a
+container. E.g. if you have a python list ``analysis_results``:
+
+.. code:: python
+
+   cont = db.Container()
+   for date, result in analysis_results:
+      rec = db.Record()
+      rec.add_parent(name="Experiment")
+      rec.add_property(name="date", value=date)
+      rec.add_property(name="result", value=result)
+      cont.append(rec)
+
+   cont.insert()
+
+Useful is also, that you can insert directly tabular data.
+
+.. code:: python
+
+   from caosadvancedtools.table_converter import from_tsv     
+          
+   recs = from_tsv("test.csv", "Experiment")     
+   print(recs)     
+   recs.insert()  
+
+With this example file
+`test.csv <uploads/4f2c8756a26a3984c0af09d206d583e5/test.csv>`__.
+
+Inheritance of Properties
+-------------------------
+
+Given, you want to insert a new RecordType “Fridge temperatur
+experiment” as a child of the existing RecordType “Experiment”. The
+latter may have an obligatory Property “date” (since every experiment is
+conducted at some time). It is a natural way of thinking, that every sub
+type of “Experiment” also has this obligatory Property—in terms of
+object oriented programing the “Fridge temperatur experiment” *inherits*
+that Property.
+
+::
+
+       rt = h.RecordType(name="Fridge temperatur experiment", 
+                                 description="RecordType which inherits all obligatory properties from Experiment"
+                                 ).add_parent(name="Experiment", inheritance="obligatory").insert()
+       
+       print(rt.get_property(name="date").importance) ### rt now has a "date"-property -> this line prints "obligatory"
+
+The parameter *``inheritance=(obligatory|recommended|fix|all|none)``* of
+``add_parent`` tells the server to assign obligatory:: properties of the
+parent to the child automatically, recommended:: properties of the
+parent to the child automatically, fix:: properties of the parent to the
+child automatically, all:: properties of the parent to the child
+automatically, none:: of the properties of the parent to child
+automatically,
+
+File Update
+-----------
+
+Updating an existing file by uploading a new version.
+
+1. Retrieve the file record of interest, e.g. by ID:
+
+.. code:: python
+
+   import caosdb as db
+
+   file_upd = db.File(id=174).retrieve()
+
+2. Set the new local file path. The remote file path is stored in the
+   file object as ``file_upd.path`` while the local path can be found in
+   ``file_upd.file``.
+
+.. code:: python
+
+   file_upd.file = "./supplements.pdf"
+
+3. Update the file:
+
+.. code:: python
+
+   file_upd.update()

src/doc/tutorials/data-model-interface.md

+# Data Models
+
+
+
+You also want to change the datamodel? Also call
+```bash
+pip3 install --user --no-deps .
+```
+in 
+```bash
+CaosDB/data_models
+```
+
+Change to the appropriate directory
+```bash
+cd CaosDB/data_models
+```
+There are "data models" defined in 
+```bash
+caosdb_models
+```
+having an ending like "_model.py"
+A set of data models is also considered to be a model
+You can create an UML representation of a model or a set of models by calling
+```bash
+./model_interface.py -u model_name [model_name2]
+```
+If you have troubles look at
+```bash
+./model_interface.py -h
+```
+You can change existing models (but be careful! I hope you know what you are doing) or add new ones by changing the appropriate files or adding a new XXXX_model.py
+Once you are done, you can sync your changes with the server
+```bash
+./model_interface.py -s model_name [model_name2]
+```

src/doc/tutorials/errors.rst

+
+Error Handling
+--------------
+
+HeartDBException
+~~~~~~~~~~~~~~~~
+
+TransactionError
+~~~~~~~~~~~~~~~~
+
+Every transaction (calling ``insert``, ``update``, ``retrieve``, or
+``delete`` on a container or an entity) may finish with errors. They
+indicate, for instance, that an entity does not exist or that you need
+to specify a data type for your property and much more. If and only if
+one or more errors occur during a transaction a ``TransactionError``
+will be raised by the transaction method. The ``TransactionError`` class
+is a container for all errors which occur during a transaction. It can
+help you to find the crucial problems with your transaction by two
+important methods: \* ``get_errors()`` which returns a list of instances
+of ``EntityError``. \* ``get_entities()`` which returns a list of
+entities in the transaction container which are erroneous.
+
+Additionally, ``print(transaction_error`` prints a tree-like
+representation of all errors regarding the transaction in question.
+
+EntityError
+~~~~~~~~~~~
+
+An ``EntityError`` represents a single error that has been returned by
+the server. You might call \* ``get_entity()`` which returns the entity
+which caused the error. \* ``get_description()`` which returns a
+description of the error. \* ``get_code()`` which returns the error code
+(if specified) or 0 (if not).
+
+In fact, the ``EntityError`` class is a subclass of
+``TransactionError``. So, it inherits the ``get_entities()``. Unless
+overridden by subclasses of ``EntityError``, it return a list with only
+one item—the entity which caused this error. Similarly, unless
+overridden by subclasses, the ``get_errors()`` method returns a list
+with only one item—``[self]``.
+
+Special Errors
+~~~~~~~~~~~~~~
+
+Subclasses of ``EntityError`` for special purposes: \*
+``EntityDoesNotExistError`` \* ``EntityHasNoDataTypeError`` \*
+``UniqueNamesError`` \* ``UnqualifiedParentsError`` - overrides
+``get_entities()``: returns all parent entities with errors. - overrides
+``get_errors()``: returns a list of EntityErrors which have been caused
+by parent entities. \* ``UnqualifiedPropertiesError`` - overrides
+``get_entities()``: returns all properties with errors. - overrides
+``get_errors()``: returns a list of EntityErrors which have been caused
+by properties.

src/doc/tutorials/first_steps.rst

@@ -113,6 +113,8 @@ You can download files (if the LinkAhead server has access to them)
 The file will be saved under target_path.
 If the files are large data files, it is often a better idea to only retrieve the path of the file and access them via a local mount.
 
+
+
 Summary
 -------
 

src/doc/tutorials/index.rst

@@ -12,4 +12,7 @@ advanced usage of the Python client.
 
    first_steps
    basic_analysis
+   Data-Insertion
+   errors
+   data-model-interface