diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7b24c1ef360433601c298f4066ed8db99b7d8ead..9f8c131968c050fb18001b5dc7c5468d0ed26dae 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -93,15 +93,12 @@ build-testenv:
# Build the sphinx documentation and make it ready for deployment by Gitlab Pages
# Special job for serving a static website. See https://docs.gitlab.com/ee/ci/yaml/README.html#pages
-pages:
+pages_prepare: &pages_prepare
+ tags: [ cached-dind ]
stage: deploy
only:
refs:
- /^release-.*$/i
- - master
- variables:
- # run pages only on gitlab.com
- - $CI_SERVER_HOST == "gitlab.com"
script:
- echo "Deploying"
- make doc
@@ -109,3 +106,8 @@ pages:
artifacts:
paths:
- public
+pages:
+ <<: *pages_prepare
+ only:
+ refs:
+ - main
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1c8a144dc939f3a58d1f9346929ad5e32d62d213..38b470db9d36675ffcef0b7e1434a08c3be7f407 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,12 +9,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added ###
-* Entity State support (experimental, no StateModel support yet). See the
- `caosdb.State` class for more information.
-* `etag` property for the `caosdb.Query` class. The etag allows to debug the
- caching and to decide whether the server has changed between queries.
-* function `_read_config_files` to read `pycaosdb.ini` files from different paths.
-
### Changed ###
### Deprecated ###
@@ -22,26 +16,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Removed ###
### Fixed ###
-* #45 - test_config_ini_via_envvar
+
+* #53 Documentation of inheritance
### Security ###
-## [0.5.1] - 2021-02-12 ##
+## [0.5.2] - 2021-06-03 ##
### Added ###
+* Entity State support (experimental, no StateModel support yet). See the
+ `caosdb.State` class for more information.
+* `etag` property for the `caosdb.Query` class. The etag allows to debug the
+ caching and to decide whether the server has changed between queries.
+* function `_read_config_files` to read `pycaosdb.ini` files from different paths.
+
### Changed ###
+* Updated error-handling tutorial in documentation to reflect the new
+ error classes
+
### Deprecated ###
### Removed ###
### Fixed ###
-
-* #43 - Error with `execute_query` when server doesn't support query caching.
+* #45 - test_config_ini_via_envvar
### Security ###
+## [0.5.1] - 2021-02-12 ##
+
+### Fixed ###
+
+* #43 - Error with `execute_query` when server doesn't support query caching.
+
## [0.5.0] - 2021-02-11 ##
### Added ###
@@ -74,18 +83,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`QueryNotUniqueError` if no or more than one possible candidate is
found, respectively.
-### Deprecated ###
-
### Removed ###
* Dynamic exception type `EntityMultiError`.
* `get_something` functions from all error object in `exceptions.py`
* `AmbiguityException`
-### Fixed ###
-
-### Security ###
-
## [0.4.1] - 2021-02-10 ##
### Added ###
@@ -95,18 +98,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
entity.
* Automated documentation builds: `make doc`
-### Changed ###
-
-### Deprecated ###
-
-### Removed ###
-
### Fixed ###
* deepcopy of `_Messages` objects
-### Security ###
-
## [0.4.0] - 2020-07-17##
### Added ###
diff --git a/README.md b/README.md
index 55645700c1762213fa35a14cb8a37c36eb43066f..04b34cbc07c98e73740b13200ed83fe067af99d2 100644
--- a/README.md
+++ b/README.md
@@ -28,14 +28,20 @@ By participating, you are expected to uphold our [Code of Conduct](https://gitla
* You found a bug, have a question, or want to request a feature? Please
[create an issue](https://gitlab.com/caosdb/caosdb-pylib/-/issues).
-* You want to contribute code? Please fork the repository and create a merge
-request in GitLab and choose this repository as target. Make sure to select
-"Allow commits from members who can merge the target branch" under Contribution
-when creating the merge request. This allows our team to work with you on your request.
-- If you have a suggestion for the [documentation](https://docs.indiscale.com/caosdb-pylib/),
+* You want to contribute code?
+ * **Forking:** Please fork the repository and create a merge request in GitLab and choose this repository as
+ target. Make sure to select "Allow commits from members who can merge the target branch" under
+ Contribution when creating the merge request. This allows our team to work with you on your
+ request.
+ * **Code style:** This project adhers to the PEP8 recommendations, you can test your code style
+ using the `autopep8` tool (`autopep8 -i -r ./`). Please write your doc strings following the
+ [NumpyDoc](https://numpydoc.readthedocs.io/en/latest/format.html) conventions.
+* If you have a suggestion for the [documentation](https://docs.indiscale.com/caosdb-pylib/),
the preferred way is also a merge request as describe above (the documentation resides in `src/doc`).
However, you can also create an issue for it.
-- You can also contact us at **info (AT) caosdb.de**.
+* You can also contact us at **info (AT) caosdb.de** and join the
+ CaosDB community on
+ [#caosdb:matrix.org](https://matrix.to/#/!unwwlTfOznjEnMMXxf:matrix.org).
## License
diff --git a/README_SETUP.md b/README_SETUP.md
index 4980b850ecd9fdc10c9fd02c6437aeb5826578b3..9da548395073643c16539cef180c4d6412dd8d46 100644
--- a/README_SETUP.md
+++ b/README_SETUP.md
@@ -75,7 +75,7 @@ current working directory will be read additionally, if it exists.
Here, we will look at the most common configuration options. For a full and
comprehensive description please check out
-[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/master/examples/pycaosdb.ini)
+[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/main/examples/pycaosdb.ini)
You can download this file and use it as a starting point.
@@ -84,7 +84,7 @@ Typically, you need to change at least the `url` and `username` fields as requir
you do not know what to put there, but for the demo instances https://demo.indiscale.com, `username=admin`
and `password=caosdb` should work).
-### Authentication ##
+### Authentication ###
The default configuration (that your are asked for your password when ever a connection is created
can be changed by setting `password_method`:
@@ -109,7 +109,7 @@ The following illustrates the recommended options:
#password_method=keyring
```
-### SSL Certificate ##
+### SSL Certificate ###
In some cases (especially if you are testing CaosDB) you might need to supply
an SSL certificate to allow SSL encryption.
@@ -118,9 +118,9 @@ an SSL certificate to allow SSL encryption.
cacert=/path/to/caosdb.ca.pem
```
-### Further Settings ##
+### Further Settings ###
As mentioned above, a complete list of options can be found in the
-[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/master/examples/pycaosdb.ini) in
+[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/main/examples/pycaosdb.ini) in
the examples folder of the source code.
## Try it out ##
@@ -138,24 +138,20 @@ Out[2]: Connection to CaosDB with 501 Records.
Note: This setup will ask you for your password whenever a new connection is created. If you do not
like this, check out the "Authentication" section in the [configuration documentation](configuration.md).
-Now would be a good time to continue with the [tutorials](tutorials.html).
+Now would be a good time to continue with the [tutorials](tutorials/index).
## Run Unit Tests
tox
-## Code Formatting
-
-autopep8 -i -r ./
-
-## Documentation #
+## Documentation ##
Build documentation in `build/` with `make doc`.
-### Requirements ##
+### Requirements ###
- `sphinx`
- `sphinx-autoapi`
- `recommonmark`
-### Troubleshooting ##
-If the client is to be executed directly from the `/src` folder, an initial `.\setup.py install --user` must be called.
\ No newline at end of file
+### Troubleshooting ###
+If the client is to be executed directly from the `/src` folder, an initial `.\setup.py install --user` must be called.
diff --git a/RELEASE_GUIDELINES.md b/RELEASE_GUIDELINES.md
index dd6d5af4c43e1eea17c70f29da837c905694fed4..e015b598117abdcd575cf17e2f095fec459a4c4c 100644
--- a/RELEASE_GUIDELINES.md
+++ b/RELEASE_GUIDELINES.md
@@ -24,9 +24,9 @@ guidelines of the CaosDB Project
variables and set `ISRELEASED` to `True`. Use the possibility to issue
pre-release versions for testing.
-5. Merge the release branch into the master branch.
+5. Merge the release branch into the main branch.
-6. Tag the latest commit of the master branch with `v<VERSION>`.
+6. Tag the latest commit of the main branch with `v<VERSION>`.
7. Delete the release branch.
@@ -35,8 +35,8 @@ guidelines of the CaosDB Project
9. Publish the release by executing `./release.sh` with uploads the caosdb
module to the Python Package Index [pypi.org](https://pypi.org).
-10. Merge the master branch back into the dev branch.
+10. Merge the main branch back into the dev branch.
-11. After the merge of master to dev, start a new development version by
+11. After the merge of main to dev, start a new development version by
setting `ISRELEASED` to `False` and by increasing at least the `MIRCO`
version in [setup.py](./setup.py)
diff --git a/examples/set_permissions.py b/examples/set_permissions.py
index 3d9012959fb036705ef9b086af35489238360c14..8162b11bfefb41b1bcdbc74b8e314f99a61d1a4e 100755
--- a/examples/set_permissions.py
+++ b/examples/set_permissions.py
@@ -25,8 +25,8 @@
As a result, only a specific user or group may access it.
-This script assumes that the user specified in the
-pycaosdb.ini configuration can create new entities.
+This script assumes that the user specified in the pycaosdb.ini
+configuration can create new entities.
"""
@@ -206,7 +206,8 @@ def create_test_entities():
After calling this function, there will be a RecordType "Human Food" with the corresponding Records
"Bread", "Tomatoes", and "Twinkies" inserted in the database.
"""
- rt = db.RecordType(name="Human Food", description="Food that can be eaten only by humans").insert()
+ rt = db.RecordType(
+ name="Human Food", description="Food that can be eaten only by humans").insert()
food = ("Bread", "Tomatoes", "Twinkies")
cont = db.Container()
diff --git a/setup.py b/setup.py
index d491a1d340796b9b4701b307003c3e62e75d6001..e1d39458ea8d1b0b17ea12a82ebd7133b27b045a 100755
--- a/setup.py
+++ b/setup.py
@@ -47,7 +47,7 @@ from setuptools import find_packages, setup
MAJOR = 0
MINOR = 5
-MICRO = 2
+MICRO = 3
PRE = "" # e.g. rc0, alpha.1, 0.beta-23
ISRELEASED = False
diff --git a/src/caosdb/__init__.py b/src/caosdb/__init__.py
index 4043bbed3283c07d5a1ff7e3dbd77593a7f82fb1..7e06885fe495c1e8c4ccc99b7d0c0f8ff8c34b5b 100644
--- a/src/caosdb/__init__.py
+++ b/src/caosdb/__init__.py
@@ -49,6 +49,10 @@ from caosdb.common.models import (ACL, ALL, FIX, NONE, OBLIGATORY, RECOMMENDED,
from caosdb.configuration import _read_config_files, configure, get_config
from caosdb.connection.connection import configure_connection, get_connection
from caosdb.exceptions import *
-from caosdb.version import version as __version__
+try:
+ from caosdb.version import version as __version__
+except ModuleNotFoundError:
+ version = "uninstalled"
+ __version__ = version
_read_config_files()
diff --git a/src/caosdb/common/models.py b/src/caosdb/common/models.py
index e38ca3399e4e03a8ac491d8aa9f5e1fd39008e2e..16001ed8847ec853c21851e101aea54f7377d632 100644
--- a/src/caosdb/common/models.py
+++ b/src/caosdb/common/models.py
@@ -40,26 +40,23 @@ from sys import hexversion
from tempfile import NamedTemporaryFile
from warnings import warn
-from caosdb.common.datatype import BOOLEAN, DATETIME, DOUBLE, INTEGER, TEXT, is_reference, is_list_datatype
-from caosdb.common.versioning import Version
+from caosdb.common.datatype import (BOOLEAN, DATETIME, DOUBLE, INTEGER, TEXT,
+ is_list_datatype, is_reference)
from caosdb.common.state import State
from caosdb.common.utils import uuid, xml2str
+from caosdb.common.versioning import Version
from caosdb.configuration import get_config
from caosdb.connection.connection import get_connection
from caosdb.connection.encode import MultipartParam, multipart_encode
-from caosdb.exceptions import (AmbiguousEntityError,
- AuthorizationError,
- CaosDBException, CaosDBConnectionError,
- ConsistencyError,
- EmptyUniqueQueryError,
+from caosdb.exceptions import (AmbiguousEntityError, AuthorizationError,
+ CaosDBConnectionError, CaosDBException,
+ ConsistencyError, EmptyUniqueQueryError,
EntityDoesNotExistError, EntityError,
- EntityHasNoDatatypeError,
- MismatchingEntitiesError,
- QueryNotUniqueError, TransactionError,
- UniqueNamesError,
+ EntityHasNoDatatypeError, HTTPURITooLongError,
+ MismatchingEntitiesError, QueryNotUniqueError,
+ TransactionError, UniqueNamesError,
UnqualifiedParentsError,
- UnqualifiedPropertiesError,
- HTTPURITooLongError)
+ UnqualifiedPropertiesError)
from lxml import etree
_ENTITY_URI_SEGMENT = "Entity"
@@ -431,14 +428,39 @@ class Entity(object):
def add_parent(self, parent=None, **kwargs): # @ReservedAssignment
"""Add a parent to this entity.
- The first parameter is meant to identify the parent entity. So the method expects an instance of
- Entity, an integer or a string here. Even though, by means of the **kwargs parameter you may pass
- more parameters to this method. Accepted keywords are: id, name, inheritance. Any other keyword is
- ignored right now but this may change in the future.
+ Parameters
+ ----------
+ parent : Entity or int or str or None
+ The parent entity, either specified by the Entity object
+ itself, or its id or its name. Default is None.
+ **kwargs : dict, optional
+ Additional keyword arguments for specifying the parent by
+ name or id, and for specifying the mode of inheritance.
+
+ id : int
+ Integer id of the parent entity. Ignored if `parent`
+ is not None.
+ name : str
+ Name of the parent entity. Ignored if `parent is not
+ none`.
+ inheritance : str
+ One of ``obligatory``, ``recommended``, ``suggested``, or ``fix``. Specifies the
+ minimum importance which parent properties need to have to be inherited by this
+ entity. If no `inheritance` is given, no properties will be inherited by the child.
+ This parameter is case-insensitive.
+
+ Note that the behaviour is currently not yet specified when assigning parents to
+ Records, it only works for inheritance of RecordTypes (and Properties).
+
+ For more information, it is recommended to look into the
+ :ref:`data insertion tutorial<tutorial-inheritance-properties>`.
+
+ Raises
+ ------
+ UserWarning
+ If neither a `parent` parameter, nor the `id`, nor `name`
+ parameter is passed to this method.
- @param parent: An entity, an id or a name.
- @param **kwargs: Accepted keywords: id, name, inheritance.
- @raise UserWarning: If neither a 'parent' parameter, nor the 'id', nor 'name' parameter is passed to this method.
"""
name = (kwargs['name'] if 'name' in kwargs else None)
pid = (kwargs['id'] if 'id' in kwargs else None)
@@ -555,6 +577,7 @@ class Entity(object):
entity, or name.
"""
+
if isinstance(key, int):
for p in self.parents:
if p.id is not None and int(p.id) == int(key):
@@ -563,14 +586,17 @@ class Entity(object):
if key.id is not None:
# first try by id
found = self.get_parent(int(key.id))
+
if found is not None:
return found
# otherwise by name
+
return self.get_parent(key.name)
else:
for p in self.parents:
if (p.name is not None
and str(p.name).lower() == str(key).lower()):
+
return p
return None
@@ -655,6 +681,7 @@ class Entity(object):
special_selector = None
# iterating through the entity tree according to the selector
+
for subselector in selector:
# selector does not match the structure, we cannot get a
# property of non-entity
@@ -666,11 +693,13 @@ class Entity(object):
# selector does not match the structure, we did not get a
# property
+
if prop is None:
return None
# if the property is a reference, we are interested in the
# corresponding entities attributes
+
if isinstance(prop.value, Entity):
ref = prop.value
@@ -679,6 +708,7 @@ class Entity(object):
ref = prop
# if we saved a special selector before, apply it
+
if special_selector is None:
return prop.value
else:
@@ -810,6 +840,7 @@ class Entity(object):
assert isinstance(xml, etree._Element)
# unwrap wrapped entity
+
if self._wrapped_entity is not None:
xml = self._wrapped_entity.to_xml(xml, add_properties)
@@ -856,6 +887,8 @@ class Entity(object):
xml.append(v_elem)
elif self.value == "":
xml.append(etree.Element("EmptyString"))
+ elif str(self.value) == "nan":
+ xml.text = "NaN"
else:
xml.text = str(self.value)
@@ -1047,6 +1080,7 @@ class Entity(object):
if len(c) == 1:
c[0].messages.extend(c.messages)
+
return c[0]
raise QueryNotUniqueError("This retrieval was not unique!!!")
@@ -1433,6 +1467,23 @@ class Property(Entity):
property=property, value=value, **copy_kwargs)
def add_parent(self, parent=None, **kwargs):
+ """Add a parent Entity to this Property.
+
+ Parameters
+ ----------
+ parent : Entity or int or str or None, optional
+ The parent entity
+ **kwargs : dict, optional
+ Additional keyword arguments specifying the parent Entity
+ by id or name, and specifying the inheritance level. See
+ :py:meth:`Entity.add_parent` for more information. Note
+ that by default, `inheritance` is set to ``fix``.
+
+ See Also
+ --------
+ Entity.add_parent
+
+ """
copy_kwargs = kwargs.copy()
if 'inheritance' not in copy_kwargs:
@@ -1511,6 +1562,24 @@ class RecordType(Entity):
**copy_kwargs)
def add_parent(self, parent=None, **kwargs):
+ """Add a parent to this RecordType
+
+ Parameters
+ ----------
+ parent : Entity or int or str or None, optional
+ The parent entity, either specified by the Entity object
+ itself, or its id or its name. Default is None.
+ **kwargs : dict, optional
+ Additional keyword arguments specifying the parent Entity by id or
+ name, and specifying the inheritance level. See
+ :py:meth:`Entity.add_parent` for more information. Note
+ that by default, `inheritance` is set to ``obligatory``.
+
+ See Also
+ --------
+ Entity.add_parent
+
+ """
copy_kwargs = kwargs.copy()
if 'inheritance' not in copy_kwargs:
@@ -2083,6 +2152,7 @@ class _Messages(dict):
else:
raise TypeError(
"('description', 'body'), ('body'), or 'body' expected.")
+
if isinstance(value, Message):
body = value.body
description = value.description
@@ -2252,6 +2322,7 @@ def _deletion_sync(e_local, e_remote):
except KeyError:
# deletion info wasn't there
e_local.messages = e_remote.messages
+
return
_basic_sync(e_local, e_remote)
@@ -2444,6 +2515,7 @@ class Container(list):
tmpid = 0
# users might already have specified some tmpids. -> look for smallest.
+
for e in self:
tmpid = min(tmpid, Container._get_smallest_tmpid(e))
tmpid -= 1
@@ -2663,6 +2735,7 @@ class Container(list):
used_remote_entities = []
# match by cuid
+
for local_entity in self:
sync_dict[local_entity] = None
@@ -2691,6 +2764,7 @@ class Container(list):
raise MismatchingEntitiesError(msg)
# match by id
+
for local_entity in self:
if sync_dict[local_entity] is None and local_entity.id is not None:
sync_remote_entities = []
@@ -2715,6 +2789,7 @@ class Container(list):
raise MismatchingEntitiesError(msg)
# match by path
+
for local_entity in self:
if (sync_dict[local_entity] is None
and local_entity.path is not None):
@@ -2744,6 +2819,7 @@ class Container(list):
raise MismatchingEntitiesError(msg)
# match by name
+
for local_entity in self:
if (sync_dict[local_entity] is None
and local_entity.name is not None):
@@ -2811,11 +2887,14 @@ class Container(list):
for container_item in container:
item_id.add(container_item.id)
+
for parents in container_item.get_parents():
is_parent.add(parents.id)
+
for references in container_item.get_properties():
if is_reference(references.datatype):
# add only if it is a reference, not a property
+
if isinstance(references.value, int):
has_references.add(references.value)
elif is_list_datatype(references.datatype):
@@ -2850,24 +2929,28 @@ class Container(list):
chunk_size = 100
item_count = len(self)
# Split Container in 'chunk_size'-sized containers (if necessary) to avoid error 414 Request-URI Too Long
+
if item_count > chunk_size:
dependencies = self._test_dependencies_in_container(self)
'''
- If there are as many dependencies as entities in the container and it is larger than chunk_size it cannot be split and deleted.
+ If there are as many dependencies as entities in the container and it is larger than chunk_size it cannot be split and deleted.
This case cannot be handled at the moment.
'''
+
if len(dependencies) == item_count:
if raise_exception_on_error:
te = TransactionError(
msg="The container is too large and with too many dependencies within to be deleted.",
container=self)
raise te
+
return self
dependencies_delete = Container() # items which have to be deleted later because of dependencies.
for i in range(0, int(item_count/chunk_size)+1):
chunk = Container()
+
for j in range(i*chunk_size, min(item_count, (i+1)*chunk_size)):
if len(dependencies):
if self[j].id in dependencies:
@@ -2880,6 +2963,7 @@ class Container(list):
if len(chunk):
chunk.delete()
dependencies_delete.delete()
+
return self
if len(self) == 0:
@@ -3517,6 +3601,7 @@ class ACL():
result._priority_grants.update(self._priority_grants)
result._priority_denials.update(other._priority_denials)
result._priority_denials.update(self._priority_denials)
+
return result
def __eq__(self, other):
@@ -3807,6 +3892,7 @@ class Query():
connection = get_connection()
flags = self.flags
+
if cache is False:
flags["cache"] = "false"
query_dict = dict(flags)
@@ -3833,9 +3919,11 @@ class Query():
if len(cresp) > 1 and raise_exception_on_error:
raise QueryNotUniqueError(
"Query '{}' wasn't unique.".format(self.q))
+
if len(cresp) == 0 and raise_exception_on_error:
raise EmptyUniqueQueryError(
"Query '{}' found no results.".format(self.q))
+
if len(cresp) == 1:
r = cresp[0]
r.messages.extend(cresp.messages)
@@ -3942,6 +4030,7 @@ class Info():
for e in xml:
m = _parse_single_xml_element(e)
+
if isinstance(m, UserInfo):
self.user_info = m
else:
@@ -4101,13 +4190,16 @@ def _evaluate_and_add_error(parent_error, ent):
Parent error with new exception(s) attached to it.
"""
+
if isinstance(ent, (Entity, QueryTemplate)):
# Check all error messages
found114 = False
found116 = False
+
for err in ent.get_errors():
# Evaluate specific EntityErrors depending on the error
# code
+
if err.code is not None:
if int(err.code) == 101: # ent doesn't exist
new_exc = EntityDoesNotExistError(entity=ent,
@@ -4124,6 +4216,7 @@ def _evaluate_and_add_error(parent_error, ent):
found114 = True
new_exc = UnqualifiedPropertiesError(entity=ent,
error=err)
+
for prop in ent.get_properties():
new_exc = _evaluate_and_add_error(new_exc,
prop)
@@ -4131,6 +4224,7 @@ def _evaluate_and_add_error(parent_error, ent):
found116 = True
new_exc = UnqualifiedParentsError(entity=ent,
error=err)
+
for par in ent.get_parents():
new_exc = _evaluate_and_add_error(new_exc,
par)
@@ -4141,21 +4235,28 @@ def _evaluate_and_add_error(parent_error, ent):
parent_error.add_error(new_exc)
# Check for possible errors in parents and properties that
# weren't detected up to here
+
if not found114:
dummy_err = EntityError(entity=ent)
+
for prop in ent.get_properties():
dummy_err = _evaluate_and_add_error(dummy_err, prop)
+
if dummy_err.errors:
parent_error.add_error(dummy_err)
+
if not found116:
dummy_err = EntityError(entity=ent)
+
for par in ent.get_parents():
dummy_err = _evaluate_and_add_error(dummy_err, par)
+
if dummy_err.errors:
parent_error.add_error(dummy_err)
elif isinstance(ent, Container):
parent_error.container = ent
+
if ent.get_errors() is not None:
parent_error.code = ent.get_errors()[0].code
# In the highly unusual case of more than one error
@@ -4163,6 +4264,7 @@ def _evaluate_and_add_error(parent_error, ent):
parent_error.msg = '\n'.join(
[x.description for x in ent.get_errors()])
# Go through all container elements and add them:
+
for elt in ent:
parent_error = _evaluate_and_add_error(parent_error, elt)
@@ -4188,10 +4290,12 @@ def raise_errors(arg0):
transaction_error = _evaluate_and_add_error(TransactionError(),
arg0)
# Raise if any error was found
+
if len(transaction_error.all_errors) > 0:
raise transaction_error
# Cover the special case of an empty container with error
# message(s) (e.g. query syntax error)
+
if (transaction_error.container is not None and
transaction_error.container.has_errors()):
raise transaction_error
diff --git a/src/caosdb/connection/connection.py b/src/caosdb/connection/connection.py
index 2cb31f425b14bf382fae32f62af79a4bd6ef0301..9e273a56778737033fda9f342f967f56946b501b 100644
--- a/src/caosdb/connection/connection.py
+++ b/src/caosdb/connection/connection.py
@@ -41,7 +41,11 @@ from caosdb.exceptions import (CaosDBException, HTTPClientError,
HTTPResourceNotFoundError,
HTTPServerError,
HTTPURITooLongError)
-from caosdb.version import version
+try:
+ from caosdb.version import version
+except ModuleNotFoundError:
+ version = "uninstalled"
+
from pkg_resources import resource_filename
from .interface import CaosDBHTTPResponse, CaosDBServerConnection
diff --git a/src/caosdb/exceptions.py b/src/caosdb/exceptions.py
index f02a4630356726f99d8439fd821b6dd327ab22c7..fdd2e11f1dfb8857f86942df2534d732bad9a793 100644
--- a/src/caosdb/exceptions.py
+++ b/src/caosdb/exceptions.py
@@ -189,8 +189,8 @@ class TransactionError(CaosDBException):
error_t. If direct_children_only is True, only direct children
are checked.
- Parameters:
- -----------
+ Parameters
+ ----------
error_t : EntityError
error type to be checked
direct_children_only: bool, optional
@@ -199,8 +199,8 @@ class TransactionError(CaosDBException):
children, i.e., all errors in self.all_errors are
used. Default is false.
- Returns:
- --------
+ Returns
+ -------
has_error : bool
True if at least one of the children is of type error_t,
False otherwise.
diff --git a/src/doc/Makefile b/src/doc/Makefile
index 5458c5300efc82e55686bc1cd6934182c5c8e39a..64219c5957ee963e84f9305685f2ec4e8ed3d761 100644
--- a/src/doc/Makefile
+++ b/src/doc/Makefile
@@ -32,6 +32,7 @@ PY_BASEDIR = ../caosdb
SOURCEDIR = .
BUILDDIR = ../../build/doc
+
.PHONY: doc-help Makefile
# Put it first so that "make" without argument is like "make help".
@@ -44,4 +45,4 @@ doc-help:
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
apidoc:
- @$(SPHINXAPIDOC) -o _apidoc $(PY_BASEDIR)
+ @$(SPHINXAPIDOC) -o _apidoc --separate $(PY_BASEDIR)
diff --git a/src/doc/conf.py b/src/doc/conf.py
index f276f325273b71d4b697bc57990259e842b2dbc3..b05fa1c71c1dcd0b59916594818449d2ebc574bd 100644
--- a/src/doc/conf.py
+++ b/src/doc/conf.py
@@ -8,16 +8,18 @@
# -- Path setup --------------------------------------------------------------
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
+# If extensions (or modules to document with autodoc) are in another directory, add these
+# directories to sys.path here. This is particularly necessary if this package is installed at a
+# different version, for example via `pip install`.
#
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('../caosdb'))
-
+# If the directory is relative to the documentation root, use os.path.abspath to make it absolute,
+# like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('..'))
-import sphinx_rtd_theme
+import sphinx_rtd_theme # noqa: E402
# -- Project information -----------------------------------------------------
@@ -27,9 +29,10 @@ copyright = '2020, IndiScale GmbH'
author = 'Daniel Hornung'
# The short X.Y version
-version = '0.4.0'
+version = '0.5.2'
# The full version, including alpha/beta/rc tags
-release = '0.4.0-rc'
+# release = '0.5.2-rc2'
+release = '0.5.2'
# -- General configuration ---------------------------------------------------
@@ -43,6 +46,7 @@ release = '0.4.0-rc'
# ones.
extensions = [
'sphinx.ext.autodoc',
+ 'sphinx.ext.autosectionlabel',
'sphinx.ext.intersphinx',
'sphinx.ext.napoleon', # For Google style docstrings
"recommonmark", # For markdown files.
@@ -54,7 +58,6 @@ templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
-#
source_suffix = ['.rst', '.md']
# The master toctree document.
@@ -81,6 +84,7 @@ pygments_style = None
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
+
html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme
@@ -182,14 +186,22 @@ epub_exclude_files = ['search.html']
# -- Extension configuration -------------------------------------------------
+# True to prefix each section label with the name of the document it is in, followed by a colon. For
+# example, index:Introduction for a section called Introduction that appears in document
+# index.rst. Useful for avoiding ambiguity when the same section heading appears in different
+# documents.
+#
+# Note: This stops "normal" links from working, so it should be kept at False.
+# autosectionlabel_prefix_document = True
+
# -- Options for intersphinx -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#confval-intersphinx_mapping
intersphinx_mapping = {
"python": ("https://docs.python.org/", None),
- "caosdb-mysqlbackend": ("https://caosdb.gitlab.io/caosdb-mysqlbackend/",
+ "caosdb-mysqlbackend": ("https://docs.indiscale.com/caosdb-mysqlbackend/",
None),
- "caosdb-server": ("https://caosdb.gitlab.io/caosdb-server/", None),
+ "caosdb-server": ("https://docs.indiscale.com/caosdb-server/", None),
}
diff --git a/src/doc/configuration.md b/src/doc/configuration.md
index b2de2781d5adff4d59cb3648cd912e142327f676..6e53542f661dcae94622fef24a67cecf7491df9c 100644
--- a/src/doc/configuration.md
+++ b/src/doc/configuration.md
@@ -50,5 +50,5 @@ debugging (which I hope will not be necessary for this tutorial) or if you want
the internals of the protocol.
A complete list of options can be found in the
-[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/master/examples/pycaosdb.ini) in
+[pycaosdb.ini file](https://gitlab.com/caosdb/caosdb-pylib/-/blob/main/examples/pycaosdb.ini) in
the examples folder of the source code.
diff --git a/src/doc/index.rst b/src/doc/index.rst
index 76a2f88f6d31dd9b5f17995b6d54ccc63eb33631..bd29c6c56acf5c173e94ae6471a6aeba56ea4b93 100644
--- a/src/doc/index.rst
+++ b/src/doc/index.rst
@@ -12,12 +12,12 @@ Welcome to PyCaosDB's documentation!
Concepts <concepts>
Configuration <configuration>
Administration <administration>
- API documentation<_apidoc/modules>
+ API documentation<_apidoc/caosdb>
This is the documentation for the Python client library for CaosDB, ``PyCaosDB``.
-This documentation helps you to :doc:`get started<getting_started>`, explains the most important
-:doc:`concepts<concepts>` and offers a range of :doc:`tutorials<tutorials>`.
+This documentation helps you to :doc:`get started<README_SETUP>`, explains the most important
+:doc:`concepts<concepts>` and offers a range of :doc:`tutorials<tutorials/index>`.
Indices and tables
diff --git a/src/doc/tutorials/Data-Insertion.rst b/src/doc/tutorials/Data-Insertion.rst
index 2ead1cc259c807ee12f2459f7e452558eb1b63a2..22fb9461d6916003b2dad496ff3487df335c8dcc 100644
--- a/src/doc/tutorials/Data-Insertion.rst
+++ b/src/doc/tutorials/Data-Insertion.rst
@@ -4,9 +4,9 @@ Data Insertion
Data Models
~~~~~~~~~~~
-Data is stored and structured in CaosDB using a concept of RecordTypes,
-Properties, Records etc. If you do not know what these are, please look
-at the chapter :any:`caosdb-server:Data Model` .
+Data is stored and structured in CaosDB using a concept of RecordTypes, Properties, Records etc. If
+you do not know what these are, please look at the chapter :doc:`Data
+Model<caosdb-server:Data-Model>` in the CaosDB server documentation.
In order to insert some actual data, we need to create a data model
using RecordTypes and Properties (You may skip this if you use a CaosDB
@@ -35,6 +35,42 @@ two more Properties and a RecordType:
container.extend([a, b, epsilon, recordtype])
container.insert()
+.. _tutorial-inheritance-properties:
+
+Inheritance of Properties
+-------------------------
+
+Suppose you want to create a new RecordType “2D_BarkleySimulation”
+that denotes spatially extended Barkley simulations. This is a subtype
+of the “BarkleySimulation” RecordType above and should have all its
+parameters, i.e., properties. It may be assigned more, e.g., spatial
+resolution, but we'll omit this for the sake of brevity for now.
+
+.. code:: python
+
+ rt = db.RecordType(name="2D_BarkleySimulation",
+ description="Spatially extended Barkley simulation")
+ # inherit all properties from the BarkleySimulation RecordType
+ rt.add_parent(name="BarkleySimulation", inheritance="all")
+ rt.insert()
+
+ print(rt.get_property(name="epsilon").importance) ### rt has a "epsilon" property with the same importance as "BarkleySimulation"
+
+The parameter ``inheritance=(obligatory|recommended|fix|all|none)`` of
+:py:meth:`Entity.add_parent()<caosdb.common.models.Entity.add_parent>` tells the server to assign
+all properties of the parent RecordType with the chosen importance (and properties with a higher
+importance) to the child RecordType
+automatically upon insertion. See the chapter on `importance
+<https://docs.indiscale.com/caosdb-server/specification/RecordType.html#importance>`_ in the
+documentation of the CaosDB server for more information on the importance and inheritance of
+properties.
+
+.. note::
+
+ The inherited properties will only be visible after the insertion since they are set by the
+ CaosDB server, not by the Python client.
+
+
Insert Actual Data
~~~~~~~~~~~~~~~~~~
@@ -108,32 +144,6 @@ Useful is also, that you can insert directly tabular data.
With this example file
`test.csv <uploads/4f2c8756a26a3984c0af09d206d583e5/test.csv>`__.
-Inheritance of Properties
--------------------------
-
-Given, you want to insert a new RecordType “Fridge temperatur
-experiment” as a child of the existing RecordType “Experiment”. The
-latter may have an obligatory Property “date” (since every experiment is
-conducted at some time). It is a natural way of thinking, that every sub
-type of “Experiment” also has this obligatory Property—in terms of
-object oriented programing the “Fridge temperatur experiment” *inherits*
-that Property.
-
-::
-
- rt = h.RecordType(name="Fridge temperatur experiment",
- description="RecordType which inherits all obligatory properties from Experiment"
- ).add_parent(name="Experiment", inheritance="obligatory").insert()
-
- print(rt.get_property(name="date").importance) ### rt now has a "date"-property -> this line prints "obligatory"
-
-The parameter *``inheritance=(obligatory|recommended|fix|all|none)``* of
-``add_parent`` tells the server to assign obligatory:: properties of the
-parent to the child automatically, recommended:: properties of the
-parent to the child automatically, fix:: properties of the parent to the
-child automatically, all:: properties of the parent to the child
-automatically, none:: of the properties of the parent to child
-automatically,
File Update
-----------
diff --git a/src/doc/tutorials/errors.rst b/src/doc/tutorials/errors.rst
index ba386dc31baad1021f01d53b2b12a2623d8278ad..37c53c9b527a0435f9f24ae6c6e71687e73eb963 100644
--- a/src/doc/tutorials/errors.rst
+++ b/src/doc/tutorials/errors.rst
@@ -1,53 +1,176 @@
Error Handling
---------------
+==============
-HeartDBException
-~~~~~~~~~~~~~~~~
+In case of erroneous transactions, connection problems and a lot of
+other cases, PyCaosDB may raise specific errors in order to pinpoint
+the problem as precisely as possible. Some of these errors a
+representations of errors in the CaosDB server, others stem from
+problems that occurred on the client side.
+
+The errors and exceptions are ordered hierarchically form the most
+general exceptions to specific transaction or connection problems. The
+most important error types and the hierarchy will be explained in the
+following. For more information on specific error types, see also the
+:doc:`source code<../_apidoc/caosdb.exceptions>`.
+
+.. note::
+
+ Starting from PyCaosDB 0.5, the error handling has changed
+ significantly. New error classes have been introduced and the
+ behavior of ``TransactionError`` and ``EntityError`` has been
+ re-worked. In the following, only the "new" errors are
+ discussed. Please refer to the documentation of PyCaosDB 0.4.1 and
+ earlier for the old error handling.
+
+CaosDBException
+----------------
+
+``CaosDBException`` is the most generic exception and all other error classes inherit
+from this one. Because of its generality, it doesn't tell you much
+except that some component of PyCaosDB raised an exception. If you
+want to catch all possible CaosDB errors, this is the class to use.
TransactionError
-~~~~~~~~~~~~~~~~
+----------------
Every transaction (calling ``insert``, ``update``, ``retrieve``, or
``delete`` on a container or an entity) may finish with errors. They
indicate, for instance, that an entity does not exist or that you need
to specify a data type for your property and much more. If and only if
one or more errors occur during a transaction a ``TransactionError``
-will be raised by the transaction method. The ``TransactionError`` class
-is a container for all errors which occur during a transaction. It can
-help you to find the crucial problems with your transaction by two
-important methods: \* ``get_errors()`` which returns a list of instances
-of ``EntityError``. \* ``get_entities()`` which returns a list of
-entities in the transaction container which are erroneous.
-
-Additionally, ``print(transaction_error`` prints a tree-like
+will be raised by the transaction method. The ``TransactionError``
+class is a container for all errors which occur during a
+transaction. It usually contains one or more :ref:`entity
+errors<EntityError>` which you can inspect in order to learn why the
+transaction failed. For this inspection, there are some helpful
+attributes and methods provided by the ``TransactionError``:
+
+* ``entities``: a list of all entities that directly caused at least one error
+ in this transaction.
+
+* ``errors``: a list of all ``EntityError`` objects that directly caused the
+ transaction to fail.
+
+* ``all_entities``, ``all_errors``: sets of all entities and errors
+ that, directly or indirectly, caused either this ``TransactionError`` or any of the
+ ``EntityError`` objects it contains.
+
+* ``has_error(error_t)``: Check whether an error of type ``error_t``
+ occurred during the transaction.
+
+Additionally, ``print(transaction_error)`` prints a tree-like
representation of all errors regarding the transaction in question.
EntityError
-~~~~~~~~~~~
+-----------
-An ``EntityError`` represents a single error that has been returned by
-the server. You might call \* ``get_entity()`` which returns the entity
-which caused the error. \* ``get_description()`` which returns a
-description of the error. \* ``get_code()`` which returns the error code
-(if specified) or 0 (if not).
+An ``EntityError`` specifies the entity and the error proper that
+caused a transaction to fail. It is never raised on its own but is
+contained in a ``TransactionError`` (which may or may not contain
+other ``EntityError`` objects) which is then raised. ``EntityError``
+has several :ref:`subclasses<Special Errors>` that further specify the
+error that occurred.
-In fact, the ``EntityError`` class is a subclass of
-``TransactionError``. So, it inherits the ``get_entities()``. Unless
-overridden by subclasses of ``EntityError``, it return a list with only
-one item—the entity which caused this error. Similarly, unless
-overridden by subclasses, the ``get_errors()`` method returns a list
-with only one item—``[self]``.
+The ``EntityError`` class is in fact a subclass of
+``TransactionError``. Thus, it has the same methods and attributes as
+the ``TransactionError`` explained
+:ref:`above<TransactionError>`. This is important in case of an
+``EntityError`` that was caused by other faulty entities (e.g., broken
+parents or properties). In that case these problematic entities and
+errors can again be inspected by visiting the ``entities`` and
+``errors`` lists as above.
Special Errors
~~~~~~~~~~~~~~
-Subclasses of ``EntityError`` for special purposes: \*
-``EntityDoesNotExistError`` \* ``EntityHasNoDataTypeError`` \*
-``UniqueNamesError`` \* ``UnqualifiedParentsError`` - overrides
-``get_entities()``: returns all parent entities with errors. - overrides
-``get_errors()``: returns a list of EntityErrors which have been caused
-by parent entities. \* ``UnqualifiedPropertiesError`` - overrides
-``get_entities()``: returns all properties with errors. - overrides
-``get_errors()``: returns a list of EntityErrors which have been caused
-by properties.
+Subclasses of ``EntityError`` for special purposes:
+
+* ``EntityDoesNotExistError``
+
+* ``EntityHasNoDataTypeError``
+
+* ``UniqueNamesError``
+
+* ``UnqualifiedParentsError``
+
+* ``UnqualifiedPropertiesError``
+
+* ``ConsistencyError``
+
+* ``AuthorizationError``
+
+* ``AmbiguousEntityError``
+
+BadQueryError
+-------------
+
+A ``BadQueryError`` is raised when a query could not be processed by
+the server. In contrast to a ``TransactionError`` it is not
+necessarily caused by problematic entities or
+containers. ``BadQueryError`` has the two important subclasses
+``EmptyUniqueQueryError`` and ``QueryNotUniqueError`` for queries with
+``unique=True`` which found no or ambiguous entities, respectively.
+
+HTTP Errors
+-----------
+
+An ``HTTPClientError`` or an ``HTTPServerError`` is raised in case of
+http(s) connection problems caused by the Python client or the CaosDB
+server, respectively. There are the following subclasses of
+``HTTPClientError`` that are used to specify the connection problem:
+
+* ``HTTPURITooLongError``: The URI of the request was too long to be
+ processed by the server.
+
+* ``HTTPForbiddenError``: You're not allowed to access this resource.
+
+* ``HTTPResourceNotFoundError``: The requested resource doesn't exist.
+
+Other Errors
+------------
+
+There are further subclasses of ``CaosDBException`` that are raised in
+case of faulty configurations or other problems. They should be rather
+self-explanatory from their names; see the :doc:`source code<../_apidoc/caosdb.exceptions>`
+for further information.
+
+* ``ConfigurationError``
+
+* ``LoginFailedError``
+
+* ``MismatchingEntitiesError``
+
+* ``ServerConfigurationException``
+
+Examples
+--------
+
+.. code-block:: python3
+
+ import caosdb as db
+
+ def link_and_insert(entity, linked, link=True):
+ """Link the ENTITY to LINKED and insert it."""
+ if link:
+ entity.add_property(db.Property(name="link", value=linked))
+ try:
+ entity.insert()
+ except db.TransactionError as tre:
+ # Unique names problem may be worked around by using another name
+ if tre.has_error(db.UniqueNamesError):
+ for ent_error in tre.errors:
+ if (isinstance(ent_error, db.UniqueNamesError)
+ and entity in ent_error.entities):
+ entity.name = entity.name + "_new" # Try again with new name.
+ link_and_insert(entity, linked, link=False)
+ break
+ # Unqualified properties will be handled by the caller
+ elif tre.has_error(db.UnqualifiedPropertiesError):
+ for ent_error in tre.errors:
+ if (isinstance(ent_error, db.UnqualifiedPropertiesError_
+ and entity in ent_error.entities):
+ raise RuntimeError("One of the properties was unqualified: " + str(ent_error))
+ # Other problems are not covered by this tutorial
+ else:
+ raise NotImplementedError("Unhandled TransactionError: " + str(tre))
diff --git a/src/doc/tutorials/first_steps.rst b/src/doc/tutorials/first_steps.rst
index 3eb804776960ba93acb9baf78824f0cf8a06e1f7..34b96bbeca416107fb34feb4707b9ef46fc49fe7 100644
--- a/src/doc/tutorials/first_steps.rst
+++ b/src/doc/tutorials/first_steps.rst
@@ -2,14 +2,15 @@ First Steps
===========
You should have a working connection to a CaosDB instance now. If not, please check out the
-:doc:`Getting Started secton</getting_started>`.
+:doc:`Getting Started secton</README_SETUP>`.
If you are not yet familiar with Records, RecordTypes and Properties used in CaosDB,
-please check out the respective part in the `Web Interface Tutorial`_.
-You should also know the basics of the CaosDB Query Language (a tutorial is here_).
+please check out the respective part in the `Web Interface tutorial`_.
+You should also know the basics of the CaosDB Query Language (a tutorial is
+`here <https://docs.indiscale.com/caosdb-webui/tutorials/query.html>`_).
-We recommend that you connect to the demo instance in order to try out the following
-examples. You can do this with
+We recommend that you connect to the `demo instance`_ (hosted by `Indiscale`_) in order to try out
+the following examples. You can do this with
>>> import caosdb as db
>>> _ = db.configure_connection(
@@ -19,7 +20,7 @@ examples. You can do this with
... password="caosdb")
or by using corresponding settings in the configuration file
-(see :doc:`Getting Started secton</getting_started>`.).
+(see :doc:`Getting Started secton</README_SETUP>`.).
However, you can also translate the examples to the data model that you have at hand.
Let's start with a simple query.
@@ -124,9 +125,6 @@ the result Records and their properties.
The next tutorial shows how to make some meaningful use of this.
-
-.. _here: https://gitlabio.something
.. _`demo instance`: https://demo.indiscale.com
.. _`IndiScale`: https://indiscale.com
-.. _`Web Interface Tutorial`: https://caosdb.gitlab.io/caosdb-webui/tutorials/model.html
-.. _here: https://caosdb.gitlab.io/caosdb-webui/tutorials/cql.html
+.. _`Web Interface tutorial`: https://docs.indiscale.com/caosdb-webui/tutorials/first_steps.html