Merge branch 'dev' into f-fsm

CHANGELOG.md

@@ -11,6 +11,81 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * Entity State support (experimental, no StateModel support yet). See the `caosdb.State` class for
   more information.
+
+### Changed ###
+
+### Deprecated ###
+
+### Removed ###
+
+### Fixed ###
+
+### Security ###
+
+## [0.5.1] - 2021-02-12 ##
+
+### Added ###
+
+### Changed ###
+
+### Deprecated ###
+
+### Removed ###
+
+### Fixed ###
+
+* #43 - Error with `execute_query` when server doesn't support query caching.
+
+### Security ###
+
+## [0.5.0] - 2021-02-11 ##
+
+### Added ###
+
+* New exceptions `HTTPForbiddenException` and
+  `HTTPResourceNotFoundException` for HTTP 403 and 404 errors,
+  respectively.
+* `BadQueryError`, `EmptyUniqueQueryError`, and `QueryNotUniqueError`
+  for bad (unique) queries.
+* Added `cache` paramter to `execute_query` and `Query.execute` which indicates
+  whether server may use the cache for the query execution.
+* Added `cached` property to the `Query` class which indicates whether the
+  server used the cache for the execution of the last query.
+* Documentation moved from wiki to this repository and enhanced.
+
+### Changed ###
+
+* Renaming of `URITooLongException` to `HTTPURITooLongError`.
+* Raising of entity exceptions and transaction errors. Whenever any
+  transaction fails, a `TransactionError` is raised. If one ore more
+  entities caused that failure, corresponding entity errors are
+  attached as direct and indirect children of the
+  `TransactionError`. They can be accessed via the `get_errors`
+  (direct children) and `get_all_errors` (direct and indirect
+  children) methods; the causing entities are accessed by
+  `get_entities` and `get_all_entities`. The `has_error` method can be
+  used to check whether a `TransactionError` was caused by a specific
+  `EntityError`.
+* Unique queries will now result in `EmptyUniqueQueryError` or
+  `QueryNotUniqueError` if no or more than one possible candidate is
+  found, respectively.
+
+### Deprecated ###
+
+### Removed ###
+
+* Dynamic exception type `EntityMultiError`. 
+* `get_something` functions from all error object in `exceptions.py`
+* `AmbiguityException`
+
+### Fixed ###
+
+### Security ###
+
+## [0.4.1] - 2021-02-10 ##
+
+### Added ###
+
 * Versioning support (experimental). The version db.Version class can
   represents particular entity versions and also the complete history of an
   entity.
@@ -81,7 +156,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 * import bugs in apiutils
 
-## [0.2.4] -- 2020-04-23
+## [0.2.4] - 2020-04-23
 
 ### Added
 

DEPENDENCIES.md

+* caosdb-server == 0.3
+* Python >= 3.5
+* pip >= 20.0.2
+
+
+Any other dependencies are being installed via pip

README.md

@@ -16,9 +16,9 @@ project](https://gitlab.com/caosdb/caosdb) for more information.
 
 # License
 
-Copyright (C) 2018 Research Group Biomedical Physics, Max Planck Institute for
-Dynamics and Self-Organization Göttingen.
-Copyright (C) 2020 Indiscale GmbH <info@indiscale.com>
+* Copyright (C) 2018 Research Group Biomedical Physics, Max Planck Institute
+  for Dynamics and Self-Organization Göttingen.
+* Copyright (C) 2020-2021 Indiscale GmbH <info@indiscale.com>
 
 All files in this repository are licensed under a [GNU Affero General Public
 License](LICENCE.md) (version 3 or later).

README_SETUP.md

@@ -30,6 +30,23 @@ packages you will ever need out of the box.  If you prefer, you may also install
 After installation, open an Anaconda prompt from the Windows menu and continue in the [Generic
 installation](#generic-installation) section.
 
+#### iOS ####
+
+If there is no Python 3 installed yet, there are two main ways to obtain it: Either get the binary
+package from [python.org](https://www.python.org/downloads/) or, for advanced users, install via [Homebrew](https://brew.sh/).  After installation
+from python.org, it is recommended to also update the TLS certificates for Python (this requires
+administrator rights for your user):
+
+```sh
+# Replace this with your Python version number:
+cd /Applications/Python\ 3.9/
+
+# This needs administrator rights:
+sudo ./Install\ Certificates.command
+```
+
+After these steps, you may continue with the [Generic installation](#generic-installation).
+
 #### Generic installation ####
 
 To install PyCaosDB locally, use `pip3` (also called `pip` on some systems):

RELEASE_GUIDELINES.md

@@ -16,21 +16,27 @@ guidelines of the CaosDB Project
 1. Create a release branch from the dev branch. This prevents further changes
    to the code base and a never ending release process. Naming: `release-<VERSION>`
 
-2. Check all general prerequisites.
+2. Update CHANGELOG.md
 
-3. Prepare [setup.py](./setup.py): Update the `MAJOR`, `MINOR`, `MICRO`, `PRE`
+3. Check all general prerequisites.
+
+4. Prepare [setup.py](./setup.py): Check the `MAJOR`, `MINOR`, `MICRO`, `PRE`
    variables and set `ISRELEASED` to `True`. Use the possibility to issue
    pre-release versions for testing.
 
-4. Merge the release branch into the master branch.
+5. Merge the release branch into the master branch.
 
-5. Tag the latest commit of the master branch with `v<VERSION>`.
+6. Tag the latest commit of the master branch with `v<VERSION>`.
 
-6. Delete the release branch.
+7. Delete the release branch.
 
-7. Remove possibly existing `./dist` directory with old release.
+8. Remove possibly existing `./dist` directory with old release.
 
-8. Publish the release by executing `./release.sh` with uploads the caosdb
+9. Publish the release by executing `./release.sh` with uploads the caosdb
    module to the Python Package Index [pypi.org](https://pypi.org).
 
-9. Merge the master branch back into the dev branch.
+10. Merge the master branch back into the dev branch.
+
+11. After the merge of master to dev, start a new development version by
+    setting `ISRELEASED` to `False` and by increasing at least the `MIRCO`
+    version in [setup.py](./setup.py)

examples/set_permissions.py

@@ -51,26 +51,26 @@ out : tuple
     try:
         human_user = admin._retrieve_user("jane")
         _activate_user("jane")
-    except db.EntityDoesNotExistError:
+    except db.ResourceNotFoundError:
         human_user = admin._insert_user(
             "jane", password="Human_Rememberable_Password_1234", status="ACTIVE")
 
     try:
         alien_user = admin._retrieve_user("xaxys")
         _activate_user("xaxys")
-    except db.EntityDoesNotExistError:
+    except db.ResourceNotFoundError:
         alien_user = admin._insert_user("xaxys", password="4321_Syxax",
                                         status="ACTIVE")
 
     # At the moment, the return value is only "ok" for successful insertions.
     try:
         human_role = admin._retrieve_role("human")
-    except db.EntityDoesNotExistError:
+    except db.ResourceNotFoundError:
         human_role = admin._insert_role("human", "An Earthling.")
 
     try:
         alien_role = admin._retrieve_role("alien")
-    except db.EntityDoesNotExistError:
+    except db.ResourceNotFoundError:
         alien_role = admin._insert_role("alien", "An Extra-terrestrial.")
 
     admin._set_roles("jane", ["human"])
@@ -111,9 +111,11 @@ Returns
 out : Container
     A container of retrieved entities, the length is given by the parameter count.
     """
-    cont = db.execute_query("FIND RECORD Guitar", flags={"P": "0L{n}".format(n=count)})
+    cont = db.execute_query("FIND RECORD Guitar", flags={
+                            "P": "0L{n}".format(n=count)})
     if len(cont) != count:
-        raise db.CaosDBException(msg="Incorrect number of entitities returned.")
+        raise db.CaosDBException(
+            msg="Incorrect number of entitities returned.")
     return cont
 
 
@@ -138,7 +140,8 @@ general : bool, optional
 
     # Set general permissions
     if general:
-        grant = admin.PermissionRule(action="grant", permission="RETRIEVE:OWNER")
+        grant = admin.PermissionRule(
+            action="grant", permission="RETRIEVE:OWNER")
         deny = admin.PermissionRule(action="deny", permission="RETRIEVE:FILE")
 
         admin._set_permissions(role=role_grant, permission_rules=[grant])
@@ -189,9 +192,12 @@ None
         for ent in cont:
             ent.retrieve()
         print("Successfully retrieved all entities.")
-    except db.AuthorizationException:
+    except db.TransactionError as te:
+        if te.has_error(db.AuthorizationError):
             print(ent)
             print("Could not retrieve this entity although it should have been possible!")
+        else:
+            raise te
 
     # Switch to user without permissions
     db.configure_connection(username=denied_user[0], password=denied_user[1],
@@ -206,8 +212,11 @@ None
             denied_all = False
             print(ent)
             print("Could retrieve this entity although it should not have been possible!")
-        except db.AuthorizationException:
-            pass
+        except db.TransactionError as te:
+            # Only do something if an error wasn't caused by an
+            # AuthorizationError
+            if not te.has_error(db.AuthorizationError):
+                raise te
     if denied_all:
         print("Retrieval of all entities was successfully denied.")
 

release.sh

 #!/bin/bash
-rm -r dist/ build/ .eggs/
+rm -rf dist/ build/ .eggs/
 python setup.py sdist bdist_wheel
 python -m twine upload -s dist/*

setup.py

@@ -46,8 +46,8 @@ from setuptools import find_packages, setup
 ########################################################################
 
 MAJOR = 0
-MINOR = 4
-MICRO = 1
+MINOR = 5
+MICRO = 2
 PRE = ""  # e.g. rc0, alpha.1, 0.beta-23
 ISRELEASED = False
 

src/caosdb/__init__.py

@@ -48,8 +48,8 @@ from caosdb.common.models import (ACL, ALL, FIX, NONE, OBLIGATORY, RECOMMENDED,
                                   get_known_permissions, raise_errors)
 from caosdb.configuration import configure, get_config
 from caosdb.connection.connection import configure_connection, get_connection
-from caosdb.exceptions import *
 from caosdb.version import version as __version__
+from caosdb.exceptions import *
 
 # read configuration these files
 

src/caosdb/apiutils.py

@@ -34,10 +34,10 @@ from collections.abc import Iterable
 from subprocess import call
 
 from caosdb.common.datatype import (BOOLEAN, DATETIME, DOUBLE, FILE, INTEGER,
-                                    REFERENCE, TEXT)
+                                    REFERENCE, TEXT, is_reference)
 from caosdb.common.models import (Container, Entity, File, Property, Query,
                                   Record, RecordType, get_config,
-                                  execute_query, is_reference)
+                                  execute_query)
 
 
 def new_record(record_type, name=None, description=None,
@@ -244,6 +244,8 @@ class CaosDBPythonEntity(object):
             return (val, False)
         elif pr[0:4] == "LIST":
             return self._type_converted_list(val, pr)
+        elif isinstance(val, Entity):
+            return (convert_to_python_object(val), False)
         else:
             return (int(val), True)
 

src/caosdb/common/administration.py

@@ -30,8 +30,9 @@ from lxml import etree
 
 from caosdb.common.utils import xml2str
 from caosdb.connection.connection import get_connection
-from caosdb.exceptions import (AuthorizationException, ClientErrorException,
-                               EntityDoesNotExistError, ServerConfigurationException)
+from caosdb.exceptions import (HTTPClientError,
+                               HTTPForbiddenError,
+                               HTTPResourceNotFoundError)
 
 
 def set_server_property(key, value):
@@ -112,10 +113,10 @@ def _retrieve_user(name, realm=None, **kwargs):
     con = get_connection()
     try:
         return con._http_request(method="GET", path="User/" + (realm + "/" + name if realm is not None else name), **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to retrieve this user."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "User does not exist."
         raise
 
@@ -124,10 +125,10 @@ def _delete_user(name, **kwargs):
     con = get_connection()
     try:
         return con._http_request(method="DELETE", path="User/" + name, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to delete this user."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "User does not exist."
         raise
 
@@ -150,13 +151,13 @@ def _update_user(name, realm=None, password=None, status=None,
         params["entity"] = str(entity)
     try:
         return con.put_form_data(entity_uri_segment="User/" + (realm + "/" + name if realm is not None else name), params=params, **kwargs).read()
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "User does not exist."
         raise
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to update this user."
         raise
-    except ClientErrorException as e:
+    except HTTPClientError as e:
         if e.status == 409:
             e.msg = "Entity does not exist."
         raise
@@ -179,10 +180,10 @@ def _insert_user(name, password=None, status=None, email=None, entity=None, **kw
         params["entity"] = entity
     try:
         return con.post_form_data(entity_uri_segment="User", params=params, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to insert a new user."
         raise e
-    except ClientErrorException as e:
+    except HTTPClientError as e:
         if e.status == 409:
             e.msg = "User name is already in use."
 
@@ -195,10 +196,10 @@ def _insert_role(name, description, **kwargs):
     con = get_connection()
     try:
         return con.post_form_data(entity_uri_segment="Role", params={"role_name": name, "role_description": description}, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to insert a new role."
         raise
-    except ClientErrorException as e:
+    except HTTPClientError as e:
         if e.status == 409:
             e.msg = "Role name is already in use. Choose a different name."
         raise
@@ -208,10 +209,10 @@ def _update_role(name, description, **kwargs):
     con = get_connection()
     try:
         return con.put_form_data(entity_uri_segment="Role/" + name, params={"role_description": description}, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to update this role."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "Role does not exist."
         raise
 
@@ -220,10 +221,10 @@ def _retrieve_role(name, **kwargs):
     con = get_connection()
     try:
         return con._http_request(method="GET", path="Role/" + name, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to retrieve this role."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "Role does not exist."
         raise
 
@@ -232,10 +233,10 @@ def _delete_role(name, **kwargs):
     con = get_connection()
     try:
         return con._http_request(method="DELETE", path="Role/" + name, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to delete this role."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "Role does not exist."
         raise
 
@@ -249,14 +250,15 @@ def _set_roles(username, roles, realm=None, **kwargs):
     body = xml2str(xml)
     con = get_connection()
     try:
-        body = con._http_request(method="PUT", path="UserRoles/" + (realm + "/" + username if realm is not None else username), body=body, **kwargs).read()
-    except AuthorizationException as e:
+        body = con._http_request(method="PUT", path="UserRoles/" + (realm + "/" +
+                                                                    username if realm is not None else username), body=body, **kwargs).read()
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to set this user's roles."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "User does not exist."
         raise
-    except ClientErrorException as e:
+    except HTTPClientError as e:
         if e.status == 409:
             e.msg = "Role does not exist."
         raise
@@ -272,11 +274,12 @@ def _set_roles(username, roles, realm=None, **kwargs):
 def _get_roles(username, realm=None, **kwargs):
     con = get_connection()
     try:
-        body = con._http_request(method="GET", path="UserRoles/" + (realm + "/" + username if realm is not None else username), **kwargs).read()
-    except AuthorizationException as e:
+        body = con._http_request(method="GET", path="UserRoles/" + (
+            realm + "/" + username if realm is not None else username), **kwargs).read()
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to retrieve this user's roles."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "User does not exist."
         raise
     ret = set()
@@ -316,10 +319,10 @@ Returns
     con = get_connection()
     try:
         return con._http_request(method="PUT", path="PermissionRules/" + role, body=body, **kwargs).read()
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to set this role's permissions."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "Role does not exist."
         raise
 
@@ -328,10 +331,10 @@ def _get_permissions(role, **kwargs):
     con = get_connection()
     try:
         return PermissionRule._parse_body(con._http_request(method="GET", path="PermissionRules/" + role, **kwargs).read())
-    except AuthorizationException as e:
+    except HTTPForbiddenError as e:
         e.msg = "You are not permitted to retrieve this role's permissions."
         raise
-    except EntityDoesNotExistError as e:
+    except HTTPResourceNotFoundError as e:
         e.msg = "Role does not exist."
         raise
 

src/caosdb/common/datatype.py

@@ -25,7 +25,7 @@
 
 import re
 
-from ..exceptions import AmbiguityException, EntityDoesNotExistError
+from ..exceptions import EmptyUniqueQueryError, QueryNotUniqueError
 
 DOUBLE = "DOUBLE"
 REFERENCE = "REFERENCE"
@@ -91,9 +91,9 @@ def get_id_of_datatype(datatype):
 
     Raises
     ------
-    AmbiguityException
+    QueryNotUniqueError
         If there are more than one entities with the same name as the datatype.
-    EntityDoesNotExistError
+    EmptyUniqueQueryError
         If there is no entity with the name of the datatype.
     """
     from caosdb import execute_query
@@ -107,11 +107,11 @@ def get_id_of_datatype(datatype):
     res = [el for el in res if el.name.lower() == datatype.lower()]
 
     if len(res) > 1:
-        raise AmbiguityException(
+        raise QueryNotUniqueError(
             "Name {} did not lead to unique result; Missing "
             "implementation".format(datatype))
     elif len(res) != 1:
-        raise EntityDoesNotExistError(
+        raise EmptyUniqueQueryError(
             "No RecordType named {}".format(datatype))
 
     return res[0].id

src/caosdb/connection/authentication/auth_token.py

@@ -30,7 +30,7 @@ An Authentictor which only uses only a pre-supplied authentication token.
 from __future__ import absolute_import, unicode_literals, print_function
 from .interface import AbstractAuthenticator, CaosDBServerConnection
 from caosdb.connection.utils import auth_token_to_cookie
-from caosdb.exceptions import LoginFailedException
+from caosdb.exceptions import LoginFailedError
 
 
 def get_authentication_provider():
@@ -68,7 +68,7 @@ class AuthTokenAuthenticator(AbstractAuthenticator):
         self._login()
 
     def _login(self):
-        raise LoginFailedException("The authentication token is expired or you "
+        raise LoginFailedError("The authentication token is expired or you "
                                "have been logged out otherwise. The "
                                "auth_token authenticator cannot log in "
                                "again. You must provide a new "

src/caosdb/connection/authentication/interface.py

@@ -31,7 +31,7 @@ import logging
 from caosdb.connection.utils import urlencode
 from caosdb.connection.interface import CaosDBServerConnection
 from caosdb.connection.utils import parse_auth_token, auth_token_to_cookie
-from caosdb.exceptions import LoginFailedException
+from caosdb.exceptions import LoginFailedError
 
 # meta class compatible with Python 2 *and* 3:
 ABC = ABCMeta('ABC', (object, ), {'__slots__': ()})
@@ -197,9 +197,9 @@ class CredentialsAuthenticator(AbstractAuthenticator):
 
         # we need a username for this:
         if username is None:
-            raise LoginFailedException("No username was given.")
+            raise LoginFailedError("No username was given.")
         if password is None:
-            raise LoginFailedException("No password was given")
+            raise LoginFailedError("No password was given")
 
         headers = {}
         headers["Content-Type"] = "application/x-www-form-urlencoded"
@@ -210,7 +210,7 @@ class CredentialsAuthenticator(AbstractAuthenticator):
 
         response.read()  # clear socket
         if response.status != 200:
-            raise LoginFailedException("LOGIN WAS NOT SUCCESSFUL")
+            raise LoginFailedError("LOGIN WAS NOT SUCCESSFUL")
         self.on_response(response)
         return response
 

src/caosdb/connection/authentication/keyring.py

@@ -30,7 +30,7 @@ retrieve the password.
 import sys
 import imp
 from getpass import getpass
-from caosdb.exceptions import ConfigurationException
+from caosdb.exceptions import ConfigurationError
 from .external_credentials_provider import ExternalCredentialsProvider
 from .interface import CredentialsAuthenticator
 
@@ -67,7 +67,7 @@ def _get_external_keyring():
 
 def _call_keyring(**config):
     if "username" not in config:
-        raise ConfigurationException("Your configuration did not provide a "
+        raise ConfigurationError("Your configuration did not provide a "
                                  "`username` which is needed by the "
                                  "`KeyringCaller` to retrieve the "
                                  "password in question.")

src/caosdb/connection/authentication/pass.py

@@ -28,7 +28,7 @@ password.
 """
 
 from subprocess import check_output, CalledProcessError
-from caosdb.exceptions import ConfigurationException
+from caosdb.exceptions import ConfigurationError
 from .interface import CredentialsAuthenticator
 from .external_credentials_provider import ExternalCredentialsProvider
 
@@ -50,7 +50,7 @@ def get_authentication_provider():
 
 def _call_pass(**config):
     if "password_identifier" not in config:
-        raise ConfigurationException("Your configuration did not provide a "
+        raise ConfigurationError("Your configuration did not provide a "
                                  "`password_identifier` which is needed "
                                  "by the `PassCaller` to retrieve the "
                                  "password in question.")

src/caosdb/connection/authentication/unauthenticated.py

@@ -30,7 +30,7 @@ cookies.
 """
 from __future__ import absolute_import, unicode_literals, print_function
 from .interface import AbstractAuthenticator, CaosDBServerConnection
-from caosdb.exceptions import LoginFailedException
+from caosdb.exceptions import LoginFailedError
 
 
 def get_authentication_provider():
@@ -70,7 +70,7 @@ class Unauthenticated(AbstractAuthenticator):
         self._login()
 
     def _login(self):
-        raise LoginFailedException("This caosdb client is configured to stay "
+        raise LoginFailedError("This caosdb client is configured to stay "
                                "unauthenticated. Change your "
                                "`password_method` and provide an "
                                "`auth_token` or credentials if you want "

src/caosdb/connection/connection.py

@@ -33,11 +33,14 @@ from errno import EPIPE as BrokenPipe
 from socket import error as SocketError
 
 from caosdb.configuration import get_config
-from caosdb.exceptions import (AuthorizationException, CaosDBException,
-                               ClientErrorException, ConfigurationException,
-                               ConnectionException, EntityDoesNotExistError,
-                               LoginFailedException, ServerErrorException,
-                               URITooLongException)
+from caosdb.exceptions import (CaosDBException, HTTPClientError,
+                               ConfigurationError,
+                               CaosDBConnectionError,
+                               HTTPForbiddenError,
+                               LoginFailedError,
+                               HTTPResourceNotFoundError,
+                               HTTPServerError,
+                               HTTPURITooLongError)
 from caosdb.version import version
 from pkg_resources import resource_filename
 
@@ -61,6 +64,10 @@ class _WrappedHTTPResponse(CaosDBHTTPResponse):
     def __init__(self, response):
         self.response = response
 
+    @property
+    def reason(self):
+        return self.response.reason
+
     @property
     def status(self):
         return self.response.status
@@ -88,7 +95,8 @@ class _DefaultCaosDBServerConnection(CaosDBServerConnection):
     """
 
     def __init__(self):
-        self._useragent = ("caosdb-pylib/{version} - {implementation}".format(version=version, implementation=type(self).__name__))
+        self._useragent = ("caosdb-pylib/{version} - {implementation}".format(
+            version=version, implementation=type(self).__name__))
         self._http_con = None
         self._base_path = None
 
@@ -132,7 +140,7 @@ class _DefaultCaosDBServerConnection(CaosDBServerConnection):
             self._http_con.request(method=method, url=self._base_path + path,
                                    headers=headers, body=body)
         except SocketError as socket_err:
-            raise ConnectionException(
+            raise CaosDBConnectionError(
                 "Connection failed. Network or server down? " + str(socket_err)
             )
 
@@ -156,7 +164,7 @@ class _DefaultCaosDBServerConnection(CaosDBServerConnection):
 
         Raises
         ------
-        ConnectionException
+        CaosDBConnectionError
             If no url has been specified, or if the CA certificate cannot be
             loaded.
         """
@@ -193,7 +201,7 @@ class _DefaultCaosDBServerConnection(CaosDBServerConnection):
             try:
                 context.load_verify_locations(config["cacert"])
             except Exception as exc:
-                raise ConnectionException("Could not load the cacert in"
+                raise CaosDBConnectionError("Could not load the cacert in"
                                             "`{}`: {}".format(config["cacert"],
                                                               exc))
 
@@ -204,7 +212,7 @@ class _DefaultCaosDBServerConnection(CaosDBServerConnection):
             host = parsed_url.netloc
             self._base_path = parsed_url.path
         else:
-            raise ConnectionException(
+            raise CaosDBConnectionError(
                 "No connection url specified. Please "
                 "do so via caosdb.configure_connection(...) or in a config "
                 "file.")
@@ -276,7 +284,7 @@ def _get_authenticator(**config):
 
     Raises
     ------
-    ConnectionException
+    ConfigurationError
         If the password_method string cannot be resolved to a CaosAuthenticator
         class.
     """
@@ -292,7 +300,7 @@ def _get_authenticator(**config):
         return auth_provider
 
     except ImportError:
-        raise ConfigurationException("Password method \"{}\" not implemented. "
+        raise ConfigurationError("Password method \"{}\" not implemented. "
                                  "Try `plain`, `pass`, `keyring`, or "
                                  "`auth_token`."
                                  .format(config["password_method"]))
@@ -398,29 +406,24 @@ def _handle_response_status(http_response):
     # emtpy response buffer
     body = http_response.read()
 
+    if status == 404:
+        raise HTTPResourceNotFoundError("This resource has not been found.")
+    elif status > 499:
+        raise HTTPServerError(body=body)
+
+    reason = http_response.reason
+    standard_message = ("Request failed. The response returned with status "
+                        "{} - {}.".format(status, reason))
     if status == 401:
-        raise LoginFailedException(
-            "Request failed. The response returned with status "
-            "{}.".format(status))
+        raise LoginFailedError(standard_message)
     elif status == 403:
-        raise AuthorizationException(
-            "Request failed. The response returned with status "
-            "{}.".format(status))
-    elif status == 404:
-        raise EntityDoesNotExistError("This entity does not exist.")
+        raise HTTPForbiddenError(standard_message)
     elif status in (413, 414):
-        raise URITooLongException(
-            "Request failed. The response returned with status "
-            "{}.".format(status))
+        raise HTTPURITooLongError(standard_message)
     elif 399 < status < 500:
-        raise ClientErrorException(msg=("Request failed. The response returned "
-                                        "with status {}.").format(status), status=status, body=body)
-    elif status > 499:
-        raise ServerErrorException(body=body)
+        raise HTTPClientError(msg=standard_message, status=status, body=body)
     else:
-        raise CaosDBException(
-            "Request failed. The response returned with status "
-            "{}.".format(status))
+        raise CaosDBException(standard_message)
 
 
 class _Connection(object):  # pylint: disable=useless-object-inheritance
@@ -454,7 +457,7 @@ class _Connection(object):  # pylint: disable=useless-object-inheritance
         self.is_configured = True
 
         if "implementation" not in config:
-            raise ConfigurationException(
+            raise ConfigurationError(
                 "Missing CaosDBServerConnection implementation. You did not "
                 "specify an `implementation` for the connection.")
         try:
@@ -465,18 +468,18 @@ class _Connection(object):  # pylint: disable=useless-object-inheritance
                 raise TypeError("The `implementation` callable did not return "
                                 "an instance of CaosDBServerConnection.")
         except TypeError as type_err:
-            raise ConfigurationException(
+            raise ConfigurationError(
                 "Bad CaosDBServerConnection implementation. The "
                 "implementation must be a callable object which returns an "
                 "instance of `CaosDBServerConnection` (e.g. a constructor "
-                "or a factory).", type_err)
+                "or a factory).\n{}".format(type_err.args[0]))
         self._delegate_connection.configure(**config)
 
         if "auth_token" in config:
             # deprecated, needed for older scripts
             config["password_method"] = "auth_token"
         if "password_method" not in config:
-            raise ConfigurationException("Missing password_method. You did "
+            raise ConfigurationError("Missing password_method. You did "
                                      "not specify a `password_method` for"
                                      "the connection.")
         self._authenticator = _get_authenticator(
@@ -554,8 +557,8 @@ class _Connection(object):  # pylint: disable=useless-object-inheritance
             uri_segments.extend(path.split("/"))
 
             return self.retrieve(entity_uri_segments=uri_segments)
-        except EntityDoesNotExistError:
-            raise EntityDoesNotExistError("This file does not exist.")
+        except HTTPResourceNotFoundError:
+            raise HTTPResourceNotFoundError("This file does not exist.")
 
     def _login(self):
         self._authenticator.login()
@@ -576,7 +579,7 @@ class _Connection(object):  # pylint: disable=useless-object-inheritance
                                             headers=headers, body=body,
                                             reconnect=False,
                                             **kwargs)
-        except LoginFailedException:
+        except LoginFailedError:
             if kwargs.get("reconnect", True) is True:
                 self._login()
 

src/caosdb/exceptions.py

@@ -5,6 +5,8 @@
 #
 # Copyright (C) 2018 Research Group Biomedical Physics,
 # Max-Planck-Institute for Dynamics and Self-Organization Göttingen
+# Copyright (C) 2020 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2020 Florian Spreckelsen <f.spreckelsen@indiscale.com>
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License as
@@ -21,6 +23,10 @@
 #
 # ** end header
 #
+"""The exceptions module defines exceptions for HTTP Errors (4xx and 5xx and
+HTTP response codes) and for transaction errors (i.e. missing permissions,
+dependencies, non-passing consistency checks etc.).
+"""
 
 from lxml import etree
 
@@ -28,13 +34,13 @@ from lxml import etree
 class CaosDBException(Exception):
     """Base class of all CaosDB exceptions."""
 
-    def __init__(self, msg=None, *args):
-        Exception.__init__(self, msg, *args)
+    def __init__(self, msg):
+        Exception.__init__(self, msg)
         self.msg = msg
 
 
-class ConfigurationException(CaosDBException):
-    """ConfigurationException.
+class ConfigurationError(CaosDBException):
+    """ConfigurationError.
 
     Indicates a misconfiguration.
 
@@ -43,7 +49,6 @@ class ConfigurationException(CaosDBException):
     msg : str
         A descriptin of the misconfiguration. The constructor adds
         a few lines with explainingg where to find the configuration.
-    *args
 
     Attributes
     ----------
@@ -51,13 +56,11 @@ class ConfigurationException(CaosDBException):
         A description of the misconfiguration.
     """
 
-    def __init__(self, msg, *args):
-        super(ConfigurationException, self).__init__(msg +
-                                                     ConfigurationException._INFO,
-                                                     *args)
+    def __init__(self, msg):
+        super().__init__(msg + ConfigurationError._INFO)
 
     _INFO = ("\n\nPlease check your ~/.pycaosdb.ini and your $PWD/"
-             ".pycaosdb.ini. Do at least one of them exist and are they correct?")
+             ".pycaosdb.ini. Does at least one of them exist and are they correct?")
 
 
 class ServerConfigurationException(CaosDBException):
@@ -67,14 +70,18 @@ class ServerConfigurationException(CaosDBException):
     """
 
 
-class ClientErrorException(CaosDBException):
+class HTTPClientError(CaosDBException):
+    """HTTPClientError represents 4xx HTTP client errors."""
+
     def __init__(self, msg, status, body):
         self.status = status
         self.body = body
         CaosDBException.__init__(self, msg)
 
 
-class ServerErrorException(CaosDBException):
+class HTTPServerError(CaosDBException):
+    """HTTPServerError represents 5xx HTTP server errors."""
+
     def __init__(self, body):
         xml = etree.fromstring(body)
         error = xml.xpath('/Response/Error')[0]
@@ -85,107 +92,126 @@ class ServerErrorException(CaosDBException):
         CaosDBException.__init__(self, msg)
 
 
-class ConnectionException(CaosDBException):
+class CaosDBConnectionError(CaosDBException):
     """Connection is not configured or the network is down."""
 
     def __init__(self, msg=None):
         CaosDBException.__init__(self, msg)
 
 
-class URITooLongException(CaosDBException):
+class HTTPURITooLongError(HTTPClientError):
     """The URI of the last request was too long."""
 
     def __init__(self, msg=None):
-        CaosDBException.__init__(self, msg)
+        HTTPClientError.__init__(self, msg=msg, status=414, body=None)
 
 
-class AmbiguityException(CaosDBException):
-    """A retrieval of an entity that was supposed to be uniquely identifiable
-    returned two or more entities."""
+class LoginFailedError(CaosDBException):
+    """Login failed.
+
+    Probably, your username/password pair is wrong.
+    """
 
     def __init__(self, msg=None):
-        CaosDBException.__init__(self, msg)
+        CaosDBException.__init__(self, msg=msg)
 
 
-class LoginFailedException(CaosDBException):
-    """Login failed.
+class HTTPForbiddenError(HTTPClientError):
+    """You're lacking the required permissions. Corresponds to HTTP status
+    403.
 
-    Probably, your username/password pair is wrong.
     """
 
     def __init__(self, msg=None):
-        CaosDBException.__init__(self, msg=msg)
+        HTTPClientError.__init__(self, msg=msg, status=403, body=None)
 
 
-class TransactionError(CaosDBException):
+class HTTPResourceNotFoundError(HTTPClientError):
+    """The requested resource doesn't exist; corresponds to HTTP status
+    404.
 
-    def _calc_bases(self):
-        types = dict()
-        # collect each class once
+    """
 
-        for err in self.errors:
-            types[id(type(err))] = type(err)
-        # delete redundant super classes
+    def __init__(self, msg=None):
+        HTTPClientError.__init__(self, msg=msg, status=404, body=None)
 
-        if len(types.values()) > 1:
-            # remove TransactionError
-            try:
-                del types[id(TransactionError)]
-            except KeyError:
-                pass
 
-        if len(types.values()) > 1:
-            # remove EntityError
-            try:
-                del types[id(EntityError)]
-            except KeyError:
-                pass
+class MismatchingEntitiesError(CaosDBException):
+    """Mismatching entities were found during container sync."""
 
-        ret = ()
 
-        for t in types.values():
-            ret += (t,)
+# ######################### Bad query errors ###########################
 
-        if ret == ():
-            ret = (type(self),)
 
-        return ret
+class BadQueryError(CaosDBException):
+    """Base class for query errors that are not transaction errors."""
 
-    def __init__(self, container=None, error=None, msg=None):
-        self.container = container
-        self.errors = []
-        self.msg = msg if msg is not None else str(error)
-        self.error = error
 
-    def print_errs(self):
-        print(self)
+class QueryNotUniqueError(BadQueryError):
+    """A unique query or retrieve found more than one entity."""
 
-        for err in self.errors:
-            err.print_errs()
 
-    def _convert(self):
-        t = self._calc_bases()
-        try:
-            newtype = type('TransactionError', t, {})
-        except BaseException:
-            self.print_errs()
-            raise
-        newinstance = newtype(container=self.container, error=self.msg)
-        newinstance.errors = self.errors
-        newinstance.get_entities = self.get_entities
+class EmptyUniqueQueryError(BadQueryError):
+    """A unique query or retrieve dound no result."""
 
-        return newinstance
 
-    def get_container(self):
-        '''
-        @return: The container that raised this TransactionError during the last
-        transaction.
-        '''
+# ######################### Transaction errors #########################
 
-        return self.container
+
+class TransactionError(CaosDBException):
+    """An error of this type is raised whenever any transaction fails with
+    one or more entities between client and CaosDB server. More
+    detailed errors are collected as direct and indirect children in
+    the 'errors' list (direct children) and the 'all_errors' set (set
+    of all direct and indirect children).
+
+    """
+
+    def __init__(self, error=None,
+                 msg="An error occured during the transaction.",
+                 container=None):
+        CaosDBException.__init__(self, msg=msg)
+        self.errors = []
+        self.all_errors = set()
+        self.entities = []
+        self.all_entities = set()
+        self.container = container
+        # special case of faulty container
+        if container is not None and container.get_errors() is not None:
+            self.code = container.get_errors()[0].code
+        else:
+            self.code = None
+        if error is not None:
+            self.add_error(error)
+
+    def has_error(self, error_t, direct_children_only=False):
+        """Check whether this transaction error contains an error of type
+        error_t. If direct_children_only is True, only direct children
+        are checked.
+
+        Parameters:
+        -----------
+        error_t : EntityError
+            error type to be checked
+        direct_children_only: bool, optional
+            If True, only direct children, i.e., all errors in
+            self.errors are checked. Else all direct and indirect
+            children, i.e., all errors in self.all_errors are
+            used. Default is false.
+
+        Returns:
+        --------
+        has_error : bool
+            True if at least one of the children is of type error_t,
+            False otherwise.
+
+        """
+
+        test_set = self.errors if direct_children_only else self.all_errors
+        return any([isinstance(err, error_t) for err in test_set])
 
     def add_error(self, error):
-        """Add an error to this TransactionError.
+        """Add an error as a direct child to this TransactionError.
 
         @param error: An EntityError or a list of EntityErrors.
 
@@ -196,35 +222,31 @@ class TransactionError(CaosDBException):
         """
 
         if hasattr(error, "__iter__"):
-            for e in error:
-                self.add_error(e)
+            for err in error:
+                self.add_error(err)
 
             return self
-        elif isinstance(error, TransactionError):
+        elif isinstance(error, EntityError):
             self.errors.append(error)
+            self.entities.append(error.entity)
+
+            self.all_errors.add(error)
+            self.all_errors.update(error.all_errors)
+            self.all_entities.add(error.entity)
+            self.all_entities.update(error.all_entities)
 
             return self
         else:
             raise TypeError(
-                "Argument is to be an TransactionError or a list of TransactionErrors.")
-
-    def get_errors(self):
-        '''
-        @return: A list of all EntityError objects.
-        '''
-
-        if hasattr(self, 'errors'):
-            return self.errors
-
-        return None
+                "Argument is to be an EntityError or a list of EntityErrors.")
 
     def _repr_reasons(self, indent):
-        if self.get_errors() is not None and len(self.get_errors()) > 0:
+        if self.errors is not None and len(self.errors) > 0:
             ret = "\n" + indent + "    +--| REASONS |--"
 
-            for c in self.get_errors():
+            for err in self.errors:
                 ret += '\n' + indent + '    |  -> ' + \
-                    c.__str__(indent=indent + '    |')
+                    err.__str__(indent=indent + '    |')
             ret += "\n" + indent + "    +----------------"
 
             return ret
@@ -232,8 +254,11 @@ class TransactionError(CaosDBException):
             return ''
 
     def _repr_head(self, indent):
-        return str(type(self).__name__) + ((': ' + self.msg)
-                                           if hasattr(self, 'msg') and self.msg is not None else '')
+        return indent + str(type(self).__name__) + (
+            (': ' + self.msg)
+            if hasattr(self, 'msg') and self.msg is not None
+            else ''
+        )
 
     def __str__(self, indent=''):
         ret = self._repr_head(indent=indent)
@@ -244,58 +269,21 @@ class TransactionError(CaosDBException):
     def __repr__(self):
         return self.__str__()
 
-    def get_entities(self):
-        '''
-        @return: A list of all Entity objects with errors.
-        '''
-        ret = []
-
-        if hasattr(self, 'get_entity') and self.get_entity() is not None:
-            ret.append(self.get_entity())
-
-        for error in self.errors:
-            if hasattr(error, 'get_entity'):
-                if error.get_entity() not in ret:
-                    ret.append(error.get_entity())
-#             if hasattr(error, 'get_entities'):
-#                 for e in error.get_entities():
-#                     if e not in ret:
-#                         ret.append(e)
-        return ret
-
-    def get_error(self):
-        return self.error
-
 
 class EntityError(TransactionError):
+    """This is the most basic entity error. It is constructed using an
+    entity that caused the error and the error message attached by the
+    server.
 
-    @staticmethod
-    def _sort_t(t):
-        if len(t) > 1:
-            ret = ()
-            '''remove EntityError'''
-
-            for i in range(len(t)):
-                if t[i] != EntityError:
-                    ret += (t[i],)
-            t = ret
-
-        return t
-
-    def _convert(self):
-        t = self._calc_bases()
-        # TODO is it really a good idea to create dynamically types here?
-        newtype = type('EntityMultiError', t+(Exception,), {})
-        newinstance = newtype(error=self.error, entity=self.entity)
-        setattr(newinstance, 'msg', self.msg)
-        setattr(newinstance, 'errors', self.errors)
-        setattr(newinstance, 'container', self.container)
-
-        return newinstance
+    """
 
-    def __init__(self, error=None, container=None, entity=None):
-        TransactionError.__init__(self, container=container)
+    def __init__(self, error=None, entity=None):
+        TransactionError.__init__(self)
         self.error = error
+        if hasattr(error, "code"):
+            self.code = error.code
+        else:
+            self.code = None
         self.entity = entity
 
         if error is not None and hasattr(error, "encode"):
@@ -307,34 +295,17 @@ class EntityError(TransactionError):
         else:
             self.msg = str(error)
 
-    def get_entity(self):
-        '''
-        @return: The entity that caused this error.
-        '''
-
-        if hasattr(self, 'entity'):
-            return self.entity
-
-        return None
-
     @property
     def description(self):
+        """The description of the error."""
         return self.error.description if self.error is not None else None
 
-    def get_code(self):
-        return self.error.code if self.error is not None else None
-
-    def get_error(self):
-        '''
-        @return: Error Message object of this Error.
-        '''
-
-        return self.error
-
     def _repr_head(self, indent):
         if hasattr(self, 'entity') and self.entity is not None:
-            return str(type(self.entity).__name__).upper() + " (" + str(self.entity.id) + (("," + "'" + str(self.entity.name) + "'")
-                                                                                           if self.entity.name is not None else '') + ") CAUSED " + TransactionError._repr_head(self, indent)
+            return (str(type(self.entity).__name__).upper() + " (id: " +
+                    str(self.entity.id) + ((", name: " + "'" + str(self.entity.name) + "'") if
+                                           self.entity.name is not None else '') + ") CAUSED " +
+                    TransactionError._repr_head(self, indent))
         else:
             return TransactionError._repr_head(self, indent)
 
@@ -344,15 +315,19 @@ class UniqueNamesError(EntityError):
 
 
 class UnqualifiedParentsError(EntityError):
-    """This entity has unqualified parents (call 'get_errors()' for a list of
-    errors of the parent entities or 'get_entities()' for a list of parent
-    entities with errors)."""
+    """This entity has unqualified parents (see 'errors' attribute for a
+    list of errors of the parent entities or 'entities' attribute for
+    a list of parent entities with errors).
+
+    """
 
 
 class UnqualifiedPropertiesError(EntityError):
-    """This entity has unqualified properties (call 'get_errors()' for a list
-    of errors of the properties or 'get_entities()' for a list of properties
-    with errors)."""
+    """This entity has unqualified properties (see 'errors' attribute for
+    a list of errors of the properties or 'entities' attribute for a
+    list of properties with errors).
+
+    """
 
 
 class EntityDoesNotExistError(EntityError):
@@ -364,11 +339,17 @@ class EntityHasNoDatatypeError(EntityError):
 
 
 class ConsistencyError(EntityError):
-    pass
+    """The transaction violates database consistency."""
 
 
-class AuthorizationException(EntityError):
+class AuthorizationError(EntityError):
     """You are not allowed to do what ever you tried to do.
 
-    Maybe you need more privileges or a user account at all.
+    Maybe you need more privileges or a user account.
+    """
+
+
+class AmbiguousEntityError(EntityError):
+    """A retrieval of the entity was not possible because there is more
+    than one possible candidate.
     """