Skip to content
Snippets Groups Projects
Commit 238a814a authored by Henrik tom Wörden's avatar Henrik tom Wörden
Browse files

Merge branch 'f-linkahead-rename-tmp' into 'dev'

Helper MR

See merge request !114
parents e06fa0b1 a4bc8bb4
Branches
Tags
2 merge requests!114Helper MR,!112LinkAhead rename lvl 0
Pipeline #41949 passed
Showing
with 128 additions and 1463 deletions
...@@ -13,6 +13,7 @@ __pycache__/ ...@@ -13,6 +13,7 @@ __pycache__/
dist/ dist/
build/ build/
src/caosdb/version.py src/caosdb/version.py
src/linkahead/version.py
# documentation # documentation
_apidoc _apidoc
......
# #
# This file is a part of the CaosDB Project. # This file is a part of the LinkAhead Project.
# #
# Copyright (C) 2018 Research Group Biomedical Physics, # Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen # Max-Planck-Institute for Dynamics and Self-Organization Göttingen
...@@ -44,7 +44,7 @@ code_style: ...@@ -44,7 +44,7 @@ code_style:
- make style - make style
allow_failure: true allow_failure: true
# pylint tests for pycaosdb # pylint tests for pylinkahead
pylint: pylint:
tags: [ docker ] tags: [ docker ]
stage: linting stage: linting
...@@ -61,7 +61,7 @@ unittest_py3.7: ...@@ -61,7 +61,7 @@ unittest_py3.7:
image: python:3.7 image: python:3.7
script: &python_test_script script: &python_test_script
# Python docker has problems with tox and pip so use plain pytest here # Python docker has problems with tox and pip so use plain pytest here
- touch ~/.pycaosdb.ini - touch ~/.pylinkahead.ini
- pip install nose pytest pytest-cov python-dateutil jsonschema>=4.4.0 - pip install nose pytest pytest-cov python-dateutil jsonschema>=4.4.0
- pip install . - pip install .
- python -m pytest unittests - python -m pytest unittests
...@@ -82,7 +82,7 @@ unittest_py3.9: ...@@ -82,7 +82,7 @@ unittest_py3.9:
script: script:
# verify that this actually is Python 3.9 # verify that this actually is Python 3.9
- python3 -c "import sys; assert sys.version.startswith('3.9')" - python3 -c "import sys; assert sys.version.startswith('3.9')"
- touch ~/.pycaosdb.ini - touch ~/.pylinkahead.ini
- make unittest - make unittest
......
...@@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ...@@ -20,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* `Message.__init__` signature changed and `type` defaults to "Info" now. * `Message.__init__` signature changed and `type` defaults to "Info" now.
* `Message.__eq__` changed. Equality is equality of `type`, `code`, and * `Message.__eq__` changed. Equality is equality of `type`, `code`, and
`description` now. `description` now.
* Rename from CaosDB to LinkAhead. For proper migration, follow the instructions
in `migration_to_linkahead.md` and check the documentation at [docs.indiscale.com](https://docs.indiscale.com/caosdb-pylib/README_SETUP.html#migration).
### Deprecated ### ### Deprecated ###
......
# ** header v3.0 # ** header v3.0
# This file is a part of the CaosDB Project. # This file is a part of the LinkAhead Project.
# #
# Copyright (C) 2020 IndiScale GmbH <info@indiscale.com> # Copyright (C) 2020 IndiScale GmbH <info@indiscale.com>
# Copyright (C) 2020 Daniel Hornung <d.hornung@indiscale.com> # Copyright (C) 2020 Daniel Hornung <d.hornung@indiscale.com>
...@@ -40,7 +40,7 @@ style: ...@@ -40,7 +40,7 @@ style:
.PHONY: style .PHONY: style
lint: lint:
pylint --unsafe-load-any-extension=y -d all -e E,F src/caosdb/common pylint --unsafe-load-any-extension=y -d all -e E,F src/linkahead/common
.PHONY: lint .PHONY: lint
unittest: unittest:
......
...@@ -41,7 +41,7 @@ the preferred way is also a merge request as describe above (the documentation r ...@@ -41,7 +41,7 @@ the preferred way is also a merge request as describe above (the documentation r
However, you can also create an issue for it. However, you can also create an issue for it.
* You can also contact us at **info (AT) caosdb.org** and join the * You can also contact us at **info (AT) caosdb.org** and join the
CaosDB community on CaosDB community on
[#caosdb:matrix.org](https://matrix.to/#/!unwwlTfOznjEnMMXxf:matrix.org). [#linkahead:matrix.org](https://matrix.to/#/!unwwlTfOznjEnMMXxf:matrix.org).
## License ## License
......
...@@ -128,3 +128,6 @@ Build documentation in `build/` with `make doc`. ...@@ -128,3 +128,6 @@ Build documentation in `build/` with `make doc`.
### Troubleshooting ### ### Troubleshooting ###
If the client is to be executed directly from the `/src` folder, an initial `.\setup.py install --user` must be called. If the client is to be executed directly from the `/src` folder, an initial `.\setup.py install --user` must be called.
## Migration ##
TODO
# Release Guidelines for the CaosDB Python Client Library # Release Guidelines for the CaosDB Python Client Library
This document specifies release guidelines in addition to the general release This document specifies release guidelines in addition to the general release
guidelines of the CaosDB Project guidelines of the LinkAhead Project
([RELEASE_GUIDELINES.md](https://gitlab.com/caosdb/caosdb/blob/dev/RELEASE_GUIDELINES.md)) ([RELEASE_GUIDELINES.md](https://gitlab.com/caosdb/caosdb/blob/dev/RELEASE_GUIDELINES.md))
## General Prerequisites ## General Prerequisites
......
#!/usr/bin/env python3
# encoding: utf-8
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
# Copyright (C) 2021 Henrik tom Wörden <h.tomwoerden@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
import os
base_root = "src/linkahead/"
initcontent = """
from {module} import *
from warnings import warn
warn(("CaosDB was renamed to LinkAhead. Please import this library as `import {module}`. Using the"
" old name, starting with caosdb, is deprecated."), DeprecationWarning)
"""
for root, dirs, files in os.walk(base_root, topdown=False):
if root.endswith("__pycache__"):
continue
cdir = os.path.join("src/caosdb", root[len(base_root):])
os.makedirs(cdir, exist_ok=True)
for fi in files:
if not fi.endswith(".py"):
continue
path = os.path.join(cdir, fi)
with open(path, 'w') as cur:
if fi == "__init__.py":
cur.write(initcontent.format(module=".".join(
os.path.join(root, fi[:-3]).split('/')[1:-1])))
else:
cur.write(initcontent.format(module=".".join(
os.path.join(root, fi[:-3]).split('/')[1:])))
#!/usr/bin/env python3 #!/usr/bin/env python3
"""A small example to get started with caosdb-pylib. """A small example to get started with caosdb-pylib.
Make sure that a `pycaosdb.ini` is readable at one of the expected locations. Make sure that a `pylinkahead.ini` is readable at one of the expected locations.
""" """
import random import random
......
# To be found be the caosdb package, the INI file must be located either in # To be found be the caosdb package, the INI file must be located either in
# - $CWD/pycaosdb.ini # - $CWD/pylinkahead.ini
# - $HOME/.pycaosdb.ini # - $HOME/.pylinkahead.ini
# - the location given in the env variable PYCAOSDBINI # - the location given in the env variable PYCAOSDBINI
[Connection] [Connection]
......
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# #
# ** header v3.0 # ** header v3.0
# This file is a part of the CaosDB Project. # This file is a part of the LinkAhead Project.
# #
# Copyright (C) 2018 Research Group Biomedical Physics, # Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen # Max-Planck-Institute for Dynamics and Self-Organization Göttingen
......
#!/usr/bin/env python3 #!/usr/bin/env python3
# ** header v3.0 # ** header v3.0
# This file is a part of the CaosDB Project. # This file is a part of the LinkAhead Project.
# #
# Copyright (c) 2019 IndiScale GmbH # Copyright (c) 2019 IndiScale GmbH
# Copyright (c) 2019 Daniel Hornung <d.hornung@indiscale.com> # Copyright (c) 2019 Daniel Hornung <d.hornung@indiscale.com>
...@@ -25,7 +25,7 @@ ...@@ -25,7 +25,7 @@
As a result, only a specific user or group may access it. As a result, only a specific user or group may access it.
This script assumes that the user specified in the pycaosdb.ini This script assumes that the user specified in the pylinkahead.ini
configuration can create new entities. configuration can create new entities.
""" """
......
git merge linkahead-rename-step-1
# resolve potential conflicts and commit
rm -rf src/linkahead
git mv src/caosdb/ src/linkahead
rm -rf src/caosdb
python3 create_slim_linkahead_wrapper.py
git add src
git ci -m "MAINT: rename caosdb to linkahead (module)"
git merge linkahead-rename-step-2
# resolve potential conflicts and commit
git merge dev
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
# -*- encoding: utf-8 -*- # -*- encoding: utf-8 -*-
# #
# #
"""caosdb""" """linkahead"""
import os import os
import subprocess import subprocess
import sys import sys
...@@ -91,22 +91,23 @@ def git_version(): ...@@ -91,22 +91,23 @@ def git_version():
def get_version_info(): def get_version_info():
# Adding the git rev number needs to be done inside write_version_py(), # Adding the git rev number needs to be done inside write_version_py(),
# otherwise the import of caosdb.version messes up the build under # otherwise the import of linkahead.version messes up the build under
# Python 3. # Python 3.
FULLVERSION = VERSION FULLVERSION = VERSION
# Magic which is only really needed in the pipelines. Therefore: a lot of dark pipeline magic.
if os.path.exists('.git'): if os.path.exists('.git'):
GIT_REVISION = git_version() GIT_REVISION = git_version()
elif os.path.exists('caosdb_pylib_commit'): elif os.path.exists('linkahead_pylib_commit'):
with open('caosdb_pylib_commit', 'r') as f: with open('linkahead_pylib_commit', 'r') as f:
GIT_REVISION = f.read().strip() GIT_REVISION = f.read().strip()
elif os.path.exists('src/caosdb/version.py'): elif os.path.exists('src/linkahead/version.py'):
# must be a source distribution, use existing version file # must be a source distribution, use existing version file
try: try:
from caosdb.version import git_revision as GIT_REVISION from linkahead.version import git_revision as GIT_REVISION
except ImportError: except ImportError:
raise ImportError("Unable to import git_revision. Try removing " raise ImportError("Unable to import git_revision. Try removing "
"src/caosdb/version.py and the build directory " "src/linkahead/version.py and the build directory "
"before building.") "before building.")
else: else:
GIT_REVISION = "Unknown" GIT_REVISION = "Unknown"
...@@ -117,9 +118,9 @@ def get_version_info(): ...@@ -117,9 +118,9 @@ def get_version_info():
return FULLVERSION, GIT_REVISION return FULLVERSION, GIT_REVISION
def write_version_py(filename='src/caosdb/version.py'): def write_version_py(filename='src/linkahead/version.py'):
cnt = """ cnt = """
# THIS FILE IS GENERATED FROM caosdb SETUP.PY # THIS FILE IS GENERATED FROM linkahead SETUP.PY
# #
short_version = '%(version)s' short_version = '%(version)s'
version = '%(version)s' version = '%(version)s'
...@@ -154,14 +155,14 @@ def setup_package(): ...@@ -154,14 +155,14 @@ def setup_package():
write_version_py() write_version_py()
metadata = dict( metadata = dict(
name='caosdb', name='linkahead',
version=get_version_info()[0], version=get_version_info()[0],
description='Python Interface for CaosDB', description='Python Interface for LinkAhead',
long_description=long_description, long_description=long_description,
long_description_content_type="text/markdown", long_description_content_type="text/markdown",
author='Timm Fitschen', author='Timm Fitschen',
author_email='t.fitschen@indiscale.com', author_email='t.fitschen@indiscale.com',
url='https://www.caosdb.org', url='https://www.linkahead.org',
license="AGPLv3+", license="AGPLv3+",
classifiers=[ classifiers=[
"Programming Language :: Python :: 3", "Programming Language :: Python :: 3",
...@@ -185,9 +186,12 @@ def setup_package(): ...@@ -185,9 +186,12 @@ def setup_package():
tests_require=["pytest", "pytest-cov", "coverage>=4.4.2", tests_require=["pytest", "pytest-cov", "coverage>=4.4.2",
"jsonschema>=4.4.0"], "jsonschema>=4.4.0"],
package_data={ package_data={
'caosdb': ['cert/indiscale.ca.crt', 'schema-pycaosdb-ini.yml'], 'linkahead': ['cert/indiscale.ca.crt', 'schema-pycaosdb-ini.yml'],
}, },
scripts=["src/caosdb/utils/caosdb_admin.py"] scripts=[
"src/linkahead/utils/caosdb_admin.py",
"src/linkahead/utils/linkahead_admin.py"
]
) )
try: try:
setup(**metadata) setup(**metadata)
......
# -*- coding: utf-8 -*-
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
"""CaosDB Python bindings. from linkahead import *
from warnings import warn
Tries to read from the inifile specified in the environment variable `PYCAOSDBINI` or alternatively warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead`. Using the"
in `~/.pycaosdb.ini` upon import. After that, the ini file `pycaosdb.ini` in the current working " old name, starting with caosdb, is deprecated."), DeprecationWarning)
directory will be read additionally, if it exists.
"""
from os import environ, getcwd
# Import of the connection function (which is used to connect to the DB):
from os.path import expanduser, join
# Import of convenience methods:
import caosdb.apiutils
from caosdb.common import administration
from caosdb.common.datatype import (BOOLEAN, DATETIME, DOUBLE, FILE, INTEGER,
LIST, REFERENCE, TEXT)
from caosdb.common.state import State, Transition
# Import of the basic API classes:
from caosdb.common.models import (ACL, ALL, FIX, NONE, OBLIGATORY, RECOMMENDED,
SUGGESTED, Container, DropOffBox, Entity,
File, Info, Message, Permissions, Property,
Query, QueryTemplate, Record, RecordType,
delete, execute_query, get_global_acl,
get_known_permissions, raise_errors)
from caosdb.utils.get_entity import get_entity_by_name, get_entity_by_path, get_entity_by_id
from caosdb.configuration import _read_config_files, configure, get_config
from caosdb.connection.connection import configure_connection, get_connection
from caosdb.exceptions import *
try:
from caosdb.version import version as __version__
except ModuleNotFoundError:
version = "uninstalled"
__version__ = version
_read_config_files()
This diff is collapsed.
# -*- coding: utf-8 -*-
#
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2023 IndiScale GmbH <info@indiscale.com>
# Copyright (C) 2023 Henrik tom Wörden <h.tomwoerden@indiscale.com>
# Copyright (C) 2023 Daniel Hornung <d.hornung@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
""" from linkahead.cached import *
This module provides some cached versions of functions that retrieve Entities from a remote server. from warnings import warn
See also warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead.cached`. Using the"
======== " old name, starting with caosdb, is deprecated."), DeprecationWarning)
- ``cache_initialize(...)`` : Re-initialize the cache.
- ``cache_clear()`` : Clear the cache.
- ``cached_query(query)`` : A cached version of ``execute_query(query)``.
- ``cached_get_entity_by(...)`` : Get an Entity by name, id, ...
"""
from enum import Enum
from functools import lru_cache
from typing import Union
from .utils import get_entity
from .common.models import execute_query, Entity, Container
# roughly 1GB for typical entity sizes
DEFAULT_SIZE = 33333
# This dict cache is solely for filling the real cache manually (e.g. to reuse older query results)
_DUMMY_CACHE = {}
class AccessType(Enum):
"""Different access types for cached queries. Needed for filling the cache manually with
:func:`cache_fill` .
"""
QUERY = 1
PATH = 2
EID = 3
NAME = 4
def cached_get_entity_by(eid: Union[str, int] = None, name: str = None, path: str = None, query:
str = None) -> Entity:
"""Return a single entity that is identified uniquely by one argument.
You must supply exactly one argument.
If a query phrase is given, the result must be unique. If this is not what you need, use
:func:`cached_query` instead.
"""
count = 0
if eid is not None:
count += 1
if name is not None:
count += 1
if path is not None:
count += 1
if query is not None:
count += 1
if count != 1:
raise ValueError("You must supply exactly one argument.")
if eid is not None:
return _cached_access(AccessType.EID, eid, unique=True)
if name is not None:
return _cached_access(AccessType.NAME, name, unique=True)
if path is not None:
return _cached_access(AccessType.PATH, path, unique=True)
if query is not None:
return _cached_access(AccessType.QUERY, query, unique=True)
raise ValueError("Not all arguments may be None.")
def cached_query(query_string) -> Container:
"""A cached version of :func:`caosdb.execute_query<caosdb.common.models.execute_query>`.
All additional arguments are at their default values.
"""
return _cached_access(AccessType.QUERY, query_string, unique=False)
@lru_cache(maxsize=DEFAULT_SIZE)
def _cached_access(kind: AccessType, value: Union[str, int], unique=True):
# This is the function that is actually cached.
# Due to the arguments, the cache has kind of separate sections for cached_query and
# cached_get_entity_by with the different AccessTypes. However, there is only one cache size.
# The dummy dict cache is only for filling the cache manually, it is deleted afterwards.
if value in _DUMMY_CACHE:
return _DUMMY_CACHE[value]
if kind == AccessType.QUERY:
return execute_query(value, unique=unique)
if kind == AccessType.NAME:
return get_entity.get_entity_by_name(value)
if kind == AccessType.EID:
return get_entity.get_entity_by_id(value)
if kind == AccessType.PATH:
return get_entity.get_entity_by_path(value)
raise ValueError(f"Unknown AccessType: {kind}")
def cache_clear() -> None:
"""Empty the cache that is used by `cached_query` and `cached_get_entity_by`."""
_cached_access.cache_clear()
def cache_info():
"""Return info about the cache that is used by `cached_query` and `cached_get_entity_by`.
Returns
-------
out: named tuple
See the standard library :func:`functools.lru_cache` for details."""
return _cached_access.cache_info()
def cache_initialize(maxsize=DEFAULT_SIZE) -> None:
"""Create a new cache with the given size for `cached_query` and `cached_get_entity_by`.
This implies a call of :func:`cache_clear`, the old cache is emptied.
"""
cache_clear()
global _cached_access
_cached_access = lru_cache(maxsize=maxsize)(_cached_access.__wrapped__)
def cache_fill(items: dict, kind: AccessType = AccessType.EID, unique: bool = True) -> None:
"""Add entries to the cache manually.
This allows to fill the cache without actually submitting queries. Note that this does not
overwrite existing entries with the same keys.
Parameters
----------
items: dict
A dictionary with the entries to go into the cache. The keys must be compatible with the
AccessType given in ``kind``
kind: AccessType, optional
The AccessType, for example ID, name, path or query.
unique: bool, optional
If True, fills the cache for :func:`cached_get_entity_by`, presumably with
:class:`caosdb.Entity<caosdb.common.models.Entity>` objects. If False, the cache should be filled
with :class:`caosdb.Container<caosdb.common.models.Container>` objects, for use with
:func:`cached_query`.
"""
# 1. add the given items to the corresponding dummy dict cache
_DUMMY_CACHE.update(items)
# 2. call the cache function with each key (this only results in a dict look up)
for key in items.keys():
_cached_access(kind, key, unique=unique)
# 3. empty the dummy dict cache again
_DUMMY_CACHE.clear()
"""Commonly used classes for CaosDB."""
from linkahead.common import *
from warnings import warn
warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead.common`. Using the"
" old name, starting with caosdb, is deprecated."), DeprecationWarning)
# -*- coding: utf-8 -*-
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen
# Copyright (C) 2020 Timm Fitschen <t.fitschen@indiscale.com>
# Copyright (C) 2020 IndiScale GmbH <info@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
"""missing docstring.""" from linkahead.common.administration import *
from warnings import warn
import re warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead.common.administration`. Using the"
import string " old name, starting with caosdb, is deprecated."), DeprecationWarning)
import random
from caosdb.common.utils import xml2str
from caosdb.connection.connection import get_connection
from caosdb.exceptions import (EntityDoesNotExistError, HTTPClientError,
HTTPForbiddenError, HTTPResourceNotFoundError,
ServerConfigurationException)
from lxml import etree
def set_server_property(key, value):
"""set_server_property.
Set a server property.
Parameters
----------
key : str
The name of the server property.
value : str
The value of the server property.
Returns
-------
None
"""
con = get_connection()
try:
con._form_data_request(method="POST", path="_server_properties",
params={key: value}).read()
except EntityDoesNotExistError:
raise ServerConfigurationException(
"Debug mode in server is probably disabled.") from None
def get_server_properties():
"""get_server_properties.
Get all server properties as a dict.
Returns
-------
dict
The server properties.
"""
con = get_connection()
try:
body = con._http_request(
method="GET", path="_server_properties")
except EntityDoesNotExistError:
raise ServerConfigurationException(
"Debug mode in server is probably disabled.") from None
xml = etree.parse(body)
props = dict()
for elem in xml.getroot():
props[elem.tag] = elem.text
return props
def get_server_property(key):
"""get_server_property.
Get a server property.
Parameters
----------
key : str
The name of the server property
Returns
-------
value : str
The string value of the server property.
Raises
------
KeyError
If the server property is no defined.
"""
return get_server_properties()[key]
def generate_password(length: int):
"""Create a random password that fulfills the security requirements
Parameters
----------
length : int
Length of the generated password. Has to be greater than 7.
Returns
-------
password : string
Generated random password of the given length
Raises
------
ValueError:
If the length is less than 8.
"""
minimum_password_length = 8
if length < minimum_password_length:
raise ValueError("CaosDB passwords have to be at least {} characters.".format(
minimum_password_length))
sample_letters = string.ascii_letters + string.digits + "!#$%*+-/:;?_"
password = ''.join((random.choice(sample_letters) for i in range(length)))
while not re.match(r"(?=.*[A-Z])(?=.*[a-z])(?=.*\d)(?=.*[\W_]).{8,}",
password):
password = ''.join((random.choice(sample_letters)
for i in range(length)))
return password
def _retrieve_user(name, realm=None, **kwargs):
con = get_connection()
try:
return con._http_request(method="GET", path="User/" + (realm + "/" + name if realm is not None else name), **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to retrieve this user."
raise
except HTTPResourceNotFoundError as e:
e.msg = "User does not exist."
raise
def _delete_user(name, **kwargs):
con = get_connection()
try:
return con._http_request(method="DELETE", path="User/" + name, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to delete this user."
raise
except HTTPResourceNotFoundError as e:
e.msg = "User does not exist."
raise
def _update_user(name, realm=None, password=None, status=None,
email=None, entity=None, **kwargs):
con = get_connection()
params = {}
if password is not None:
params["password"] = password
if status is not None:
params["status"] = status
if email is not None:
params["email"] = email
if entity is not None:
params["entity"] = str(entity)
try:
return con.put_form_data(entity_uri_segment="User/" + (realm + "/" + name if realm is not None else name), params=params, **kwargs).read()
except HTTPResourceNotFoundError as e:
e.msg = "User does not exist."
raise e
except HTTPForbiddenError as e:
e.msg = "You are not permitted to update this user."
raise e
except HTTPClientError as e:
for elem in etree.fromstring(e.body):
if elem.tag == "Error":
e.msg = elem.get("description")
raise
def _insert_user(name, password=None, status=None, email=None, entity=None, **kwargs):
con = get_connection()
params = {"username": name}
if password is not None:
params["password"] = password
if status is not None:
params["status"] = status
if email is not None:
params["email"] = email
if entity is not None:
params["entity"] = entity
try:
return con.post_form_data(entity_uri_segment="User", params=params, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to insert a new user."
raise e
except HTTPClientError as e:
for elem in etree.fromstring(e.body):
if elem.tag == "Error":
e.msg = elem.get("description")
raise e
def _insert_role(name, description, **kwargs):
con = get_connection()
try:
return con.post_form_data(entity_uri_segment="Role", params={"role_name": name, "role_description": description}, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to insert a new role."
raise
except HTTPClientError as e:
if e.status == 409:
e.msg = "Role name is already in use. Choose a different name."
raise
def _update_role(name, description, **kwargs):
con = get_connection()
try:
return con.put_form_data(entity_uri_segment="Role/" + name, params={"role_description": description}, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to update this role."
raise
except HTTPResourceNotFoundError as e:
e.msg = "Role does not exist."
raise
def _retrieve_role(name, **kwargs):
con = get_connection()
try:
return con._http_request(method="GET", path="Role/" + name, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to retrieve this role."
raise
except HTTPResourceNotFoundError as e:
e.msg = "Role does not exist."
raise
def _delete_role(name, **kwargs):
con = get_connection()
try:
return con._http_request(method="DELETE", path="Role/" + name, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to delete this role."
raise
except HTTPResourceNotFoundError as e:
e.msg = "Role does not exist."
raise
def _set_roles(username, roles, realm=None, **kwargs):
xml = etree.Element("Roles")
for r in roles:
xml.append(etree.Element("Role", name=r))
body = xml2str(xml)
con = get_connection()
try:
body = con._http_request(method="PUT", path="UserRoles/" + (realm + "/" +
username if realm is not None else username), body=body, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to set this user's roles."
raise
except HTTPResourceNotFoundError as e:
e.msg = "User does not exist."
raise
except HTTPClientError as e:
if e.status == 409:
e.msg = "Role does not exist."
raise
ret = set()
for r in etree.fromstring(body)[0]:
if r.tag == "Role":
ret.add(r.get("name"))
return ret
def _get_roles(username, realm=None, **kwargs):
con = get_connection()
try:
body = con._http_request(method="GET", path="UserRoles/" + (
realm + "/" + username if realm is not None else username), **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to retrieve this user's roles."
raise
except HTTPResourceNotFoundError as e:
e.msg = "User does not exist."
raise
ret = set()
for r in etree.fromstring(body).xpath('/Response/Roles')[0]:
if r.tag == "Role":
ret.add(r.get("name"))
return ret
def _set_permissions(role, permission_rules, **kwargs):
"""Set permissions for a role.
Parameters
----------
role : str
The role for which the permissions are set.
permission_rules : iterable<PermissionRule>
An iterable with PermissionRule objects.
**kwargs :
Additional arguments which are passed to the HTTP request.
Returns
-------
None
"""
xml = etree.Element("PermissionRules")
for p in permission_rules:
xml.append(p._to_xml())
body = xml2str(xml)
con = get_connection()
try:
return con._http_request(method="PUT", path="PermissionRules/" + role, body=body, **kwargs).read()
except HTTPForbiddenError as e:
e.msg = "You are not permitted to set this role's permissions."
raise
except HTTPResourceNotFoundError as e:
e.msg = "Role does not exist."
raise
def _get_permissions(role, **kwargs):
con = get_connection()
try:
return PermissionRule._parse_body(con._http_request(method="GET", path="PermissionRules/" + role, **kwargs).read())
except HTTPForbiddenError as e:
e.msg = "You are not permitted to retrieve this role's permissions."
raise
except HTTPResourceNotFoundError as e:
e.msg = "Role does not exist."
raise
class PermissionRule():
"""Permission rules.
Parameters
----------
action : str
Either "grant" or "deny"
permission : str
For example ``RETRIEVE:*``.
priority : bool, optional
Whether the priority shall be set, defaults is False.
"""
@staticmethod
def _parse_boolean(bstr):
return str(bstr) in ["True", "true", "TRUE", "yes"]
def __init__(self, action, permission, priority=False):
self._action = action
self._permission = permission
self._priority = PermissionRule._parse_boolean(priority)
def _to_xml(self):
xml = etree.Element(self._action)
xml.set("permission", self._permission)
if self._priority is True:
xml.set("priority", "true")
return xml
@staticmethod
def _parse_element(elem):
return PermissionRule(elem.tag, elem.get(
"permission"), elem.get("priority"))
@staticmethod
def _parse_body(body):
xml = etree.fromstring(body)
ret = set()
for c in xml:
if c.tag in ["Grant", "Deny"]:
ret.add(PermissionRule._parse_element(c))
return ret
def __str__(self):
return str(self._action) + "(" + str(self._permission) + ")" + \
("P" if self._priority is True else "")
def __repr__(self):
return str(self)
def __hash__(self):
return hash(str(self).lower())
def __eq__(self, other):
return str(other).lower() == str(self).lower()
# -*- coding: utf-8 -*-
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2020 IndiScale GmbH
# Copyright (C) 2020 Henrik tom Wörden, IndiScale GmbH
# Copyright (C) 2020 Daniel Hornung (d.hornung@indiscale.com)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
import re from linkahead.common.datatype import *
from warnings import warn
from ..exceptions import EmptyUniqueQueryError, QueryNotUniqueError warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead.common.datatype`. Using the"
" old name, starting with caosdb, is deprecated."), DeprecationWarning)
DOUBLE = "DOUBLE"
REFERENCE = "REFERENCE"
TEXT = "TEXT"
DATETIME = "DATETIME"
INTEGER = "INTEGER"
FILE = "FILE"
BOOLEAN = "BOOLEAN"
def LIST(datatype):
if hasattr(datatype, "name"):
datatype = datatype.name
return "LIST<" + str(datatype) + ">"
def get_list_datatype(datatype):
""" returns the datatype of the elements in the list """
if not isinstance(datatype, str):
return None
match = re.match("LIST(<|&lt;)(?P<datatype>.*)(>|&gt;)", datatype)
if match is not None:
return match.group("datatype")
else:
return None
def is_list_datatype(datatype):
""" returns whether the datatype is a list """
return get_list_datatype(datatype) is not None
def is_reference(datatype):
"""Returns whether the value is a reference
FILE and REFERENCE properties are examples, but also datatypes that are
RecordTypes.
Parameters
----------
datatype : str
The datatype to check.
Returns
-------
bool
True if the datatype is a not base datatype or a list of a base datatype.
Otherwise False is returned.
"""
if datatype is None:
raise ValueError("Cannot decide whether datatype is reference if None"
" is supplied")
if datatype in [DOUBLE, BOOLEAN, INTEGER, TEXT, DATETIME]:
return False
elif is_list_datatype(datatype):
return is_reference(get_list_datatype(datatype))
else:
return True
def get_referenced_recordtype(datatype):
"""Return the record type of the referenced datatype.
Raises
------
ValueError
In cases where datatype is not a reference, the list does not have
a referenced record type or the datatype is a FILE.
Parameters
----------
datatype : str
The datatype to check.
Returns
-------
str
String containing the name of the referenced datatype.
"""
if not is_reference(datatype):
raise ValueError("datatype must be a reference")
if is_list_datatype(datatype):
datatype = get_list_datatype(datatype)
if datatype is None:
raise ValueError("list does not have a list datatype")
if datatype == FILE:
raise ValueError(
"FILE references are not considered references with a record type")
return datatype
def get_id_of_datatype(datatype):
""" returns the id of a Record Type
This is not trivial, as queries may also return children. A check comparing
names is necessary.
Parameters
----------
datatype : string
A datatype, e.g. DOUBLE, or LIST<Person>
Returns
-------
The id of the RecordType with the same name as the datatype.
Raises
------
QueryNotUniqueError
If there are more than one entities with the same name as the datatype.
EmptyUniqueQueryError
If there is no entity with the name of the datatype.
"""
from caosdb import execute_query
if is_list_datatype(datatype):
datatype = get_list_datatype(datatype)
q = "FIND RECORDTYPE {}".format(datatype)
# we cannot use unique=True here, because there might be subtypes
res = execute_query(q)
res = [el for el in res if el.name.lower() == datatype.lower()]
if len(res) > 1:
raise QueryNotUniqueError(
"Name {} did not lead to unique result; Missing "
"implementation".format(datatype))
elif len(res) != 1:
raise EmptyUniqueQueryError(
"No RecordType named {}".format(datatype))
return res[0].id
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment