Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • caosdb/src/caosdb-pylib
1 result
Show changes
Commits on Source (225)
Showing with 362 additions and 108 deletions
......@@ -13,6 +13,7 @@ __pycache__/
dist/
build/
src/caosdb/version.py
src/linkahead/version.py
# documentation
_apidoc
......
#
# This file is a part of the CaosDB Project.
# This file is a part of the LinkAhead Project.
#
# Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen
......@@ -44,7 +44,7 @@ code_style:
- make style
allow_failure: true
# pylint tests for pycaosdb
# pylint tests for pylinkahead
pylint:
tags: [ docker ]
stage: linting
......@@ -54,18 +54,25 @@ pylint:
allow_failure: true
# run unit tests
unittest_py3.8:
unittest_py3.7:
tags: [ docker ]
stage: test
needs: [ ]
image: python:3.8
image: python:3.7
script: &python_test_script
# Python docker has problems with tox and pip so use plain pytest here
- touch ~/.pycaosdb.ini
- pip install nose pytest pytest-cov python-dateutil jsonschema==4.0.1
- touch ~/.pylinkahead.ini
- pip install nose pytest pytest-cov python-dateutil jsonschema>=4.4.0
- pip install .
- python -m pytest unittests
unittest_py3.8:
tags: [ docker ]
stage: test
needs: [ ]
image: python:3.8
script: *python_test_script
# This needs to be changed once Python 3.9 isn't the standard Python in Debian
# anymore.
unittest_py3.9:
......@@ -75,7 +82,7 @@ unittest_py3.9:
script:
# verify that this actually is Python 3.9
- python3 -c "import sys; assert sys.version.startswith('3.9')"
- touch ~/.pycaosdb.ini
- touch ~/.pylinkahead.ini
- make unittest
......@@ -86,6 +93,13 @@ unittest_py3.10:
image: python:3.10
script: *python_test_script
unittest_py3.11:
tags: [ docker ]
stage: test
needs: [ ]
image: python:3.11
script: *python_test_script
# Trigger building of server image and integration tests
trigger_build:
stage: deploy
......
## Summary
*Please give a short summary of what the issue is.*
## Expected Behavior
*What did you expect how the software should behave?*
## Actual Behavior
*What did the software actually do?*
## Steps to Reproduce the Problem
*Please describe, step by step, how others can reproduce the problem. Please try these steps for yourself on a clean system.*
1.
2.
3.
## Specifications
- Version: *Which version of this software?*
- Platform: *Which operating system, which other relevant software versions?*
## Possible fixes
*Do you have ideas how the issue can be resolved?*
# Summary
*Insert a meaningful description for this merge request here: What is the new/changed behavior?
Which bug has been fixed? Are there related issues?*
# Focus
*Point the reviewer to the core of the code change. Where should they start reading? What should
they focus on (e.g. security, performance, maintainability, user-friendliness, compliance with the
specs, finding more corner cases, concrete questions)?*
# Test Environment
*How to set up a test environment for manual testing?*
# Check List for the Author
Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab
comments for the reviewer. They should guide the reviewer through the changes, explain your changes
and also point out open questions. For further good practices have a look at [our review
guidelines](https://gitlab.com/caosdb/caosdb/-/blob/dev/REVIEW_GUIDELINES.md)
- [ ] All automated tests pass
- [ ] Reference related issues
- [ ] Up-to-date CHANGELOG.md (or not necessary)
- [ ] Up-to-date JSON schema (or not necessary)
- [ ] Appropriate user and developer documentation (or not necessary)
- Update / write published documentation (`make doc`).
- How do I use the software? Assume "stupid" users.
- How do I develop or debug the software? Assume novice developers.
- [ ] Annotations in code (Gitlab comments)
- Intent of new code
- Problems with old code
- Why this implementation?
# Check List for the Reviewer
- [ ] I understand the intent of this MR
- [ ] All automated tests pass
- [ ] Up-to-date CHANGELOG.md (or not necessary)
- [ ] Appropriate user and developer documentation (or not necessary), also in published
documentation.
- [ ] The test environment setup works and the intended behavior is reproducible in the test
environment
- [ ] In-code documentation and comments are up-to-date.
- [ ] Check: Are there specifications? Are they satisfied?
For further good practices have a look at [our review guidelines](https://gitlab.com/caosdb/caosdb/-/blob/dev/REVIEW_GUIDELINES.md).
/assign me
/target_branch dev
......@@ -5,18 +5,17 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] ##
## [Unreleased] - Date
### Added ###
* `apiutils.EntityMergeConflictError` class for unresesolvable merge conflicts
when merging two entities
* `apiutils.merge_entities` now has a `merge_id_with_resolved_entity` keyword
which allows to identify property values with each other in case that one is
an id and the other is an Entity with this id. Default is ``False``, so no
change to the default behavior.
### Changed ###
* `apiutils.merge_entities` now raises an `EntityMergeConflictError` in case of
unresolvable merge conflicts.
### Deprecated ###
### Removed ###
......@@ -27,7 +26,112 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Documentation ###
* [Fixed](https://gitlab.com/caosdb/caosdb-pylib/-/issues/79) `{action}_entity_permissions` help line.
## [0.13.2] - 2023-12-15
### Fixed ###
* [#113](https://gitlab.com/linkahead/linkahead-pylib/-/issues/113) Container could fail to delete when there were reference properties.
* HTTP status 431 (Headers too long) now also raises an URI too long exception.
## [0.13.1] - 2023-10-11 ##
### Fixed ###
* no Error when no configuration file is used
[Issue](https://gitlab.com/linkahead/linkahead-pylib/-/issues/107)
## [0.13.0] - 2023-10-10 ##
### Added ###
* New `page_length` parameter for `caosdb.execute_query` and
`caosdb.Query.execute`. See docstrings for more details.
* `Entity.remove_value_from_property` function that removes a given value from a
property and optionally removes the property if it is empty afterwards.
### Changed ###
* `_Messages` is now `Messages` and inherits from list instead of dict
* `Message.__init__` signature changed and `type` defaults to "Info" now.
* `Message.__eq__` changed. Equality is equality of `type`, `code`, and
`description` now.
* Rename from CaosDB to LinkAhead. For proper migration, follow the instructions
in `migration_to_linkahead.md` and check the documentation at [docs.indiscale.com](https://docs.indiscale.com/caosdb-pylib/README_SETUP.html#migration).
### Deprecated ###
* The API of Messages has been simplified and some ways to interact with
messages have been deprecated. Warnings are raised correspondingly.
* `Message.get_code`. Use the `code` property instead.
### Fixed ###
- Detection for cyclic references when converting entites using the high level API.
## [0.12.0] - 2023-06-02 ##
### Added ###
- Added location argument to `src/caosdb/utils/checkFileSystemConsistency.py`
- Entity getters: `get_entity_by_<name/id/path>`
- Cached versions of entity getters and of `execute_query` (`cached_query`)
### Deprecated ###
- getOriginUrlIn, getDiffIn, getBranchIn, getCommitIn (formerly apiutils) have been
moved to caosdb.utils.git_utils
### Fixed ###
- Fixed `src/caosdb/utils/checkFileSystemConsistency.py`
### Documentation ###
* [#83](https://gitlab.com/caosdb/caosdb-pylib/-/issues/83) - Improved
documentation on adding REFERENCE properties, both in the docstring of
`Entity.add_property` and in the data-insertion tutorial.
## [0.11.2] - 2023-03-14 ##
### Fixed ###
- root logger is no longer used to create warnings. Fixes undesired output in
stderr
## [0.11.1] - 2023-03-07 ##
(Florian Spreckelsen)
### Changed ###
* Renamed `caosdb.common.models._Parents` to `caosdb.common.models._ParentList`.
### Fixed ###
* [caosdb-pylib#90](https://gitlab.com/caosdb/caosdb-pylib/-/issues/90): `Entity.get_parents_recursively()` did not work for unretrieved parents.
## [0.11.0] - 2023-01-19 ##
(Florian Spreckelsen)
### Added ###
* `apiutils.EntityMergeConflictError` class for unresesolvable merge conflicts
when merging two entities
* Re-introduced support for Python 3.7
### Changed ###
* `apiutils.merge_entities` now raises an `EntityMergeConflictError` in case of
unresolvable merge conflicts.
### Fixed ###
* [#82](https://gitlab.com/caosdb/caosdb-pylib/-/issues/82) Merging an entity
with properties with missing datatype leads to Exception - The correct
exception is raised in case of a missing LIST datatype.
### Documentation ###
* [Fixed](https://gitlab.com/caosdb/caosdb-pylib/-/issues/79)
`{action}_entity_permissions` help line.
## [0.10.0] - 2022-11-14
(Florian Spreckelsen)
......
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: Fitschen
given-names: Timm
orcid: https://orcid.org/0000-0002-4022-432X
- family-names: Schlemmer
given-names: Alexander
orcid: https://orcid.org/0000-0003-4124-9649
- family-names: Hornung
given-names: Daniel
orcid: https://orcid.org/0000-0002-7846-6375
- family-names: tom Wörden
given-names: Henrik
orcid: https://orcid.org/0000-0002-5549-578X
- family-names: Parlitz
given-names: Ulrich
orcid: https://orcid.org/0000-0003-3058-1435
- family-names: Luther
given-names: Stefan
orcid: https://orcid.org/0000-0001-7214-8125
title: CaosDB - Pylib
version: 0.13.2
doi: 10.3390/data4020083
date-released: 2023-10-11
* caosdb-server >= 0.8.0
* caosdb-server >= 0.12.0
* Python >= 3.8
* pip >= 20.0.2
......
# ** header v3.0
# This file is a part of the CaosDB Project.
# This file is a part of the LinkAhead Project.
#
# Copyright (C) 2020 IndiScale GmbH <info@indiscale.com>
# Copyright (C) 2020 Daniel Hornung <d.hornung@indiscale.com>
......@@ -40,7 +40,7 @@ style:
.PHONY: style
lint:
pylint --unsafe-load-any-extension=y -d all -e E,F src/caosdb/common
pylint --unsafe-load-any-extension=y -d all -e E,F src/linkahead/common
.PHONY: lint
unittest:
......
......@@ -41,7 +41,7 @@ the preferred way is also a merge request as describe above (the documentation r
However, you can also create an issue for it.
* You can also contact us at **info (AT) caosdb.org** and join the
CaosDB community on
[#caosdb:matrix.org](https://matrix.to/#/!unwwlTfOznjEnMMXxf:matrix.org).
[#linkahead:matrix.org](https://matrix.to/#/!unwwlTfOznjEnMMXxf:matrix.org).
## License
......
......@@ -128,3 +128,6 @@ Build documentation in `build/` with `make doc`.
### Troubleshooting ###
If the client is to be executed directly from the `/src` folder, an initial `.\setup.py install --user` must be called.
## Migration ##
TODO
# Release Guidelines for the CaosDB Python Client Library
This document specifies release guidelines in addition to the general release
guidelines of the CaosDB Project
guidelines of the LinkAhead Project
([RELEASE_GUIDELINES.md](https://gitlab.com/caosdb/caosdb/blob/dev/RELEASE_GUIDELINES.md))
## General Prerequisites
......@@ -24,6 +24,7 @@ guidelines of the CaosDB Project
- `version` variables in `src/doc/conf.py`
- Version on [setup.py](./setup.py): Check the `MAJOR`, `MINOR`, `MICRO`, `PRE` variables and set
`ISRELEASED` to `True`. Use the possibility to issue pre-release versions for testing.
- `CITATION.cff` (update version and date)
5. Merge the release branch into the main branch.
......
#!/usr/bin/env python3
# encoding: utf-8
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
# Copyright (C) 2021 Henrik tom Wörden <h.tomwoerden@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
import os
base_root = "src/linkahead/"
initcontent = """
from {module} import *
from warnings import warn
warn(("CaosDB was renamed to LinkAhead. Please import this library as `import {module}`. Using the"
" old name, starting with caosdb, is deprecated."), DeprecationWarning)
"""
for root, dirs, files in os.walk(base_root, topdown=False):
if root.endswith("__pycache__"):
continue
cdir = os.path.join("src/caosdb", root[len(base_root):])
os.makedirs(cdir, exist_ok=True)
for fi in files:
if not fi.endswith(".py"):
continue
path = os.path.join(cdir, fi)
with open(path, 'w') as cur:
if fi == "__init__.py":
cur.write(initcontent.format(module=".".join(
os.path.join(root, fi[:-3]).split('/')[1:-1])))
else:
cur.write(initcontent.format(module=".".join(
os.path.join(root, fi[:-3]).split('/')[1:])))
#!/usr/bin/env python3
"""A small example to get started with caosdb-pylib.
Make sure that a `pycaosdb.ini` is readable at one of the expected locations.
Make sure that a `pylinkahead.ini` is readable at one of the expected locations.
"""
import random
......
# To be found be the caosdb package, the INI file must be located either in
# - $CWD/pycaosdb.ini
# - $HOME/.pycaosdb.ini
# - $CWD/pylinkahead.ini
# - $HOME/.pylinkahead.ini
# - the location given in the env variable PYCAOSDBINI
[Connection]
......
......@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
#
# ** header v3.0
# This file is a part of the CaosDB Project.
# This file is a part of the LinkAhead Project.
#
# Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen
......
#!/usr/bin/env python3
# ** header v3.0
# This file is a part of the CaosDB Project.
# This file is a part of the LinkAhead Project.
#
# Copyright (c) 2019 IndiScale GmbH
# Copyright (c) 2019 Daniel Hornung <d.hornung@indiscale.com>
......@@ -25,7 +25,7 @@
As a result, only a specific user or group may access it.
This script assumes that the user specified in the pycaosdb.ini
This script assumes that the user specified in the pylinkahead.ini
configuration can create new entities.
"""
......
git merge linkahead-rename-step-1
# resolve potential conflicts and commit
rm -rf src/linkahead
git mv src/caosdb/ src/linkahead
rm -rf src/caosdb
python3 create_slim_linkahead_wrapper.py
git add src
git ci -m "MAINT: rename caosdb to linkahead (module)"
git merge linkahead-rename-step-2
# resolve potential conflicts and commit
git merge dev
#!/bin/bash
rm -rf dist/ build/ .eggs/
python setup.py sdist bdist_wheel
python -m twine upload -s dist/*
export PKGNAME=caosdb
python setup.py sdist bdist_wheel
python -m twine upload dist/*
......@@ -2,8 +2,9 @@
# -*- encoding: utf-8 -*-
#
#
"""caosdb"""
"""linkahead"""
import os
import re
import subprocess
import sys
......@@ -47,13 +48,13 @@ from setuptools import find_packages, setup
ISRELEASED = False
MAJOR = 0
MINOR = 10
MICRO = 1
MINOR = 13
MICRO = 3
# Do not tag as pre-release until this commit
# https://github.com/pypa/packaging/pull/515
# has made it into a release. Probably we should wait for pypa/packaging>=21.4
# https://github.com/pypa/packaging/releases
PRE = "" # "dev" # e.g. rc0, alpha.1, 0.beta-23
PRE = "" # "dev" # e.g. rc0, alpha.1, 0.beta-23
if PRE:
VERSION = "{}.{}.{}-{}".format(MAJOR, MINOR, MICRO, PRE)
......@@ -91,25 +92,26 @@ def git_version():
def get_version_info():
# Adding the git rev number needs to be done inside write_version_py(),
# otherwise the import of caosdb.version messes up the build under
# otherwise the import of linkahead.version messes up the build under
# Python 3.
FULLVERSION = VERSION
# Magic which is only really needed in the pipelines. Therefore: a lot of dark pipeline magic.
GIT_REVISION = "Unknown"
if os.path.exists('.git'):
GIT_REVISION = git_version()
elif os.path.exists('caosdb_pylib_commit'):
with open('caosdb_pylib_commit', 'r') as f:
elif os.path.exists('linkahead_pylib_commit'):
with open('linkahead_pylib_commit', 'r') as f:
GIT_REVISION = f.read().strip()
elif os.path.exists('src/caosdb/version.py'):
elif os.path.exists('src/linkahead/version.py'):
# must be a source distribution, use existing version file
try:
from caosdb.version import git_revision as GIT_REVISION
except ImportError:
raise ImportError("Unable to import git_revision. Try removing "
"src/caosdb/version.py and the build directory "
"before building.")
else:
GIT_REVISION = "Unknown"
with open('src/linkahead/version.py') as fi:
rev_pattern = re.compile(r"^git_revision = '(?P<rev>.*)'$")
for line in fi.readlines():
match = rev_pattern.match(line)
if match is not None:
GIT_REVISION = match.group('rev')
break
if not ISRELEASED:
FULLVERSION += '.dev0+' + GIT_REVISION[:7]
......@@ -117,9 +119,9 @@ def get_version_info():
return FULLVERSION, GIT_REVISION
def write_version_py(filename='src/caosdb/version.py'):
def write_version_py(filename='src/linkahead/version.py'):
cnt = """
# THIS FILE IS GENERATED FROM caosdb SETUP.PY
# THIS FILE IS GENERATED FROM linkahead SETUP.PY
#
short_version = '%(version)s'
version = '%(version)s'
......@@ -153,15 +155,21 @@ def setup_package():
# Rewrite the version file everytime
write_version_py()
if 'PKGNAME' in os.environ and os.environ['PKGNAME'] == 'caosdb':
pname = 'caosdb'
pdesc = 'Deprecated! Please install linkahead.'
else:
pname = 'linkahead'
pdesc = 'Python Interface for LinkAhead'
metadata = dict(
name='caosdb',
name=pname,
version=get_version_info()[0],
description='Python Interface for CaosDB',
description=pdesc,
long_description=long_description,
long_description_content_type="text/markdown",
author='Timm Fitschen',
author_email='t.fitschen@indiscale.com',
url='https://www.caosdb.org',
url='https://www.linkahead.org',
license="AGPLv3+",
classifiers=[
"Programming Language :: Python :: 3",
......@@ -171,23 +179,26 @@ def setup_package():
"Topic :: Scientific/Engineering :: Information Analysis",
],
packages=find_packages('src'),
python_requires='>=3.8',
python_requires='>=3.7',
package_dir={'': 'src'},
install_requires=['lxml>=4.6.3',
"requests[socks]>=2.26",
"python-dateutil>=2.8.2",
'PyYAML>=5.4.1',
'future',
],
],
extras_require={'keyring': ['keyring>=13.0.0'],
'jsonschema': ['jsonschema>=4.4.0']},
setup_requires=["pytest-runner>=2.0,<3dev"],
tests_require=["pytest", "pytest-cov", "coverage>=4.4.2",
"jsonschema>=4.4.0"],
package_data={
'caosdb': ['cert/indiscale.ca.crt', 'schema-pycaosdb-ini.yml'],
'linkahead': ['cert/indiscale.ca.crt', 'schema-pycaosdb-ini.yml'],
},
scripts=["src/caosdb/utils/caosdb_admin.py"]
scripts=[
"src/linkahead/utils/caosdb_admin.py",
"src/linkahead/utils/linkahead_admin.py"
]
)
try:
setup(**metadata)
......
# -*- coding: utf-8 -*-
#
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2018 Research Group Biomedical Physics,
# Max-Planck-Institute for Dynamics and Self-Organization Göttingen
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#
# ** end header
#
"""CaosDB Python bindings.
from linkahead import *
from warnings import warn
Tries to read from the inifile specified in the environment variable `PYCAOSDBINI` or alternatively
in `~/.pycaosdb.ini` upon import. After that, the ini file `pycaosdb.ini` in the current working
directory will be read additionally, if it exists.
"""
from os import environ, getcwd
# Import of the connection function (which is used to connect to the DB):
from os.path import expanduser, join
# Import of convenience methods:
import caosdb.apiutils
from caosdb.common import administration
from caosdb.common.datatype import (BOOLEAN, DATETIME, DOUBLE, FILE, INTEGER,
LIST, REFERENCE, TEXT)
from caosdb.common.state import State, Transition
# Import of the basic API classes:
from caosdb.common.models import (ACL, ALL, FIX, NONE, OBLIGATORY, RECOMMENDED,
SUGGESTED, Container, Entity,
File, Info, Message, Permissions, Property,
Query, QueryTemplate, Record, RecordType,
delete, execute_query, get_global_acl,
get_known_permissions, raise_errors,
Directory, Link)
from caosdb.configuration import _read_config_files, configure, get_config
from caosdb.connection.connection import configure_connection, get_connection
from caosdb.exceptions import *
try:
from caosdb.version import version as __version__
except ModuleNotFoundError:
version = "uninstalled"
__version__ = version
_read_config_files()
warn(("CaosDB was renamed to LinkAhead. Please import this library as `import linkahead`. Using the"
" old name, starting with caosdb, is deprecated."), DeprecationWarning)