Merge branch 'dev' into f-better-macro-doc

CHANGELOG.md

@@ -5,7 +5,22 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [Unreleased] ##
+
+### Added ###
+
+### Changed ###
+
+### Deprecated ###
+
+### Removed ###
+
+### Fixed ###
+
+### Security ###
+
+## [0.1.0] - 2022-10-11
+(Florian Spreckelsen)
 
 ### Added
 
@@ -23,10 +38,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 * MAINT: Renamed module from `newcrawler` to `caoscrawler`
 * MAINT: Removed global converters from `crawl.py`
 
-### Deprecated
-
-### Removed
-
 ### Fixed
 
 * FIX: #12
@@ -37,6 +48,3 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   hints to be compatible to python 3.8
 * [#9](https://gitlab.com/caosdb/caosdb-crawler/-/issues/9): Sclaras of types
   different than string can now be given in cfood definitions
-
-
-### Security

README.md

@@ -38,9 +38,9 @@ The original authors of this package are:
 
 # License
 
-Copyright (C) 2021 Research Group Biomedical Physics, Max Planck Institute for
+Copyright (C) 2021-2022 Research Group Biomedical Physics, Max Planck Institute for
 Dynamics and Self-Organization Göttingen.
-Copyright (C) 2021 IndiScale GmbH
+Copyright (C) 2021-2022 IndiScale GmbH
 
 All files in this repository are licensed under a [GNU Affero General Public
 License](LICENCE) (version 3 or later).

RELEASE_GUIDELINES.md

+# Release Guidelines for the CaosDB Python Client Library
+
+This document specifies release guidelines in addition to the general release
+guidelines of the CaosDB Project
+([RELEASE_GUIDELINES.md](https://gitlab.com/caosdb/caosdb/blob/dev/RELEASE_GUIDELINES.md))
+
+## General Prerequisites
+
+* All tests are passing.
+* FEATURES.md is up-to-date and a public API is being declared in that document.
+* CHANGELOG.md is up-to-date.
+* dependencies in `setup.cfg` are up-to-date.
+
+## Steps
+
+1. Create a release branch from the dev branch. This prevents further changes
+   to the code base and a never ending release process. Naming: `release-<VERSION>`
+
+2. Update CHANGELOG.md
+
+3. Check all general prerequisites.
+
+4. Update the version:
+   - `version` variables in `src/doc/conf.py`
+   - Version in [setup.cfg](./setup.cfg): Check the `MAJOR`, `MINOR`, `MICRO`, `PRE` variables and set
+     `ISRELEASED` to `True`. Use the possibility to issue pre-release versions for testing.
+
+5. Merge the release branch into the main branch.
+
+6. Tag the latest commit of the main branch with `v<VERSION>`.
+
+7. Delete the release branch.
+
+8. Remove possibly existing `./dist` directory with old release.
+
+9. Publish the release by executing `./release.sh` with uploads the caosdb
+   module to the Python Package Index [pypi.org](https://pypi.org).
+
+10. Merge the main branch back into the dev branch.
+
+11. After the merge of main to dev, start a new development version by
+    increasing at least the micro version in [setup.cfg](./setup.cfg) and
+    preparing CHANGELOG.md.
+
+12. Create releases on gitlab.com and gitlab.indiscale.com that contain (at
+    least) the most recent section of the CHANGELOG as the description and link
+    to the PyPi package.

integrationtests/test_use_case_simple_presentation.py

@@ -42,13 +42,15 @@ DATADIR = os.path.join(os.path.dirname(__file__), "test_data",
                        "extroot", "use_case_simple_presentation")
 
 # TODO: remove this
-@pytest.fixture
-def clear_database():
-    # TODO(fspreck): Remove once the corresponding advancedtools function can be
-    # used.
-    ents = db.execute_query("FIND ENTITY WITH ID>99")
-    if ents:
-        ents.delete()
+
+
+@pytest.fixture
+def clear_database():
+    # TODO(fspreck): Remove once the corresponding advancedtools function can be
+    # used.
+    ents = db.execute_query("FIND ENTITY WITH ID>99")
+    if ents:
+        ents.delete()
 
 
 def test_complete_crawler(

release.sh

+#!/bin/bash
+rm -rf dist/ build/ .eggs/
+python setup.py sdist bdist_wheel
+python -m twine upload -s dist/*

setup.cfg

 [metadata]
 name = caoscrawler
-version = 0.1
+version = 0.1.1
 author = Alexander Schlemmer
 author_email = alexander.schlemmer@ds.mpg.de
 description = A new crawler for caosdb
@@ -10,19 +10,19 @@ long_description_content_type = text/markdown
 # project_urls
 classifiers =
             Programming Language :: Python :: 3
-            License :: OSI Approved :: AGPLv3
+            License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
             Operating System :: OS Independent
 
 [options]
 package_dir =
             = src
 packages = find:
-python_requires = >=3.6
+python_requires = >=3.8
 install_requires =
 	importlib-resources
 	caosdb
-	caosadvancedtools
-    yaml-header-tools>=0.2.1
+	caosadvancedtools >= 0.6.0
+    yaml-header-tools >= 0.2.1
     pyyaml
     odfpy #make optional
     pandas

src/caoscrawler/converters.py

@@ -218,15 +218,15 @@ def create_records(values: GeneralStore,
             keys_modified.append((name, key))
             propvalue, collection_mode = handle_value(value, values)
 
-            if key in SPECIAL_PROPERTIES:
+            if key.lower() in SPECIAL_PROPERTIES:
                 # e.g. description, name, etc.
                 # list mode does not work for them
-                if key == "path" and not propvalue.startswith(os.path.sep):
+                if key.lower() == "path" and not propvalue.startswith(os.path.sep):
                     propvalue = os.path.sep + propvalue
 
                     # Convert relative to absolute paths:
                     propvalue = os.path.normpath(propvalue)
-                setattr(c_record, key, propvalue)
+                setattr(c_record, key.lower(), propvalue)
             else:
 
                 if c_record.get_property(key) is None:

src/doc/macros.rst

@@ -56,6 +56,9 @@ The same version using cfood macros could be defined as follows:
       - name: README
         filename: ^README.md$
 
+The "MarkdownFile" key and its value will be replaced by everything that is 
+given below "definition" in the Macro.
+        
 The expanded version of `ExperimentalData` will look like:
 
 .. _example_files_2_expanded: