F pipeline python versions

Summary

For #124 (closed). Adds unittest support for all supported Python versions.

Focus

Contrary to our initial plan, I decided to abandon pyenv since it is a pain in the ass to be run within a debian-based pipeline image and rather use the official Python 3.X docker images. These also have their problems with tox (or rather, pip within tox, see, e.g. https://gitlab.indiscale.com/caosdb/src/caosdb-pylib/-/jobs/141105), so plain pytest needs to be used instead.

Test Environment

No manual testing, just verify that the unittest_py3.X jobs actually run on the correct Python version (take a look at the final pipeline output.)

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

.gitlab-ci.yml

@@ -54,7 +54,21 @@ pylint:
   allow_failure: true
 
 # run unit tests
-unittest:
+unittest_py3.8:
+  tags: [ docker ]
+  stage: test
+  needs: [ ]
+  image: python:3.8
+  script: &python_test_script
+    # Python docker has problems with tox and pip so use plain pytest here
+    - touch ~/.pycaosdb.ini
+    - pip install nose pytest pytest-cov python-dateutil jsonschema==4.0.1
+    - pip install .
+    - python -m pytest unittests
+
+# This needs to be changed once Python 3.9 isn't the standard Python in Debian
+# anymore.
+unittest_py3.9:
   tags: [ docker ]
   stage: test
   needs: [ ]
@@ -62,6 +76,14 @@ unittest:
     - touch ~/.pycaosdb.ini
     - make unittest
 
+
+unittest_py3.10:
+  tags: [ docker ]
+  stage: test
+  needs: [ ]
+  image: python:3.10
+  script: *python_test_script
+
 # Trigger building of server image and integration tests
 trigger_build:
   stage: deploy
@@ -88,12 +110,12 @@ build-testenv:
   stage: setup
   only:
       - schedules
-  script: 
+  script:
     - cd unittests/docker
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
       # use here general latest or specific branch latest...
     - docker pull $CI_REGISTRY_IMAGE|| true
-    - docker build 
+    - docker build
       --pull
       --build-arg COMMIT=$CI_COMMIT_SHORT_SHA
       --cache-from $CI_REGISTRY_IMAGE
@@ -105,7 +127,7 @@ build-testenv:
 pages_prepare: &pages_prepare
   tags: [ cached-dind ]
   stage: deploy
-  needs: [ code_style, pylint, unittest ]
+  needs: [ code_style, pylint, unittest_py3.8, unittest_py3.9, unittest_py3.10 ]
   only:
     refs:
       - /^release-.*$/i

[Florian Spreckelsen]

We need to use pytest here instead of tox because pip-installing the prerequisites within tox using a non-system Python fails.

[Florian Spreckelsen]

This can be left unchanged for as long as Python 3.9 and Debian bullseye remain our basis.

[Florian Spreckelsen]

At least the test script is the same for all non-system Pythons (Python 3.11, here we come ;-) )