Merge branch 'f-doc-structure' into 'dev'

Document server code structure

See merge request !15

.gitlab-ci.yml

@@ -76,20 +76,21 @@ trigger_build:
 
 # Build the sphinx documentation and make it ready for deployment by Gitlab Pages
 # Special job for serving a static website. See https://docs.gitlab.com/ee/ci/yaml/README.html#pages
-pages:
+pages_prepare: &pages_prepare
   tags: [ cached-dind ]
   stage: deploy
   only:
     refs:
       - /^release-.*$/i
-      - master
-    variables:
-      # run pages only on gitlab.com
-      - $CI_SERVER_HOST == "gitlab.com"
   script:
-    - echo "Deploying"
+    - echo "Deploying..."
     - make doc
     - cp -r build/doc/html public
   artifacts:
     paths:
       - public
+pages:
+  <<: *pages_prepare
+  only:
+    refs:
+      - main

CHANGELOG.md

@@ -20,9 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   to determine whether the server's state has changed between queries.
 * Basic caching for queries. The caching is enabled by default and can be
   controlled by the usual "cache" flag.
+* Documentation for the overall server structure.
 * Add `BEFORE`, `AFTER`, `UNTIL`, `SINCE` keywords for query transaction
-  filters.
-
 
 ### Changed
 

README_SETUP.md

@@ -197,7 +197,11 @@ Stand-alone documentation is built using Sphinx: `make doc`
 
 ### Requirements ##
 
+- plantuml
+- recommonmark
 - sphinx
+- sphinx-rtd-theme
+- sphinxcontrib-plantuml
 - javasphinx :: `pip3 install --user javasphinx`
   - Alternative, if javasphinx fails because python3-sphinx is too recent:
     (`l_` not found):

src/doc/CaosDB-Query-Language.md

@@ -121,7 +121,7 @@ Examples:
 ##### `d1>d2`: Transitive, non-symmetric relation.
 Semantics depend on the flavors of d1 and d2. If both are... 
 ###### [UTCDateTime](Datatype#datetime) 
-* ''True'' iff the time of d1 is after the the time of d2 according to [https://en.wikipedia.org/wiki/Coordinated_Universal_Time](UTC)
+* ''True'' iff the time of d1 is after the the time of d2 according to [UTC](https://en.wikipedia.org/wiki/Coordinated_Universal_Time).
 * ''False'' otherwise.
 
 ###### [SemiCompleteDateTime](Datatype#datetime)

src/doc/Permissions.rst

-#Permissions
+Permissions
+===========
 
 CaosDB has a fine grained role based permission system. Each interaction
 with the server is governed by the current rules of the user, by default

src/doc/administration/configuration.rst

@@ -19,18 +19,21 @@ In each of these directories, the server looks for the following files:
 
 ``server.conf``
    General server configuration options.  The possible configuration options are documented inside
-   the `default file <https://gitlab.com/caosdb/caosdb-server/-/blob/dev/conf/core/server.conf>`_.
+   the `default file
+   <https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/blob/dev/conf/core/server.conf>`__.
 
 ``global_entity_permissions.xml``
-   :ref:`Permissions <concepts:Permissions>` which are automatically set, based on user roles.
-   See the `default file <https://gitlab.com/caosdb/caosdb-server/-/blob/dev/conf/core/global_entity_permissions.xml>`_.
+   :doc:`Permissions<../Permissions>` which are automatically set, based on user roles.  See the
+   `default file
+   <https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/blob/dev/conf/core/global_entity_permissions.xml>`__.
 
 ``usersources.ini``
    This file defines possible sources which are checked when a user tries to authenticate.  Each
    defined source has a special section, the possible options are defined separately for each user
    source.  At the moment the best place to look for this specific documentation is at the API
    documentation of :java:type:`UserSource` and its implementing classes.  The provided `template
-   file <https://gitlab.com/caosdb/caosdb-server/-/blob/dev/conf/core/usersources.ini.template>`_
+   file
+   <https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/blob/dev/conf/core/usersources.ini.template>`__
    also has some information.  The general concept about authentication realms is described in
    :java:type:`UserSources`.
 
@@ -38,13 +41,13 @@ In each of these directories, the server looks for the following files:
    Configuration for dispensed authentication tokens, which can be used to authenticate to CaosDB
    without the need of a user/password combination.  Possible use cases are server-side scripts or
    initial setup after the server start.  There is more documentation inside the `template file
-   <https://gitlab.com/caosdb/caosdb-server/-/blob/dev/conf/core/authtoken.example.yaml>`_.
+   <https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/blob/dev/conf/core/authtoken.example.yaml>`__.
 
 ``cache.ccf``
    Configuration for the Java Caching System (JCS) which can be used by the server.  More
    documentation is `upstream
    <http://commons.apache.org/proper/commons-jcs/getting_started/intro.html>`_ and inside `the file
-   <https://gitlab.com/caosdb/caosdb-server/-/blob/dev/conf/core/cache.ccf>`_.
+   <https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/blob/dev/conf/core/cache.ccf>`_.
 
 ``log4j2-default.properties``, ``log4j2-debug.properties``
    Configuration for logging, following the standard described by the `log4j library

src/doc/administration/maintenance.rst

@@ -21,12 +21,17 @@ The command could look like::
     tar czvf /path/to/new/backup /path/to/caosdb/filesystem.tar.gz
 
 
-You can also save the content of CaosDB using XML. This is **not recommended** since it is less reliable than a real SQL backup. However there may be cases in which an XML backup is desirable, e.g., when transferring entities between two different CaosDB instances.
-Collect the entities that you want to export in a :any:`caosdb-pylib:caosdb.common.models.Container`, here 
-named ``cont``. Then you can export the XML with::
+You can also save the content of CaosDB using XML. This is **not recommended** since it produces
+less reproducible results than a plain SQL backup. However there may be cases in which an XML backup
+is necessary, e.g., when transferring entities between two different CaosDB instances.
+
+Collect the entities that you want to export in a
+:any:`Container<caosdb-pylib:caosdb.common.models.Container>`, named ``cont`` here. Then you can
+export the XML with::
 
      from caosadvancedtools.export_related import invert_ids
      from lxml import etree
+
      invert_ids(cont)
      xml = etree.tounicode(cont.to_xml(
          local_serialization=True), pretty_print=True)
@@ -38,11 +43,11 @@ named ``cont``. Then you can export the XML with::
 Restoring a Backup
 ------------------
 
-.. note :
+.. warning::
     CaosDB should be offline before restoring data.
 
 If you want to restore the internal file system, simply replace it. E.g. if your
-Backup is a tarball::
+backup is a tarball::
 
     tar xvf /path/to/caosroot.tar.gz
 
@@ -52,16 +57,17 @@ You find the documentation on how to restore the data in the SQL-Backend  :any:`
 
 If you want to restore the entities exported to XML, you can do::
 
-     cont = db.Container()                                                                                                                                
-     with open("caosdb_data.xml") as fi:                                                                                                                           
-         cont = cont.from_xml(fi.read())                                                                                                                  
+     cont = db.Container()
+     with open("caosdb_data.xml") as fi:
+         cont = cont.from_xml(fi.read())
      cont.insert()
-                                            
+
 User Management
 ---------------
-The configuration of authentication mechanisms is done via the 
-``usersources.ini`` file (see :any:`configuration`).
 
-We recommend the Python tools (:any:`caosdb-pylib:Administration`) for further administrative tasks (e.g. setting
-user passwords).
+The configuration of authentication mechanisms is done via the ``usersources.ini`` file (see
+:any:`configuration`).
+
+We recommend the Python tools (:any:`caosdb-pylib:administration`) for further administrative tasks
+(e.g. setting user passwords).
 

src/doc/administration/server_side_scripting.rst

@@ -31,7 +31,7 @@ The script has to be executable and must be placed somewhere in one of the direc
 
 Users will need the ``SCRIPTING:EXECUTE:path:to:the:script`` permission. Here the path to the script is of course relativet to the ``SERVER_SIDE_SCRIPTING_BIN_DIRS`` where it is located.
 
-For more information see the :doc:`specification of the API <../specs/Server-side-scripting>`
+For more information see the :doc:`specification of the API <../specification/Server-side-scripting>`
 
 Environment
 ------------

src/doc/conf.py

@@ -47,6 +47,7 @@ extensions = [
     "recommonmark",            # For markdown files.
     "sphinx.ext.autosectionlabel",  # Allow reference sections using its title
     "sphinx_rtd_theme",
+    "sphinxcontrib.plantuml",  # PlantUML diagrams
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -190,14 +191,18 @@ epub_exclude_files = ['search.html']
 #     '<namespace_here>' : ('<base_url_here>', 'javadoc'),
 # }
 
+javadoc_url_map = {
+    'org.restlet': ('https://javadocs.restlet.talend.com/2.4/jse/api', 'javadoc'),
+}
+
 
 # -- Options for intersphinx -------------------------------------------------
 
 # https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#confval-intersphinx_mapping
 intersphinx_mapping = {
     "python": ("https://docs.python.org/", None),
-    "caosdb-pylib": ("https://caosdb.gitlab.io/caosdb-pylib/", None),
-    "caosdb-mysqlbackend": ("https://caosdb.gitlab.io/caosdb-mysqlbackend/", None),
+    "caosdb-pylib": ("https://docs.indiscale.com/caosdb-pylib/", None),
+    "caosdb-mysqlbackend": ("https://docs.indiscale.com/caosdb-mysqlbackend/", None),
 }
 
 

src/doc/development/benchmarking.md

+# Benchmarking CaosDB #
+
+Please refer to the file `doc/devel/Benchmarking.md` in the CaosDB sources for developer resources
+how to do benchmarking and profiling of CaosDB.

src/doc/development/devel.rst

+Developing CaosDB
+=================
+
+.. toctree::
+   :glob:
+   :maxdepth: 2
+
+   Structure of the Java code <structure>
+   Benchmarking CaosDB <benchmarking>
+
+CaosDB is an Open-Source project, so anyone may modify the source as they like. These pages aim to
+provide some help for all CaosDB developers.
+
+More generally, these are the most common ways to contribute to CaosDB:
+
+- You found a bug, have a question, or want to request a feature? Please `create an issue
+  <https://gitlab.com/caosdb/caosdb-server/-/issues>`_.
+- You want to contribute code? Please fork the repository and create a merge request in GitLab and
+  choose this repository as target. Make sure to select "Allow commits from members who can merge
+  the target branch" under Contribution when creating the merge request. This allows our team to
+  work with you on your request.
+- If you have a suggestion for this `documentation <https://docs.indiscale.com/caosdb-server/>`_,
+  the preferred way is also a merge request as describe above (the documentation resides in
+  ``src/doc``).  However, you can also create an issue for it.
+- You can also contact the developers at *info (AT) caosdb.de*.

src/doc/development/structure.rst

+CaosDB's Internal Structure
+===========================
+
+The CaosDB server
+
+- builds upon the `Restlet <https://restlet.talend.com/>`_ framework to provide a REST interface to
+  the network.  See the :ref:`HTTP Resources` section for more information.
+- uses an SQL database (MariaDB or MySQL) as the backend for data storage.  This is documented in
+  the :ref:`MySQL Backend` section.
+- has an internal scheduling framework to organize the required backend jobs.  Read more on this in
+  the :ref:`Transactions and Schedules<transactions>` section.
+- may use a number of authentication providers. Documentation for this is still to come.
+
+.. _HTTP Resources:
+
+HTTP Resources
+--------------
+
+HTTP resources are implemented in the :java:ref:`resource<org.caosdb.server.resource>` package, in
+classes inheriting from :java:ref:`AbstractCaosDBServerResource` (which inherits Restlet's
+:java:ref:`Resource<org.restlet.resource.Resource>` class).  The main :java:ref:`CaosDBServer` class
+defines which HTTP resource (for example ``/Entity/{specifier}``) will be handled by which class
+(:java:ref:`EntityResource` in this case).
+
+Implementing classes need to overwrite for example the ``httpGetInChildClass()`` method (or methods
+corresponding to other HTTP request methods such as POST, PUT, ...).  Typically, they might call the
+``execute()`` method of a :java:ref:`Transaction<org.caosdb.server.transaction.Transaction>` object.
+Transactions are explained in detail in the :ref:`Transactions and Schedules<transactions>` section.
+
+.. uml::
+
+  @startuml
+  abstract AbstractCaosDBServerResource {
+    {abstract} httpGetInChildClass()
+    {abstract} httpPostInChildClass()
+    {abstract} ...InChildClass()
+  }
+  abstract RetrieveEntityResource
+  class EntityResource
+  AbstractCaosDBServerResource <|-- RetrieveEntityResource
+  RetrieveEntityResource <|-- EntityResource
+  @enduml
+
+.. _MySQL Backend:
+
+MySQL Backend
+-------------
+
+The MySQL backend in CaosDB may be substituted by other backends, but at the time of writing this
+documentation, only MySQL (MariaDB is used for testing) is implemented.  There are the following
+main packages which handle the backend:
+
+:java:ref:`backend.interfaces<interfaces>`
+    Interfaces which backends may implement.  The main method for most interfaces is
+    ``execute(...)`` with arguments depending on the specific interface, and benchmarking methods
+    (``getBenchmark()`` and ``setTransactionBenchmark(b)`` may also be required.
+
+:java:ref:`backend.implementation.MySQL<MySQL>`
+    MySQL implementations of the interfaces.  Typical "simple" implementations create a prepared SQL
+    statement from the arguments to ``execute(...)`` and send it to the SQL server.  They may also
+    have methods for undoing and cleanup, using an :java:ref:`UndoHandler`.
+
+:java:ref:`backend.transaction<backend.transaction>` classes
+    Subclasses of the abstract :java:ref:`BackendTransaction` which implement the ``execute()``
+    method.  These classes may use specific backend implementations (like for example the MySQL
+    implementations) to interact with the backend database.
+
+For example, the structure when getting an Entity ID by name looks like this:
+
+.. uml::
+
+  @startuml
+  together {
+    abstract BackendTransaction {
+      HashMap impl // stores all implementations
+      {abstract} execute()
+    }
+    note left of BackendTransaction::impl
+      Stores the
+      implementation
+      for each
+      interface."
+    end note
+    package ...backend.interfaces {
+      interface GetIDByNameImpl {
+        {abstract} execute(String name, String role, String limit)
+      }
+    }
+  }
+  together {
+    package ...backend.transaction {
+      class GetIDByName extends BackendTransaction {
+        execute()
+      }
+    }
+    package ...backend.implementation.MySQL {
+      class MySQLGetIDByName implements GetIDByNameImpl {
+        execute(String name, String role, String limit)
+      }
+    }
+  }
+
+  GetIDByName::execute --r-> MySQLGetIDByName
+  @enduml
+
+.. _transactions:
+
+Transactions and Schedules
+--------------------------
+
+In CaosDB, several client requests may be handled concurrently.  This poses no problem as long as
+only read-only requests are processed, but writing transactions need to block other requests.
+Therefore all transactions (between their first and last access) block write transactions other than
+themselves from writing to the backend, while read transactions may happen at any time, except when
+a write transaction actually writes to the backend.
+
+.. note::
+
+  There is a fine distinction between write transactions on the CaosDB server and actually writing
+  to the backend, since even transactions which need only very short write access to the backend may
+  require extensive read access before, for example to check for permissions or to check if the
+  intended write action makes sense (linked entities must exist, they may need to be of the correct
+  RecordType, etc.).
+
+The request handling in CaosDB is organized in the following way:
+
+- HTTP resources usually create a :java:ref:`Transaction` object and call its
+  :java:ref:`Transaction.execute()` method.  Entities are passed to and from the transaction via
+  :java:ref:`TransactionContainers<TransactionContainer>` (basically normal
+  :java:ref:`Containers<Container>`, enriched with some metadata).
+- The Transaction keeps a :java:ref:`Schedule` of related :java:ref:`Jobs<Job>` (each also wrapping
+  a specific Transaction), which may be called at different stages, called
+  :java:ref:`TransactionStages<TransactionStage>`.
+- The Transaction's ``execute()`` method, when called, in turn calls a number of methods for
+  initialization, checks, preparations, cleanup etc.  Additionally the scheduled jobs are executed
+  at their specified stages, for example all jobs scheduled for ``INIT`` are executed immediately
+  after calling ``Transaction.init()``.  Please consult the API documentation for
+  :java:ref:`Transaction.execute()` for details.
+
+  Most importantly, the (abstract) method ``transaction()`` is called by ``execute()``, which in
+  inheriting classes typically interacts with the backend via :java:ref:`execute(BackendTransaction,
+  Access)<Transaction.execute(K t, Access access)>`, which in turn calls the
+  ``BackendTransaction``'s :java:ref:`BackendTransaction.executeTransaction()` method (just a thin
+  wrapper around its ``execute()`` method).
+
+Summarized, the classes are connected like this:
+
+.. uml::
+
+  @startuml
+  hide empty members
+
+  class Container
+  class TransactionContainer extends Container
+
+  abstract Transaction {
+    Schedule schedule
+    TransactionContainer container
+    execute()
+    execute(BackendTransaction t, Access access)\n    // -> t.executeTransaction(t)
+  }
+
+  class Schedule
+  class ScheduledJob
+  abstract Job {
+    TransactionStage stage
+    Transaction transaction
+    execute(BackendTransaction t)\n    // -> transaction.execute(t, transaction.access)
+  }
+
+  Schedule "*" *- ScheduledJob
+  ScheduledJob *- Job
+  Job o--d- Transaction
+
+  TransactionContainer -* Transaction::container
+  Transaction::schedule *- Schedule
+  @enduml
+

src/doc/index.rst

@@ -12,15 +12,15 @@ Welcome to caosdb-server's documentation!
    Concepts <concepts>
    Query Language <CaosDB-Query-Language>
    administration
-   development/*
+   Development <development/devel>
    specification/index.rst
    Glossary
    API documentation<_apidoc/packages>
 
 Welcome to the CaosDB, the flexible semantic data management toolkit!
 
-This documentation helps you to :doc:`get started<getting_started>`, explains the most important
-:doc:`concepts<concepts>` and offers a range of :doc:`tutorials<tutorials>`.
+This documentation helps you to :doc:`get started<README_SETUP>`, explains the most important
+:doc:`concepts<concepts>` and has information if you want to :doc:`develop<development/devel>` CaosDB yourself.
 
 
 Indices and tables

src/doc/roles.md

@@ -10,7 +10,7 @@ users may have the same role, and there may be roles without any users.
 
 The user and their roles are always returned by the server in answers to requests
 and can thus be interpreted and used by clients.  The most important use though
-is [permission](permissions) checking in the server: Access and
+is [permission](Permissions) checking in the server: Access and
 modification of
 entities can be controlled via roles, so that users of a given role are allowed
 or denied certain actions.  Incidentally, the permission to edit the permissions
@@ -32,4 +32,4 @@ There are some special roles, which are automatically assigned to users:
 
 Except for the `anonymous` role, these special roles are not returned by the
 server, but can nevertheless be used to define
-[permissions](permissions.rst).
+[permissions](Permissions).

src/doc/specification/AbstractProperty.md

+.. note ::
+
+  This document has not been updated for a long time. Although it is concerned with the mostly
+  stable API, its content may no longer reflect the actual CaosDB behavior.
+
 # AbstractProperty Specification
 
 ## Introduction
-An `AbstractProperty` is one of the basal [objects of HeartDB](./HeartDBObject).
+An `AbstractProperty` is one of the basal objects of HeartDB.
 An `AbstractProperty` MUST have the following _qualities_ (shortcut in brackets):
 * a persistent id (`id`)
 * an unique name (`name`)
@@ -54,7 +59,7 @@ Any xml representation of an `AbstractProperty` that is retrieved from the Heart
 
         <Property id="$id" name="$name" description="$description" generator="$generator" creator="$creator" created="$created" type="file" />
 '''General Notes:
-* If the called Property does not exist or if the Property called without permission, the HeartDB Server will return an [Error](./Errorcodes).
+* If the called Property does not exist or if the Property called without permission, the HeartDB Server will return an Error.
 
 ### POST AbstractProperty
 Any xml representation of an `AbstractProperty` that is to be posted to the HeartDB server MUST have exactly ONE of the following forms, depending on the `AbstractProperty's` type:
@@ -80,7 +85,7 @@ Any xml representation of an `AbstractProperty` that is to be posted to the Hear
 * The `AbstractProperty's` `id` and timestamp (`created`) will be generated by the HeartDB Server.
 * The `AbstractProperty's` creator will be determined by the HeartDB Server depending on it's policy configuration.
 * Any given attribute beyond these will be *ignored*.
-* If the `<Property/>` tag isn't compliant with these the HeartDB Server will return an [Error](./Errorcodes).
+* If the `<Property/>` tag isn't compliant with these the HeartDB Server will return an Error.
 
 ----
 ## Examples

src/doc/specification/Fileserver.md

@@ -35,7 +35,8 @@ where
 ### HTTP upload stream
 #### Files
 
-There is an example on file upload using cURL described in detail in [the curl section of this wiki](manuals/curl/curl-access).
+There is an example on file upload using cURL described in detail in [the curl section of this
+documentation](../administration/curl-access.md).
 
 File upload via HTTP is implemented in a [rfc1867](http://www.ietf.org/rfc/rfc1867.txt) consistent way. This is a de-facto standard that defines a file upload as a part of an HTML form submission. This concept shall not be amplified here. But it has to be noticed that this protocol is not designed for uploads of complete structured folders. Therefore the HeartDB file components have to impose that structure on the upload protocol. 
 

src/doc/specification/RecordType.md

 # RecordType
-----
+
 ## Overview
 RecordTypes function as templates for [[Record|Records]], they provide a description for a type of Record and define which [[Property|Properties]] should be present.  Properties come with an _importance_ attribute which tells the user or client program how strongly necessary the Property is.  (As all other entities,) RecordTypes can be inherited from other RecordTypes (or any Entities).  When RecordTypes inherit from other RecordTypes, the _inheritance_ flag tells which properties shall be inherited.
 

src/doc/specification/index.rst

@@ -8,15 +8,13 @@ Specification
    :hidden:                                                                     
                                                                                 
    AbstractProperty                                                                  
-   C-Client  
    Fileserver  
    Record      
-   Server-side-scripting       
    Specification of the Entity API <Specification-of-the-Entity-API>
    Authentication    
    Datatype  
    Paging      
    RecordType  
-   Server side scripting <Server-side-scripting-v0.1>
+   Server side scripting <Server-side-scripting>
    Specification of the Message API <Specification-of-the-Message-API>
 

src/main/java/org/caosdb/server/database/BackendTransaction.java

@@ -145,6 +145,7 @@ public abstract class BackendTransaction implements Undoable {
 
   protected abstract void execute();
 
+  /** Like execute(), but with benchmarking measurement. */
   public final void executeTransaction() {
     final long t1 = System.currentTimeMillis();
     execute();

src/main/java/org/caosdb/server/jobs/Job.java

@@ -60,13 +60,18 @@ import org.reflections.Reflections;
  * @todo Describe me.
  */
 public abstract class Job {
+  /** All known Job classes, by name (actually lowercase getSimpleName()). */
+  static HashMap<String, Class<? extends Job>> allClasses = null;
+
+  private static List<Class<? extends Job>> loadAlways;
+
   private Transaction<? extends TransactionContainer> transaction = null;
   private Mode mode = null;
+  private final TransactionStage stage;
+  private EntityInterface entity = null;
 
   public abstract JobTarget getTarget();
 
-  private EntityInterface entity = null;
-
   protected <S, T> HashMap<S, T> getCache(final String name) {
     return getTransaction().getCache(name);
   }
@@ -112,9 +117,9 @@ public abstract class Job {
 
   protected Job() {
     if (this.getClass().isAnnotationPresent(JobAnnotation.class)) {
-      this.time = this.getClass().getAnnotation(JobAnnotation.class).time();
+      this.stage = this.getClass().getAnnotation(JobAnnotation.class).stage();
     } else {
-      this.time = JobExecutionTime.CHECK;
+      this.stage = TransactionStage.CHECK;
     }
   }
 
@@ -230,22 +235,33 @@ public abstract class Job {
     }
   }
 
-  static HashMap<String, Class<? extends Job>> allClasses = null;
-  private static List<Class<? extends Job>> loadAlways;
-
+  /**
+   * Create a Job object with the given parameters.
+   *
+   * <p>This static method is used by other classes to create Job objects, instead of the private
+   * constructor.
+   *
+   * @return The generated Job object.
+   */
   public static Job getJob(
       final String job,
       final Mode mode,
       final EntityInterface entity,
       final Transaction<? extends TransactionContainer> transaction) {
+    // Fill `allClasses` with available subclasses
     scanJobClasspath();
 
+    // Get matching class for Job and generate it.
     final Class<? extends Job> jobClass = allClasses.get(job.toLowerCase());
     return getJob(jobClass, mode, entity, transaction);
   }
 
+  /**
+   * Initialize {@code allClasses} with all {@code Job} classes found in the classpath.
+   *
+   * @todo Details when this has any effect.
+   */
   private static void scanJobClasspath() {
-
     if (allClasses == null || loadAlways == null) {
       allClasses = new HashMap<>();
       loadAlways = new ArrayList<>();
@@ -452,15 +468,18 @@ public abstract class Job {
     System.out.println(toString());
   }
 
-  private final JobExecutionTime time;
-
-  public JobExecutionTime getExecutionTime() {
-    return this.time;
+  public TransactionStage getTransactionStage() {
+    return this.stage;
   }
 
+  /**
+   * Return those matching jobs which are annotated with the "loadAlways" attribute.
+   *
+   * @return A list with the jobs.
+   */
   public static List<Job> loadPermanentContainerJobs(Transaction<?> transaction) {
     final ArrayList<Job> jobs = new ArrayList<>();
-    // load permanent jobs
+    // load permanent jobs: ContainerJob classes with the correct transaction
     for (Class<? extends Job> j : loadAlways) {
       if (ContainerJob.class.isAssignableFrom(j)
           && j.getAnnotation(JobAnnotation.class).transaction().isInstance(transaction)) {