fix importance bug

Summary

Fix for #146 (closed)

The place where the configuration is being set moved from the back-end (where it has been since the Middle Bronze Age) to the server (conf/core/rules.csv).

Also the Job implementing the check had some serious gaps (even not-too-cornerish cases) and TODOs which were closed/implemented. Effectively, this MR is a major refactoring of the CheckParOblPropPresent job.

Focus

• The RuleLoader and all related transaction classes have been removed. Instead the JobConfig class loads the rules for jobs from a csv file which contains the columns that have been stored to the "rules" table in the back-end previously.

• The Job class was extended by several convenient method I stole from the f-filesystem branch (most prominently resolve and isSubType.

• The Mode enum has been renamed to JobFailureSeverity because that is actually a much more speaking name. Also the values (remainders of the Early Bronze Age) have been changed to more speaking ones (and never-used values have been removed).

• On the go, I also used the new resolve function in some other places and also removed some occurrences where Jobs started other jobs explicitly. I noticed that this was not necessary and it only was there because the CheckParentValid and CheckPropertyValid jobs have been moved to the PRE_CHECK stage in earlier commits, but the then unnecessary explicit calls have never been removed - they did no harm, but they are not necessary.

Test Environment

Tests in caosdb-pyinttest!9 (merged)

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related Issues
• Up-to-date CHANGELOG.md
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there spezifications? Are they satisfied?

For further good practices have a look at our review guidelines.

conf/core/jobs.csv

+#
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Timm Fitsche <t.fitschen@indiscale.com>
+# Copyright (C) 2021 IndiScale GmbH <info@indiscale.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <https://www.gnu.org/licenses/>.
+#
+
+# Five columns:
+# DOMAIN_ID, ENTITY_ID, TRANSACTION, JOB, JOB_FAILURE_SEVERITY
+
+# general rules
+0,0,INSERT,CheckPropValid,ERROR
+0,0,INSERT,CheckParValid,ERROR
+0,0,INSERT,CheckParOblPropPresent,ERROR
+0,0,INSERT,CheckValueParsable,ERROR
+0,0,UPDATE,CheckPropValid,ERROR
+0,0,UPDATE,CheckParValid,ERROR
+0,0,UPDATE,CheckParOblPropPresent,ERROR
+0,0,UPDATE,CheckValueParsable,ERROR
+0,0,DELETE,CheckReferenceDependencyExistent,ERROR
+0,0,DELETE,CheckChildDependencyExistent,ERROR
+
+
+# role specific rules
+## recordtype rules
+0,1,INSERT,CheckNamePresent,ERROR
+0,1,INSERT,CheckPropPresent,WARN
+0,1,INSERT,SetImpToRecByDefault,ERROR
+
+0,1,UPDATE,CheckNamePresent,ERROR
+0,1,UPDATE,CheckPropPresent,WARN
+0,1,UPDATE,SetImpToRecByDefault,ERROR
+
+## record rules
+0,2,INSERT,CheckNamePresent,WARN
+0,2,INSERT,CheckPropPresent,WARN
+0,2,INSERT,CheckParPresent,ERROR
+0,2,INSERT,SetImpToFix,ERROR
+
+0,2,UPDATE,CheckNamePresent,WARN
+0,2,UPDATE,CheckPropPresent,WARN
+0,2,UPDATE,CheckParPresent,ERROR
+0,2,UPDATE,SetImpToFix,ERROR
+
+## file rules
+0,3,INSERT,CheckNamePresent,WARN
+0,3,INSERT,MatchFileProp,ERROR
+0,3,INSERT,CheckTargetPathValid,ERROR
+0,3,INSERT,SetImpToFix,ERROR
+
+0,3,UPDATE,CheckNamePresent,WARN
+0,3,UPDATE,MatchFileProp,ERROR
+0,3,UPDATE,CheckTargetPathValid,ERROR
+0,3,UPDATE,SetImpToFix,ERROR
+
+## property rules
+0,4,INSERT,CheckDatatypePresent,ERROR
+0,4,UPDATE,CheckDatatypePresent,ERROR
+0,4,INSERT,CheckNamePresent,ERROR
+0,4,UPDATE,CheckNamePresent,ERROR
+0,4,INSERT,SetImpToFix,ERROR
+0,4,UPDATE,SetImpToFix,ERROR
+
+## query template rules
+0,8,UPDATE,CheckQueryTemplate,ERROR
+0,8,INSERT,CheckQueryTemplate,ERROR
+
+# data type specific rules
+## reference rules
+0,11,INSERT,CheckRefidPresent,WARN
+0,11,INSERT,CheckRefidValid,ERROR
+0,11,INSERT,CheckRefidIsaParRefid,WARN
+
+0,11,UPDATE,CheckRefidPresent,WARN
+0,11,UPDATE,CheckRefidValid,ERROR
+0,11,UPDATE,CheckRefidIsaParRefid,WARN
+
+## integer rules
+0,12,INSERT,CheckUnitPresent,WARN
+0,12,INSERT,ParseUnit,WARN
+
+0,12,UPDATE,CheckUnitPresent,WARN
+0,12,UPDATE,ParseUnit,WARN
+
+## double rules
+0,13,INSERT,CheckUnitPresent,WARN
+0,13,INSERT,ParseUnit,WARN
+
+0,13,UPDATE,CheckUnitPresent,WARN
+0,13,UPDATE,ParseUnit,WARN
+
+## filereference rules
+0,17,INSERT,CheckRefidValid,ERROR
+0,17,INSERT,CheckRefidIsaParRefid,ERROR
+
+0,17,UPDATE,CheckRefidValid,ERROR
+0,17,UPDATE,CheckRefidIsaParRefid,ERROR