ENH: add framework for converting json schema into table templates

Summary

Insert a meaningful description for this merge request here: What is the new/changed behavior? Which bug has been fixed? Are there related issues?

Focus

Point the reviewer to the core of the code change. Where should they start reading? What should they focus on (e.g. security, performance, maintainability, user-friendliness, compliance with the specs, finding more corner cases, concrete questions)?

Test Environment

How to set up a test environment for manual testing?

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosadvancedtools/table_json_conversion/table_generator.py

@@ -22,8 +22,8 @@
 """
 This module allows to generate template tables from JSON schemas.
 """
-
 import argparse
+import re
 import sys
 from abc import ABC, abstractmethod
 from argparse import RawTextHelpFormatter
@@ -31,6 +31,7 @@ from enum import Enum
 from typing import Union
 
 from openpyxl import Workbook
+from openpyxl.workbook.child import INVALID_TITLE_REGEX
 
 
 class ColumnType(Enum):
@@ -113,23 +114,23 @@ class TableTemplateGenerator(ABC):
 
     def _get_foreign_keys(self, foreign_keys: dict, path: list) -> list:
         """ returns the foreign keys that are needed at the location to which path points """
-        path = list(path)
+        cpath = list(path)
         keys = foreign_keys
         selected_keys = None
-        while path:
+        while cpath:
             if keys is None:
                 raise ValueError(f"A foreign key definition is missing for path:"
                                  f"\n{path}\n{foreign_keys}")
-            if path[0] not in keys:
+            if cpath[0] not in keys:
                 raise ValueError(f"A foreign key definition is missing for path: \n{path}\n{keys}")
-            keys = keys[path[0]]
+            keys = keys[cpath[0]]
             if isinstance(keys, tuple):
                 selected_keys, keys = keys
             elif isinstance(keys, list):
                 selected_keys, keys = keys, None
             else:
                 selected_keys, keys = None, keys
-            path = path[1:]
+            cpath = cpath[1:]
         if selected_keys is None:
             raise ValueError(f"A foreign key definition is missing for path:"
                              f"\n{path}\n{foreign_keys}")
@@ -147,7 +148,7 @@ class TableTemplateGenerator(ABC):
 
         # if it is an array, value defs are in 'items'
         if 'type' in schema and schema['type'] == 'array':
-            if 'type' in schema['items'] and schema['items']['type'] == 'object':  # list of references; special treatment
+            if 'type' in schema['items'] and schema['items']['type'] == 'object' and len(path) > 1:  # list of references; special treatment
                 # we add a new sheet
                 sheets[".".join(path)] = self._treat_schema_element(schema['items'], sheets, path, foreign_keys)
                 for c in self._get_foreign_keys(foreign_keys, path[:-1]):
@@ -245,7 +246,7 @@ class XLSXTemplateGenerator(TableTemplateGenerator):
         wb = Workbook()
         assert wb.sheetnames == ["Sheet"]
         for sn, sheetdef in sheets.items():
-            ws = wb.create_sheet(sn)
+            ws = wb.create_sheet(re.sub(INVALID_TITLE_REGEX, '_', sn))
             # first row will by the COL_TYPE row
             # first column will be the indicator row with values COL_TYPE, PATH, IGNORE
             # the COL_TYPE row will be followed by as many PATH rows as needed