ENH: allow to provide required columns explicitly

Summary

Added

• TableImporter now accepts a existing_columns argument which demands that certain columns exist although they may have missing values

Changed

• The converters and datatype arguments of TableImporter now may have keys for nonexisting columns

Test Environment

unit test

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosadvancedtools/table_importer.py

@@ -221,7 +221,7 @@ class TableImporter():
           value check is not necessary.
 
         obligatory_columns : list, optional
-          List of column names that must not have missing values.
+          List of column names that (if they exist) must not have missing values.
 
         unique_keys : list, optional
           List of column names that in combination must be unique: each row has a unique
@@ -373,21 +373,20 @@ class TableImporter():
 
         for index, row in df.iterrows():
             # if none of the relevant information is given, skip
-
-            if np.array([pd.isnull(row.loc[key]) for key in self.obligatory_columns]).all():
-
+            if pd.isnull(row.loc[[key for key in self.obligatory_columns if key in df.columns]]).all():
                 df = df.drop(index)
 
                 continue
 
             # if any of the relevant information is missing, report it
-
             i = 0
             okay = True
 
             while okay and i < len(self.obligatory_columns):
                 key = self.obligatory_columns[i]
                 i += 1
+                if key not in df.columns:
+                    continue
 
                 if pd.isnull(row.loc[key]):
                     errmsg = (