ENH: allow to provide required columns explicitly

Summary

Added

• TableImporter now accepts a existing_columns argument which demands that certain columns exist although they may have missing values

Changed

• The converters and datatype arguments of TableImporter now may have keys for nonexisting columns

Test Environment

unit test

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosadvancedtools/table_importer.py

@@ -374,7 +374,8 @@ class TableImporter():
         for index, row in df.iterrows():
             # if none of the relevant information is given, skip
 
-            if np.array([pd.isnull(row.loc[key]) for key in self.obligatory_columns]).all():
+            if np.array([pd.isnull(row.loc[key]) for key in self.obligatory_columns
+                         if key in df.columns]).all():
 
                 df = df.drop(index)