Skip to content

Some refactoring of the main function in crawl.py (and collateral)

Henrik tom Wörden requested to merge f-main into dev

Summary

I was going to add some new stuff to main but decided to suggest some clean up first.

Focus

  1. updateList member of the Crawler was renamed to targetData. An to_be_inserted and an to_be_updated lists are created later. Thus, updateList could be confused with to_be_updated (and I did). The old updateList actually represents how Records and so on should look like, i.e. the target state of the data.
  2. Renamed and documented arguments of the main function (which is now only called main)

Test Environment

Existing tests.

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

  • All automated tests pass
  • Reference related issues
  • Up-to-date CHANGELOG.md (or not necessary)
  • Annotations in code (Gitlab comments)
    • Intent of new code
    • Problems with old code
    • Why this implementation?

Check List for the Reviewer

  • I understand the intent of this MR
  • All automated tests pass
  • Up-to-date CHANGELOG.md (or not necessary)
  • The test environment setup works and the intended behavior is reproducible in the test environment
  • In-code documentation and comments are up-to-date.
  • Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

Edited by Alexander Schlemmer

Merge request reports