Some refactoring of the main function in crawl.py (and collateral)
Summary
I was going to add some new stuff to main but decided to suggest some clean up first.
Focus
-
updateList
member of theCrawler
was renamed totargetData
. Anto_be_inserted
and anto_be_updated
lists are created later. Thus,updateList
could be confused withto_be_updated
(and I did). The oldupdateList
actually represents how Records and so on should look like, i.e. the target state of the data. - Renamed and documented arguments of the main function (which is now only called main)
Test Environment
Existing tests.
Check List for the Author
Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines
-
All automated tests pass -
Reference related issues -
Up-to-date CHANGELOG.md (ornot necessary) -
Annotations in code (Gitlab comments) - Intent of new code
- Problems with old code
- Why this implementation?
Check List for the Reviewer
-
I understand the intent of this MR -
All automated tests pass -
Up-to-date CHANGELOG.md (or not necessary) -
The test environment setup works and the intended behavior is reproducible in the test environment -
In-code documentation and comments are up-to-date. -
Check: Are there specifications? Are they satisfied?
For further good practices have a look at our review guidelines.
Edited by Alexander Schlemmer