ENH: enable logging for commandline use

Summary

Setup Logging in case of commandline use

Test Environment

Minimal setup:

mkdir dummy_path
mkdir dummy_shared
touch dummy_cfood.yml

# For terminal output:
caosdb-crawler dummy_cfood.yml dummy_path
SHARED_DIR=dummy_shared caosdb-crawler dummy_cfood.yml dummy_path

I added one debug info line for simple testing:

diff --git a/src/caoscrawler/crawl.py b/src/caoscrawler/crawl.py
index 24a172e..d1def02 100644
--- a/src/caoscrawler/crawl.py
+++ b/src/caoscrawler/crawl.py
@@ -1562,6 +1562,8 @@ def crawler_main(crawled_directory_path: str,
             handler.setLevel(logging.DEBUG if debug else logging.INFO)
             root_logger.addHandler(handler)
 
+        logger.info("OK")
+
         debug_tree = DebugTree()
         crawled_data = scan_directory(
             crawled_directory_path, cfood_file_name, restricted_path, debug_tree=debug_tree)

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Up-to-date JSON schema (or not necessary)
• Appropriate user and developer documentation (or not necessary)
  • How do I use the software? Assume "stupid" users.
  • How do I develop or debug the software? Assume novice developers.
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• Appropriate user and developer documentation (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caoscrawler/crawl.py

@@ -1549,13 +1549,13 @@ def crawler_main(crawled_directory_path: str,
     try:
         crawler = Crawler(securityMode=securityMode)
 
-        # setup logging and reporting if serverside execution
-        if "SHARED_DIR" in os.environ:
+        if "SHARED_DIR" in os.environ:  # setup logging and reporting if serverside execution
             userlog_public, htmluserlog_public, debuglog_public = configure_server_side_logging()
+            # TODO make this optional
             _create_status_record(
                 get_config_setting("public_host_url") + "/Shared/" + htmluserlog_public,
                 crawler.run_id)
-        else:
+        else:  # setup stdout logging for other cases
             root_logger = logging.getLogger()
             root_logger.setLevel(level=(logging.DEBUG if debug else logging.INFO))
             handler = logging.StreamHandler(stream=sys.stdout)