Florian Spreckelsen authored 4 years ago and Henrik tom Wörden committed 4 years ago

Also rename the former integration_tests/full_test/test_crawler.py to
integration_tests/test_crawler_with_cfoods.py to prevent confusion.

ENH: fine tune the output during crawling

See merge request caosdb/caosdb-advanced-user-tools!26

Handling of datamodel errors while crawling

See merge request caosdb/caosdb-advanced-user-tools!25

Florian Spreckelsen authored 4 years ago and Henrik tom Wörden committed 4 years ago

To be used while crawling and to collect missing proerties and/or
record types from possible errors while crawling.

Create Authorization Mechanism

See merge request caosdb/caosdb-advanced-user-tools!20

This implements a mechanism that allows to authorize changes of the crawler after a run (see https://gitlab.com/caosdb/caosdb-advanced-user-tools/-/issues/5).

Furthermoer, changes are included that allow the crawler to run as SSS.

This MR shall precede the first release. In order to prevent confusions with unreleased versions, I increased the number to 0.2.0.

This was fixed on the client-side by Timm

Resolve "import error in crawl.py"

See merge request caosdb/caosdb-advanced-user-tools!23

FIX: correct names of tags according to https://doi.org/10.3390/data5020043

See merge request caosdb/caosdb-advanced-user-tools!22

FIX: correct names of tags according to https://doi.org/10.3390/data5020043

See merge request caosdb/caosdb-advanced-user-tools!21

Refactor Crawler

See merge request caosdb/caosdb-advanced-user-tools!19

Create a non file based crawler

See merge request caosdb/caosdb-advanced-user-tools!18

Im und export

See merge request caosdb/caosdb-advanced-user-tools!16

Streamline Crawler Output

See merge request caosdb/caosdb-advanced-user-tools!17

- use logger instaed of print
- switch args to be backwards compatible
- use correct exception