Skip to content
Snippets Groups Projects
Commit b3669164 authored by Henrik tom Wörden's avatar Henrik tom Wörden
Browse files

DOC: minor rephrasing

parent 65008741
No related branches found
No related tags found
2 merge requests!222Release 0.12.0,!220F doc workflow
Pipeline #62116 passed
...@@ -4,17 +4,18 @@ Crawler Workflow ...@@ -4,17 +4,18 @@ Crawler Workflow
The LinkAhead crawler aims to provide a very flexible framework for synchronizing The LinkAhead crawler aims to provide a very flexible framework for synchronizing
data on file systems (or potentially other sources of information) with a data on file systems (or potentially other sources of information) with a
running LinkAhead instance. The workflow that is used in the scientific environment running LinkAhead instance. The workflow that is used in the scientific environment
should be choosen according to the users needs. It is also possible to combine multiple workflow or use them in parallel. should be choosen according to the users needs. It is also possible to combine
multiple workflow or use them in parallel.
In this document we will describe several workflows for crawler operation. In this document we will describe several workflows for crawler operation.
Local Crawler Operation Local Crawler Operation
----------------------- -----------------------
A very simple setup that can also reliably used for testing (e.g. in local A very simple setup that can also reliably be used for testing
docker containers) sets up the crawler on a local computer. The files that sets up the crawler on a local computer. The files that
are being crawled need to be visible to both, the local computer and the are being crawled need to be visible to both, the locally running crawler and
machine, running the LinkAhead. the LinkAhead server.
Prerequisites Prerequisites
+++++++++++++ +++++++++++++
...@@ -58,3 +59,7 @@ Running the crawler ...@@ -58,3 +59,7 @@ Running the crawler
The following command line assumes that the extroot folder visible in the LinkAhead docker container is located in "../extroot": The following command line assumes that the extroot folder visible in the LinkAhead docker container is located in "../extroot":
caosdb-crawler -i identifiables.yml --prefix /extroot --debug --provenance=provenance.yml -s update cfood.yml ../extroot/ExperimentalData/ caosdb-crawler -i identifiables.yml --prefix /extroot --debug --provenance=provenance.yml -s update cfood.yml ../extroot/ExperimentalData/
Server Side Crawler Operation
-----------------------
To be filled.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment