Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
CaosDB Crawler
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
CaosDB Crawler
Commits
b3669164
Commit
b3669164
authored
1 month ago
by
Henrik tom Wörden
Browse files
Options
Downloads
Patches
Plain Diff
DOC: minor rephrasing
parent
65008741
No related branches found
No related tags found
2 merge requests
!222
Release 0.12.0
,
!220
F doc workflow
Pipeline
#62116
passed
1 month ago
Stage: info
Stage: setup
Stage: cert
Stage: style
Stage: test
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
src/doc/workflow.rst
+10
-5
10 additions, 5 deletions
src/doc/workflow.rst
with
10 additions
and
5 deletions
src/doc/workflow.rst
+
10
−
5
View file @
b3669164
...
@@ -4,17 +4,18 @@ Crawler Workflow
...
@@ -4,17 +4,18 @@ Crawler Workflow
The LinkAhead crawler aims to provide a very flexible framework for synchronizing
The LinkAhead crawler aims to provide a very flexible framework for synchronizing
data on file systems (or potentially other sources of information) with a
data on file systems (or potentially other sources of information) with a
running LinkAhead instance. The workflow that is used in the scientific environment
running LinkAhead instance. The workflow that is used in the scientific environment
should be choosen according to the users needs. It is also possible to combine multiple workflow or use them in parallel.
should be choosen according to the users needs. It is also possible to combine
multiple workflow or use them in parallel.
In this document we will describe several workflows for crawler operation.
In this document we will describe several workflows for crawler operation.
Local Crawler Operation
Local Crawler Operation
-----------------------
-----------------------
A very simple setup that can also reliably used for testing
(e.g. in local
A very simple setup that can also reliably
be
used for testing
docker containers)
sets up the crawler on a local computer. The files that
sets up the crawler on a local computer. The files that
are being crawled need to be visible to both, the local
comput
er and
the
are being crawled need to be visible to both, the local
ly running crawl
er and
machine, running
the LinkAhead.
the LinkAhead
server
.
Prerequisites
Prerequisites
+++++++++++++
+++++++++++++
...
@@ -58,3 +59,7 @@ Running the crawler
...
@@ -58,3 +59,7 @@ Running the crawler
The following command line assumes that the extroot folder visible in the LinkAhead docker container is located in "../extroot":
The following command line assumes that the extroot folder visible in the LinkAhead docker container is located in "../extroot":
caosdb-crawler -i identifiables.yml --prefix /extroot --debug --provenance=provenance.yml -s update cfood.yml ../extroot/ExperimentalData/
caosdb-crawler -i identifiables.yml --prefix /extroot --debug --provenance=provenance.yml -s update cfood.yml ../extroot/ExperimentalData/
Server Side Crawler Operation
-----------------------
To be filled.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment