Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
CaosDB Crawler
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
CaosDB Crawler
Commits
f90b4a74
Commit
f90b4a74
authored
2 years ago
by
Henrik tom Wörden
Browse files
Options
Downloads
Patches
Plain Diff
DOC: enhance docs
parent
d1091057
No related branches found
No related tags found
2 merge requests
!91
Release 0.3
,
!67
MAINT: introduce an identifiable class
Pipeline
#30456
passed
2 years ago
Stage: info
Stage: setup
Stage: cert
Stage: style
Stage: test
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
src/doc/concepts.rst
+23
-1
23 additions, 1 deletion
src/doc/concepts.rst
src/doc/index.rst
+7
-7
7 additions, 7 deletions
src/doc/index.rst
with
30 additions
and
8 deletions
src/doc/concepts.rst
+
23
−
1
View file @
f90b4a74
...
@@ -34,8 +34,30 @@ src/converters.py
...
@@ -34,8 +34,30 @@ src/converters.py
Identifiables
Identifiables
+++++++++++++
+++++++++++++
An Identifiable of a Record is like the fingerprint of a Record.
The identifiable contains the information that is used by the CaosDB Crawler to identify Records.
In order to check whether a Record exits in the CaosDB Server, the CaosDB Crawler creates a query
using the information contained in the Identifiable.
For example, suppose a certain experiment is at most done once per day, then the identifiable could
consist of the RecordType "SomeExperiment" and the Property "date".
You can think of the properties that are used by the identifiable as dictionary. For each property
name there can be one value. However, this value can be a list such that the created query can look
like "FIND RECORD ParamenterSet WITH a=5 AND a=6". This is meaningful if there is a ParamenterSet
with two Properties with the name 'a' or if 'a' is a list containing at least the values 5 and 6.
The path of a File object can serve as a Property that identifies files and similarly the name of
Records can be used.
An identifiable can only use one RecordType eventhough the identified Records might have multiple
Parents.
Relevant sources in:
Relevant sources in:
src/identifiable_adapters.py
- src/identifiable_adapters.py
- src/identifiable.py
The Crawler
The Crawler
+++++++++++
+++++++++++
...
...
This diff is collapsed.
Click to expand it.
src/doc/index.rst
+
7
−
7
View file @
f90b4a74
Crawler
2.0
Documentation
CaosDB-
Crawler Documentation
=========================
=========================
===
.. toctree::
.. toctree::
...
@@ -17,18 +17,18 @@ Crawler 2.0 Documentation
...
@@ -17,18 +17,18 @@ Crawler 2.0 Documentation
This is the documentation for the crawler (previously known as crawler 2.0) for CaosDB, ``caosdb-crawler``.
This is the documentation for CaosDB-Crawler (previously known as crawler 2.0)
the main tool for automatic data insertion into CaosDB.
The crawler is the main date integration tool for CaosDB.
Its task is to automatically synchronize data found on file systems or in other
Its task is to automatically synchronize data found on file systems or in other
sources of data with the semantic data model of CaosDB.
sources of data with the semantic data model of CaosDB.
More specifically, data that is contained in a hierarchical structure is converted to a data
More specifically, data that is contained in a hierarchical structure is converted to a data
structure that is consistent with a predefined semantic data model.
structure that is consistent with a predefined semantic data model.
The hierarchical st
u
rcture can be for example a file tree. However it can be
The hierarchical str
u
cture can be for example a file tree. However it can be
also something different like the contents of a
json
file or a file tree with
also something different like the contents of a
JSON
file or a file tree with
json
files.
JSON
files.
This documentation helps you to :doc:`get started<README_SETUP>`, explains the most important
This documentation helps you to :doc:`get started<README_SETUP>`, explains the most important
:doc:`concepts<concepts>` and offers a range of :doc:`tutorials<tutorials/index>`.
:doc:`concepts<concepts>` and offers a range of :doc:`tutorials<tutorials/index>`.
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment