Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
C
Crawler Template
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
Crawler Template
Commits
181f0fa1
Commit
181f0fa1
authored
3 years ago
by
florian
Browse files
Options
Downloads
Patches
Plain Diff
DOC: Add excercise to README.md
parent
42912c8f
No related branches found
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
README.md
+15
-2
15 additions, 2 deletions
README.md
example_cfood.py
+4
-1
4 additions, 1 deletion
example_cfood.py
with
19 additions
and
3 deletions
README.md
+
15
−
2
View file @
181f0fa1
...
@@ -23,13 +23,26 @@ where this file resides.
...
@@ -23,13 +23,26 @@ where this file resides.
Afterwards adjust the data model, the data and the crawler implementation in
Afterwards adjust the data model, the data and the crawler implementation in
`example_cfood.py`
as you like.
`example_cfood.py`
as you like.
## Practice your crawling
After executing
`crawl.py`
you'll notice that two new experiment records with a
date and a result have been created. However, they are lacking the information
about their project. As an excercise on your way towards developing your own
crawler module, try to implement an project cfood that collects the project
information from the directory name(s) within
`data`
and creates or updates a
corresponding project record. Afterwards, extend
`example_cfood.py`
and
`crawl.py`
such that the project information is attached to the experiment
records.
## Contributing
## Contributing
Thank you very much to all contributers—
[
past, present
](
https://gitlab.com/caosdb/caosdb/-/blob/dev/HUMANS.md
)
, and prospective ones.
Thank you very much to all contributers—
[
past,
present
](
https://gitlab.com/caosdb/caosdb/-/blob/dev/HUMANS.md
)
, and prospective
ones.
### Code of Conduct
### Code of Conduct
By participating, you are expected to uphold our
[
Code of Conduct
](
https://gitlab.com/caosdb/caosdb/-/blob/dev/CODE_OF_CONDUCT.md
)
.
By participating, you are expected to uphold our
[
Code of
Conduct
](
https://gitlab.com/caosdb/caosdb/-/blob/dev/CODE_OF_CONDUCT.md
)
.
## License
## License
...
...
This diff is collapsed.
Click to expand it.
example_cfood.py
+
4
−
1
View file @
181f0fa1
...
@@ -36,12 +36,15 @@ except ImportError:
...
@@ -36,12 +36,15 @@ except ImportError:
class
ExampleCFood
(
AbstractFileCFood
):
class
ExampleCFood
(
AbstractFileCFood
):
@classmethod
@classmethod
def
get_re
(
cls
):
def
get_re
(
cls
):
# matches for example `data/2010_TestProject/2019-02-03/
README.md
`
# matches for example `data/2010_TestProject/2019-02-03/
result.yml
`
# The following groups are created (values for the above example):
# The following groups are created (values for the above example):
# - project_identifier: 2010_TestProject
# - project_identifier: 2010_TestProject
# - project_year: 2010
# - project_year: 2010
# - project_name: TestProject
# - project_name: TestProject
# - date: 2019-02-03
# - date: 2019-02-03
#
# Note that the project is not used in the following example (see
# excercise in the README.md).
return
(
r
"
.*/(?P<project_identifier>
"
return
(
r
"
.*/(?P<project_identifier>
"
r
"
(?P<project_year>\d{4})_?(?P<project_name>((?!/).)*))/
"
r
"
(?P<project_year>\d{4})_?(?P<project_name>((?!/).)*))/
"
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment