Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
CaosDB Crawler
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
GitLab community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
CaosDB Crawler
Commits
e38fed98
Commit
e38fed98
authored
Oct 17, 2022
by
Henrik tom Wörden
Browse files
Options
Downloads
Patches
Plain Diff
MAINT: doc and rename test
parent
f5086ffa
No related branches found
No related tags found
3 merge requests
!71
REL: RElease v0.2.0
,
!56
F refactor
,
!5
FIX: use identifiable instead of record
Pipeline
#29394
failed
Oct 17, 2022
Stage: info
Stage: setup
Stage: cert
Stage: style
Stage: test
Changes
2
Pipelines
1
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
src/caoscrawler/crawl.py
+7
-6
7 additions, 6 deletions
src/caoscrawler/crawl.py
unittests/test_tool.py
+1
-1
1 addition, 1 deletion
unittests/test_tool.py
with
8 additions
and
7 deletions
src/caoscrawler/crawl.py
+
7
−
6
View file @
e38fed98
...
...
@@ -193,8 +193,9 @@ class Crawler(object):
Please use SecurityMode Enum
"""
# TODO: check if this feature is really needed
# The following caches store records, where we checked whether they exist on the remote
# server. Since, it is important to know whether they exist or not, we store them into two
# different caches.
self
.
remote_existing_cache
=
IdentifiedCache
()
self
.
remote_missing_cache
=
IdentifiedCache
()
self
.
recordStore
=
RecordStore
()
...
...
@@ -525,10 +526,10 @@ class Crawler(object):
def
has_missing_object_in_references
(
self
,
record
:
db
.
Record
):
"""
returns False if any property value is a db.Entity object that
does not have an ID and also
is
not
contained in the
local
cache.
For example, if it was checked before whether such an
db.Entity object existed remotely and it wa
s not
found, it would have been added to the
local cache
.
returns False if any property value is a db.Entity object that
is contained in the
`remote_missing_
cache
`
.
If the record has such an object in the
reference properties, it means that it reference
s
a
not
her Entity, where we checked
whether it exists remotely and it was not found
.
"""
for
p
in
record
.
properties
:
# if (is_reference(p)
...
...
This diff is collapsed.
Click to expand it.
unittests/test_tool.py
+
1
−
1
View file @
e38fed98
...
...
@@ -498,7 +498,7 @@ def test_split_into_inserts_and_updates_with_copy_attr(crawler_mocked_identifiab
crawler
.
identifiableAdapter
.
retrieve_identified_record_for_record
.
assert_called
()
def
test_
no_uncached_entity
_object_in_references
(
crawler
):
def
test_
has_missing
_object_in_references
(
crawler
):
# Simulate remote server content by using the names to identify records
# There are only two known Records with name A and B
crawler
.
identifiableAdapter
.
get_registered_identifiable
=
Mock
(
side_effect
=
partial
(
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment