Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
CaosDB Crawler
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
CaosDB Crawler
Commits
d42540d0
Verified
Commit
d42540d0
authored
1 year ago
by
Daniel Hornung
Browse files
Options
Downloads
Patches
Plain Diff
MAINT Renaming of variables.
parent
c477a46e
Branches
Branches containing commit
Tags
Tags containing commit
2 merge requests
!160
STY: styling
,
!157
FIX Yet another corner case of referencing resolution resolved
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
src/caoscrawler/crawl.py
+7
-8
7 additions, 8 deletions
src/caoscrawler/crawl.py
with
7 additions
and
8 deletions
src/caoscrawler/crawl.py
+
7
−
8
View file @
d42540d0
...
@@ -730,13 +730,12 @@ class Crawler(object):
...
@@ -730,13 +730,12 @@ class Crawler(object):
self
.
treated_records_lookup
.
add
(
record
,
None
)
self
.
treated_records_lookup
.
add
(
record
,
None
)
del
flat
[
i
]
del
flat
[
i
]
resolved_references
=
True
entity_was_treated
=
True
# flat contains Entities which could not yet be checked against the remote server
# flat contains Entities which could not yet be checked against the remote server
try_to_merge_later
=
[]
try_to_merge_later
=
[]
while
resolved_references
and
len
(
flat
)
>
0
:
while
entity_was_treated
and
len
(
flat
)
>
0
:
resolved_references
=
False
entity_was_treated
=
False
referencing_entities
=
self
.
create_reference_mapping
(
referencing_entities
=
self
.
create_reference_mapping
(
all_records
)
all_records
)
# For each element we try to find out whether we can find it in the server or whether
# For each element we try to find out whether we can find it in the server or whether
# it does not yet exist. Since a Record may reference other unkown Records it might not
# it does not yet exist. Since a Record may reference other unkown Records it might not
...
@@ -766,7 +765,7 @@ class Crawler(object):
...
@@ -766,7 +765,7 @@ class Crawler(object):
referencing_entities
=
self
.
create_reference_mapping
(
all_records
)
referencing_entities
=
self
.
create_reference_mapping
(
all_records
)
del
flat
[
i
]
del
flat
[
i
]
resolved_references
=
True
entity_was_treated
=
True
# 2. Can it be checked on the remote server?
# 2. Can it be checked on the remote server?
elif
not
self
.
_has_reference_value_without_id
(
identifiable
):
elif
not
self
.
_has_reference_value_without_id
(
identifiable
):
...
@@ -782,7 +781,7 @@ class Crawler(object):
...
@@ -782,7 +781,7 @@ class Crawler(object):
record
.
path
=
identified_record
.
path
record
.
path
=
identified_record
.
path
self
.
treated_records_lookup
.
add
(
record
,
identifiable
)
self
.
treated_records_lookup
.
add
(
record
,
identifiable
)
del
flat
[
i
]
del
flat
[
i
]
resolved_references
=
True
entity_was_treated
=
True
# 3. Does it have to be new since a needed reference is missing?
# 3. Does it have to be new since a needed reference is missing?
# (Is it impossible to check this record because an identifiable references a
# (Is it impossible to check this record because an identifiable references a
...
@@ -790,7 +789,7 @@ class Crawler(object):
...
@@ -790,7 +789,7 @@ class Crawler(object):
elif
self
.
_has_missing_object_in_references
(
identifiable
,
referencing_entities
):
elif
self
.
_has_missing_object_in_references
(
identifiable
,
referencing_entities
):
self
.
treated_records_lookup
.
add
(
record
,
identifiable
)
self
.
treated_records_lookup
.
add
(
record
,
identifiable
)
del
flat
[
i
]
del
flat
[
i
]
resolved_references
=
True
entity_was_treated
=
True
for
record
in
flat
:
for
record
in
flat
:
self
.
replace_references_with_cached
(
record
,
referencing_entities
)
self
.
replace_references_with_cached
(
record
,
referencing_entities
)
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment