Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
C
caosdb-pylib
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
caosdb
Software
caosdb-pylib
Commits
f49137e6
Verified
Commit
f49137e6
authored
1 year ago
by
Daniel Hornung
Browse files
Options
Downloads
Patches
Plain Diff
MAINT: Renamed fill_cache -> cache_fill, also updated documentation.
parent
1cf952e0
No related branches found
Branches containing commit
No related tags found
Tags containing commit
2 merge requests
!107
ENH: add entity getters and cached functions
,
!100
ENH: add entity getters and cached functions
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
src/caosdb/cached.py
+17
-9
17 additions, 9 deletions
src/caosdb/cached.py
src/doc/tutorials/caching.rst
+9
-9
9 additions, 9 deletions
src/doc/tutorials/caching.rst
unittests/test_cached.py
+1
-1
1 addition, 1 deletion
unittests/test_cached.py
with
27 additions
and
19 deletions
src/caosdb/cached.py
+
17
−
9
View file @
f49137e6
...
...
@@ -2,8 +2,9 @@
#
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2023 Henrik tom Wörden <h.tomwoerden@indiscale.com>
# Copyright (C) 2023 IndiScale GmbH <info@indiscale.com>
# Copyright (C) 2023 Henrik tom Wörden <h.tomwoerden@indiscale.com>
# Copyright (C) 2023 Daniel Hornung <d.hornung@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
...
...
@@ -23,14 +24,14 @@
This module provides some cached versions of functions that retrieve Entities from a remote server.
"""
from
typing
import
Union
from
enum
import
Enum
from
functools
import
lru_cache
from
typing
import
Union
from
.utils
import
get_entity
from
.common.models
import
execute_query
,
Entity
,
Container
from
enum
import
Enum
# roughly 1GB for typical entity sizes
DEFAULT_SIZE
=
33333
...
...
@@ -40,7 +41,7 @@ _DUMMY_CACHE = {}
class
AccessType
(
Enum
):
"""
Different access types for cached queries. Needed for filling the cache manually with
:func:`
fill_
cache` .
:func:`cache
_fill
` .
"""
QUERY
=
1
...
...
@@ -84,7 +85,7 @@ If a query phrase is given, the result must be unique. If this is not what you
def
cached_query
(
query_string
)
->
Container
:
"""
A cached version of
the
:func:`caosdb.execute_query<caosdb.common.models.execute_query>`
function
.
"""
A cached version of :func:`caosdb.execute_query<caosdb.common.models.execute_query>`.
All additional arguments are at their default values.
...
...
@@ -120,21 +121,28 @@ def cache_clear() -> None:
def
cache_info
():
"""
Empty the cache that is used by `cached_query` and `cached_get_entity_by`.
"""
"""
Return info about the cache that is used by `cached_query` and `cached_get_entity_by`.
Returns
-------
out: named tuple
See the standard library :func:`functools.lru_cache` for details.
"""
return
_cached_access
.
cache_info
()
def
cache_initialize
(
maxsize
=
DEFAULT_SIZE
)
->
None
:
"""
Create a new cache with the given size for `cached_query` and `cached_get_entity_by`.
Th
e old cache is removed with all
i
t
s
content
.
Th
is implies a call of :func:`cache_clear`, the old cache
is
emptied
.
"""
cache_clear
()
global
_cached_access
_cached_access
=
lru_cache
(
maxsize
=
maxsize
)(
_cached_access
.
__wrapped__
)
def
fill_
cache
(
items
:
dict
,
kind
:
AccessType
=
AccessType
.
EID
,
unique
:
bool
=
True
)
->
None
:
def
cache
_fill
(
items
:
dict
,
kind
:
AccessType
=
AccessType
.
EID
,
unique
:
bool
=
True
)
->
None
:
"""
Add entries to the cache manually.
This allows to fill the cache without actually submitting queries. Note that this does not
...
...
This diff is collapsed.
Click to expand it.
src/doc/tutorials/caching.rst
+
9
−
9
View file @
f49137e6
...
...
@@ -22,22 +22,22 @@ function, easily created from ``get_entity_by_name`` using Python's ``lru_cache`
cached_get_by_name.cache_clear()
For convenience, PyCaosDB provides the ``caosdb.cached`` module that defines the functions
``cached_query`` and ``cached_get_by``, they use a shared cache. Let's have a look:
``cached_query`` and ``cached_get_
entity_
by``, they use a shared cache. Let's have a look:
.. code:: python
from caosdb.cached import cached_query, cached_get_by, cache_clear, cache_info, initialize
_cache
rt1 = cached_get_by(name='RT1')
from caosdb.cached import cached_query, cached_get_
entity_
by, cache_clear, cache_info,
cache_
initialize
rt1 = cached_get_
entity_
by(name='RT1')
qresult = cached_query('FIND Experiment WITH parameter=1')
# you can inspect the cache
print(cache_info())
# this will not cause a server request since it is cached
rt1 = cached_get_by(name='RT1')
rt1 = cached_get_
entity_
by(name='RT1')
# you can clear the cache with
cache_clear()
# If you want to have a cache with a custom size, you can initialize it (again). Old cached
# data is lost.
initialize
_cache(
size=10)
cache_
initialize
(max
size=10)
If you want to manually add entities to the cache, you can do it yourself. This is useful when you
...
...
@@ -45,12 +45,12 @@ have entities on hand from previous queries that you want to add.
.. code:: python
from caosdb.cached import
fill_
cache, AccessType
from caosdb.cached import cache
_fill
, AccessType
# Here, items must be a dict with Entity IDs as keys and the Entities as values.
fill_
cache(items, AccessType.EID, unique=True)
cache
_fill
(items, AccessType.EID, unique=True)
# If you now use IDs that were in items, they are taken from the cache.
e1 = cached_get_by(eid=10001)
e1 = cached_get_
entity_
by(eid=10001)
For the cached entity getter functions (``cached_get_by``) you need to set ``unique=True``.
For the cached entity getter functions (``cached_get_
entity_
by``) you need to set ``unique=True``.
This diff is collapsed.
Click to expand it.
unittests/test_cached.py
+
1
−
1
View file @
f49137e6
...
...
@@ -193,7 +193,7 @@ def test_cached_query(mocked_query):
cached_query
(
'
stuff
'
)
assert
mocked_query
.
call_count
==
2
# we fill the cache manually and make sure the element is used
fill_
cache
({
'
lol
'
:
db
.
Container
().
extend
([
db
.
Entity
(
id
=
10001
,
name
=
'
lol
'
)])},
cache
_fill
({
'
lol
'
:
db
.
Container
().
extend
([
db
.
Entity
(
id
=
10001
,
name
=
'
lol
'
)])},
AccessType
.
QUERY
,
unique
=
False
)
# there are now two elements in the cache: a and lol
assert
cache_info
().
currsize
==
2
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment