Skip to content
Snippets Groups Projects
Commit cafdd618 authored by Daniel's avatar Daniel
Browse files

Initial commit.

parents
No related branches found
No related tags found
No related merge requests found
Pipeline #624 canceled
%% Cell type:markdown id: tags:
# General Introduction
%% Cell type:markdown id: tags:
We assume that you are connected to the publicly available demo instance at demo.indiscale.com
%% Cell type:code id: tags:
``` python
# loading the module with an alias
import caosdb as db
# the most useful functions and constants can be accessed directly
# E.g. basic information about the LinkAhead instance to you are connected
#db.Info()
# Or the query interface
response = db.execute_query("FIND RECORD Guitar")
# Or classes that represent LinkAhead objects
print(db.Record)
print(db.RecordType)
# Currently it is necessary to force a login if annonymous users are allowed.
db.get_connection()._login()
```
%% Output
<class 'caosdb.common.models.Record'>
<class 'caosdb.common.models.RecordType'>
%% Cell type:markdown id: tags:
Let's start with queries.
%% Cell type:markdown id: tags:
# Queries
Queries work the same way as in the web interface. You simply provide the query string to the corresponding function. However, the result is not displayed as beautifully as in the web interface. That is why browsing through data is the strength of the web interface while the automated processing of data where you know the structure is the strength of the Python client.
%% Cell type:code id: tags:
``` python
print(response)
```
%% Output
<Entities>
<UserInfo>
</UserInfo>
<Record id="115" name="My first guitar">
<Parent id="110" name="Guitar"/>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">48.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">FALSE</Property>
</Record>
<Record id="117" name="Nice guitar">
<Parent id="110" name="Guitar"/>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">2474.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">TRUE</Property>
<Property id="107" name="Manufacturer" datatype="Manufacturer" importance="FIX" flag="inheritance:FIX">116</Property>
</Record>
<Record id="118" name="John Lennon's guitar">
<Parent id="110" name="Guitar"/>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">2120000.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">TRUE</Property>
<Property id="107" name="Manufacturer" datatype="Manufacturer" importance="FIX" flag="inheritance:FIX">116</Property>
</Record>
</Entities>
%% Cell type:markdown id: tags:
As predicted, the text output of the query result is quite hard to read. So, how do we work with such results?
%% Cell type:code id: tags:
``` python
type(response)
```
%% Output
caosdb.common.models.Container
%% Cell type:markdown id: tags:
As you can see the type of the returned object is `Container`. `Container`s are simply lists of LinkAhead objects with useful functions to interact with LinkAhead.
%% Cell type:code id: tags:
``` python
# how many elements did our query return?
len(response)
```
%% Output
3
%% Cell type:markdown id: tags:
Let's look at the elements that we got.
%% Cell type:code id: tags:
``` python
firstguitar = response[0]
print(type(firstguitar))
print(firstguitar)
```
%% Output
<class 'caosdb.common.models.Record'>
<Record id="115" name="My first guitar">
<Parent id="110" name="Guitar"/>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">48.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">FALSE</Property>
</Record>
%% Cell type:markdown id: tags:
You see that the object is a Record. It has a Parent and two Properties.
%% Cell type:code id: tags:
``` python
# parents and properties can be accessed via respective member variables
print(firstguitar.properties)
print(firstguitar.parents)
```
%% Output
<PropertyList>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">48.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">FALSE</Property>
</PropertyList>
<ParentList>
<Parent id="110" name="Guitar"/>
</ParentList>
%% Cell type:markdown id: tags:
Often it is necessary to access the value of a property.
%% Cell type:code id: tags:
``` python
# get the property object
print(firstguitar.get_property("price"))
# the value of it
print(firstguitar.get_property("price").value)
# What is this?
print(firstguitar.get_property(100))
```
%% Output
<Property id="100" name="price" datatype="DOUBLE" unit="€">48.0</Property>
48.0
<Property id="100" name="price" datatype="DOUBLE" unit="€">48.0</Property>
%% Cell type:markdown id: tags:
Why did the second version work? In the web interface we do not realize it that easily, but there is only one thing that uniquely identifies Entities in LinkAhead: the id.
In the xml output you see, that the properties have the ids 100 and 106. Often names of entities are also unique, but this is not guarenteed. Thus in many cases it is preferable or even necessary to use the id for identifying LinkAhead Entities.
%% Cell type:markdown id: tags:
Ids can also come in handy when searching. Suppose you have some complicated condition for the object that you want
%% Cell type:code id: tags:
``` python
# This condition is not that complicated and long but let's suppose it was.
record = db.execute_query("FIND Analysis with quality_factor=0.08", unique=True)
# You can use unique=True when you only expect one result Entity. An error will be
# thrown if the number of results is unequal to 1 and the resulting object will be
# an Entity and not a Container
print(type(record))
print(record.id)
```
%% Output
<class 'caosdb.common.models.Record'>
123
%% Cell type:markdown id: tags:
Using the id of the first query allows you to formulate a second query with a condition involving this object without including the potentially long and complicated subquery in this one:
%% Cell type:code id: tags:
``` python
query = "FIND Guitar WHICH IS REFERENCED BY {id}".format(id=record.id)
print(query)
guitar = db.execute_query(query, unique=True)
print(guitar)
```
%% Output
FIND Guitar WHICH IS REFERENCED BY 123
<Record id="115" name="My first guitar">
<UserInfo>
</UserInfo>
<Parent id="110" name="Guitar"/>
<Property id="100" name="price" datatype="DOUBLE" unit="€" importance="FIX" flag="inheritance:FIX">48.0</Property>
<Property id="106" name="electric" datatype="BOOLEAN" importance="FIX" flag="inheritance:FIX">FALSE</Property>
</Record>
%% Cell type:markdown id: tags:
## Tasks
1. Use `db.execute_query` to determine the number of `MusicalInstrument`s in this LinkAhead instance. Once with `FIND` and once with `Count`.
2. Create a histogram of the `quality_factor`s of analyses.
3. Restrict the values to the ones of `Analysis` Records describing guitars.
%% Cell type:code id: tags:
``` python
# Solution 1
recs = db.execute_query("FIND RECORD MusicalInstrument")
count1 = len(recs)
print(count1)
print(db.execute_query("COUNT RECORD MusicalInstrument"))
```
%% Output
4
4
%% Cell type:code id: tags:
``` python
# Solution 2
import matplotlib
matplotlib.use("nbAgg")
import matplotlib.pyplot as plt
anas = db.execute_query("FIND RECORD Analysis with quality_factor")
qfs = [el.get_property("quality_factor").value for el in anas]
plt.hist(qfs)
plt.xlabel("quality factors")
plt.ylabel("count")
plt.show()
```
%% Output
%% Cell type:code id: tags:
``` python
# Solution 3
anas = db.execute_query("FIND RECORD Analysis with quality_factor AND REFERENCES A Guitar")
qfs = [el.get_property("quality_factor").value for el in anas]
plt.hist(qfs)
plt.xlabel("quality factors")
plt.ylabel("count")
from tempfile import NamedTemporaryFile
histogram_file = NamedTemporaryFile(delete=False, suffix=".png")
histogram_file.close()
plt.savefig(histogram_file.name)
plt.show()
```
%% Output
%% Cell type:markdown id: tags:
Often we are interested in table like data for our processing. And the disentangling of the property values as above is a bit annoying. Thus there is a convenience function for that.
%% Cell type:code id: tags:
``` python
from caosadvancedtools.table_converter import to_table
# Let us retrieve the data in a table like form using `SELECT`
data = db.execute_query("SELECT quality_factor FROM RECORD Analysis with quality_factor" )
table = to_table(data)
print(table)
```
%% Output
quality_factor
0 0.08
1 0.97
2 0.84
%% Cell type:markdown id: tags:
## Files
%% Cell type:code id: tags:
``` python
# You can download files (if the LinkAhead server has access to them)
file = db.execute_query("FIND FILE *2019-023" , unique=True)
target_path = el = file.download()
import os
os.path.exists(target_path)
```
%% Output
True
%% Cell type:markdown id: tags:
If the files are large data files, it is often a better idea to only retrieve the path of the file and access them via a local mount.
%% Cell type:code id: tags:
``` python
print(file.path)
```
%% Output
/reports/report_g_2019-023.pdf
%% Cell type:markdown id: tags:
## Inserting Data
%% Cell type:markdown id: tags:
You can insert data and change the data model via the Python Client.
%% Cell type:code id: tags:
``` python
# A new RecordType is easily created
rt = db.RecordType(name="Flute", description="Here you can provide a description of this recordtype")
# One or more parents can be added
rt.add_parent(name="MusicalInstrument")
# However, the RecordType will not exist in LinkAhead until it is inserted
rt.insert()
```
%% Output
<RecordType id="142" cuid="-1--0b12c17d-d4f5-4e99-aefb-6470030cb32f" name="Flute" description="Here you can provide a description of this recordtype">
<Warning code="0" description="Entity has no properties."/>
<Parent id="108" name="MusicalInstrument" flag="inheritance:OBLIGATORY,"/>
</RecordType>
%% Cell type:code id: tags:
``` python
# Same for properties. They need a datatype, though.
p = db.Property(name="length", datatype=db.DOUBLE)
p.insert()
```
%% Output
<Property id="143" cuid="-1--bd50e699-c260-4015-be52-28ddf2174fe5" name="length" datatype="DOUBLE">
<Warning code="0" description="Entity has no unit."/>
</Property>
%% Cell type:code id: tags:
``` python
# And it is possible to update entities...
rt.add_property(p)
rt.update()
```
%% Output
<RecordType id="142" cuid="-1--0b12c17d-d4f5-4e99-aefb-6470030cb32f" name="Flute" description="Here you can provide a description of this recordtype">
<Parent id="108" name="MusicalInstrument" flag="inheritance:OBLIGATORY,"/>
<Property id="143" cuid="-1--bd50e699-c260-4015-be52-28ddf2174fe5" name="length" datatype="DOUBLE" importance="RECOMMENDED" flag="inheritance:FIX">
<Warning code="0" description="Entity has no unit."/>
</Property>
</RecordType>
%% Cell type:code id: tags:
``` python
# Creating Records is basically the same...
r = db.Record(description="Here you can provide a description of this record")
r.add_parent(name="Flute")
r.add_property(name="length", value=50, unit="cm")
r.insert()
```
%% Output
<Record id="144" cuid="-1--14d0c77a-0736-4db0-90e1-25bf6edfd3c0" description="Here you can provide a description of this record">
<Warning code="0" description="Entity has no name."/>
<Parent id="142" name="Flute" description="Here you can provide a description of this recordtype"/>
<Property id="143" name="length" datatype="DOUBLE" unit="cm" importance="FIX" flag="inheritance:FIX">50.0</Property>
</Record>
%% Cell type:markdown id: tags:
## Tasks
1. Create a new kind of Analysis. It is usually a good idea to prevent name collisions of RecordTypes (i.e. not two RecordTypes with the same name). Also users of one LinkAhead instance should coordinate their creation of RecordTypes and other data model changes. For example, it would be bad if there are two different RecordTypes with different names and properties for essentially the same thing created by two users that did not speak to each other and did not bother to look what exists... For the sake of this tutorial, create an Analysis with some unique suffix.
2. Create a Property that stores files; again with unique suffix in the name
3. Create and insert a Record with your new RecordType as parent, an description and the file with the histogram as property. (You might want to check [the wiki](https://gitlab.com/caosdb/caosdb/wikis/manuals/pylib/PyCaosDB#insertion-of-records-and-files) for information on File insertion). Furhermore, reference all Analysis Records that contributed a quality_factor to the histogram. You might also want to check the [List datatype](https://gitlab.com/caosdb/caosdb/wikis/manuals/pylib/PyCaosDB#define-a-property).
%% Cell type:code id: tags:
``` python
# Solution
p=db.Property(name="quality_hist_henrik", datatype=db.FILE)
p.insert()
rt = db.RecordType(name="QualityAnalysisHenrik", description="Analysis of qualities of Analyses")
rt.add_property(p)
rt.insert()
f = db.File(path="/Analysis/quality_factors/henriks_histo.png", file=histogram_file.name)
f.insert()
r = db.Record()
r.add_parent(rt)
r.add_property(p, value=f)
r.add_property(name="Analysis", value = anas, datatype=db.LIST("Analysis"))
r.insert()
```
%% Output
<Record id="148" cuid="-1--42281370-165a-485c-8ed5-66148db1c3bd">
<Warning code="0" description="Entity has no name."/>
<Parent id="146" name="QualityAnalysisHenrik" description="Analysis of qualities of Analyses"/>
<Property id="145" cuid="-1--dc9c0e37-0828-4445-b7d8-eed71152d9a8" name="quality_hist_henrik" datatype="FILE" importance="FIX" flag="inheritance:FIX">147</Property>
<Property id="112" name="Analysis" datatype="LIST&lt;Analysis&gt;" importance="FIX" flag="inheritance:FIX">
<Value>123</Value>
<Value>124</Value>
</Property>
</Record>
%% Cell type:code id: tags:
``` python
# Clean Up
es = db.execute_query("FIND entity which was inserted by me today")
es.delete()
```
%% Output
<Entities>
<UserInfo>
</UserInfo>
<RecordType id="-1" name="Flute" description="Here you can provide a description of this recordtype">
<Info code="10" description="This entity has been deleted successfully."/>
<Parent id="108" name="MusicalInstrument" flag="inheritance:OBLIGATORY,"/>
<Property id="143" name="length" datatype="DOUBLE" importance="RECOMMENDED" flag="inheritance:FIX"/>
</RecordType>
<Property id="-2" name="length" datatype="DOUBLE">
<Info code="10" description="This entity has been deleted successfully."/>
</Property>
<Record id="-3" description="Here you can provide a description of this record">
<Info code="10" description="This entity has been deleted successfully."/>
<Parent id="142" name="Flute" description="Here you can provide a description of this recordtype"/>
<Property id="143" name="length" datatype="DOUBLE" unit="cm" importance="FIX" flag="inheritance:FIX">50.0</Property>
</Record>
<Property id="-4" name="quality_hist_henrik" datatype="FILE">
<Info code="10" description="This entity has been deleted successfully."/>
</Property>
<RecordType id="-5" name="QualityAnalysisHenrik" description="Analysis of qualities of Analyses">
<Info code="10" description="This entity has been deleted successfully."/>
<Property id="145" name="quality_hist_henrik" datatype="FILE" importance="RECOMMENDED" flag="inheritance:FIX"/>
</RecordType>
<File id="-6" path="/Analysis/quality_factors/henriks_histo.png" checksum="99EF7574D0ECE671BC26CA385C73D50BD26C8DA6CE91FAC068280DC8C4C2EDF3126111A42ADA18D027529FE70DDEC63542AB0B2E3A2E331F47AE8AF4205E60AE" size="10955">
<Info code="10" description="This entity has been deleted successfully."/>
</File>
<Record id="-7">
<Info code="10" description="This entity has been deleted successfully."/>
<Parent id="146" name="QualityAnalysisHenrik" description="Analysis of qualities of Analyses"/>
<Property id="145" name="quality_hist_henrik" datatype="FILE" importance="FIX" flag="inheritance:FIX">147</Property>
<Property id="112" name="Analysis" datatype="LIST&lt;Analysis&gt;" importance="FIX" flag="inheritance:FIX">
<Value>123</Value>
<Value>124</Value>
</Property>
</Record>
</Entities>
%% Cell type:code id: tags:
``` python
```
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment