ENH: Implement queries and entity retrieval

.docker/caosdb_client.json

+{
+  "connections": {
+    "default": "julialib-unittest",
+    "julialib-unittest": {
+      "host": "caosdb-server",
+      "port": 8443,
+      "authentication": {
+        "type": "plain",
+        "username": "admin",
+        "password": "caosdb"
+      }
+    },
+    "local-caosdb": {
+      "host": "localhost",
+      "port": 8443,
+      "server_certificate_path": "some/path/cacert.pem",
+      "authentication": {
+        "type": "plain",
+        "username": "me",
+        "password": "secret!"
+      }
+    }
+  },
+  "extension": {
+    "this is my": "special option"
+  }
+}

.docker/install_cpplib.sh

@@ -5,8 +5,10 @@ cd caosdb-cpplib
 # check if there is a crooesponding cpp branch. Use the default branch
 # if there isn't.
 if (git show-ref --verify --quiet refs/heads/$CI_COMMIT_REF_NAME) ; then
+    echo "git checkout $CI_COMMIT_REF_NAME"
     git checkout $CI_COMMIT_REF_NAME
 else
+    echo "git checkout $CPP_DEFAULT_BRANCH"
     git checkout $CPP_DEFAULT_BRANCH
 fi
 git submodule update --init --recursive

.gitlab-ci.yml

@@ -25,6 +25,7 @@ variables:
   TRIGGERED_BY_HASH: $CI_COMMIT_SHORT_SHA
 
   # The defalt branch to use with caosdb-cpplib
+  # TODO: Change back to dev once f-consolidate-c has been merged.
   CPP_DEFAULT_BRANCH: dev
 
 image: $JULIALIB_REGISTRY_IMAGE
@@ -100,6 +101,7 @@ test:
   script:
     - .docker/install_cpplib.sh
     - export LD_LIBRARY_PATH=/root/.local/lib:$LD_LIBRARY_PATH
+    - export CAOSDB_CLIENT_CONFIGURATION=$(pwd)/.docker/caosdb_client.json
     # Let's run the tests. Substitute `coverage = false` below, if you
     # do not want coverage results.
     - julia -e 'using Pkg; Pkg.add(path=pwd());

CHANGELOG.md

@@ -14,6 +14,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 * Support for Windows
 * Support for config files
 * Minimal error handling
+* Queries
+* Retrieval by id(s)
 
 ### Changed
 

Project.toml

 name = "CaosDB"
 uuid = "091fdcf5-a163-4d8f-97f4-9adce40cd04e"
 authors = ["florian <f.spreckelsen@inidscale.com>"]
-version = "0.0.2"
+version = "0.0.3"
 
 [deps]
 Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

README_SETUP.md

@@ -35,6 +35,10 @@ the wrapped cpp library couldn't be loaded from the c interface) make
 sure you have the location of `libcaosdb.dylib` and `libccaosdb.dylib`
 in both, your `LD_LIBRARY_PATH` and your `DYLD_LIBRARY_PATH`.
 
+## Configuration
+The configuration is done the same way as the C++ client would do it. See 
+[caosdb-cpplib](https://gitlab.com/caosdb/caosdb-cpplib).
+
 ## Tests
 
 After installing, the unit tests can be executed by
@@ -44,6 +48,9 @@ julia> ]activate "/path/to/caosdb-julialib"
 julia> ]test
 ```
 
+Note that you must have a CaosDB client configuration (see above) with
+a valid `"default"` configuration for all tests to succeed.
+
 ## Code styling
 
 We use

docs/src/api.md

@@ -5,16 +5,24 @@ interface. You also find the documentation of the internal API which
 is meant for expert use only. Only use those if you know what you're
 doing.
 
+```@index
+Order = [:module, :function, :macro, :type, :constant]
+```
+
 ## Public API
 
 ```@autodocs
-Modules=[CaosDB, CaosDB.Exceptions, CaosDB.Info, CaosDB.Utility, CaosDB.Connection, CaosDB.Authentication, CaosDB.Entity]
+Modules=[CaosDB, CaosDB.Exceptions, CaosDB.Info, CaosDB.Utility,
+	CaosDB.Connection, CaosDB.Authentication, CaosDB.Entity,
+	CaosDB.Transaction]
 Private=false
 ```
 
 ## Expert-use only API functions
 
 ```@autodocs
-Modules=[CaosDB, CaosDB.Exceptions, CaosDB.Info, CaosDB.Utility, CaosDB.Connection, CaosDB.Authentication, CaosDB.Entity]
+Modules=[CaosDB, CaosDB.Exceptions, CaosDB.Info, CaosDB.Utility,
+	CaosDB.Connection, CaosDB.Authentication, CaosDB.Entity,
+	CaosDB.Transaction]
 Public=false
 ```

docs/src/examples.md

@@ -18,12 +18,128 @@ connection = CaosDB.Connection.connect()
 ```
 
 which will establish a connection and print the version of the server
-you are connected to.
+you are connected to (since `connect` is exported, `connection =
+connect()` works as well).
 
 ## Retrieve a Record
 
+With CaosDB.jl you may use the same logic of creating a transaction
+object, filling it with sub-requests, executing it, and retrieving the
+result(s) as it is done in the [C++
+client](https://docs.indiscale.com/coasdb-cpplib) (see below). This is
+handy when wanting to have several requests in the same
+transaction. However, most of the time, e.g., when retrieving one or
+multiple Records, this is not necessary. CaosDB.jl provides helper
+functions so you don't have to worry about transactions and their
+executions. Assume you want to retrieve an Entity with id=123. This is done via
+
+```julia
+using CaosDB
+entity = retrieve("123")
+```
+
+It is possible to specify the connection either by name or as a
+connection object as created above in a second argument to
+[`retrieve()`](@ref).
+
+You can then use the getter methods like [`get_name`](@ref),
+[`get_parents`](@ref), or [`get_properties`](@ref) to get the name,
+the parents, or the properties of the retrieved entity.
+
+Retrieving multiple entities works in the same way.  Type
+
+```julia
+results = retrieve(["123", "124", "125"])
+```
+
+to retrieve the entities with ids 123, 124, and 125. They can then be
+accessed as `results[1]`, `results[2]`, and `results[3]`,
+respectively. Note that [`retrieve`](@ref) returns a single entity
+when called with a single id whereas a vector of entities is returned
+when called with a list of ids.
+
+The same (and more) can be achieved by building and executing the
+transaction manually. This is done by
+
+```julia
+transaction = create_transaction()
+add_retrieve_by_id(transaction, "123")
+add_retrieve_by_id(transaction, "124")
+add_retrieve_by_id(transaction, "125")
+execute_transaction(transaction)
+results = get_results(transaction)
+```
+
+Again, a connection can be specified by name or object as an optional
+argument in the call to [`create_transaction`](@ref). Optionally, and
+in the same way as in the C++ client, a result set can be obtained via
+[`get_result_set`](@ref) which contains the resulting entities and
+can, e.g., be checked for length. However, since it is not necessary
+to interact with the result set and one usually is interested in the
+recieved entities proper, the result set can be omitted. As above, it
+is also possible to add multiple id retrievals at once.
+
+```julia
+transaction = create_transaction()
+add_retrieve_by_id(transaction, ["123", "124", "125"])
+execute_transaction(transaction)
+results = get_results(transaction)
+```
+
+is equivalent to the above Code.
+
 ## Execute queries
 
+Executing queries works very similar to the retrieval of entities via
+ids. Again it is possible to create and execute a transaction manually
+but most cases can be covered by the [`execute_query`](@ref) helper
+function.
+
+### FIND and SELECT queries
+
+In general, entities can be found using CaosDB's query language like
+
+```julia
+results = execute_query("FIND RECORD WITH name Like 'some_pattern' AND id>122")
+```
+
+`results` is a (possibly empty) vector of entities that can be
+inspected as above. Similar to `retrieve`, a connection can be
+specified by name or by a connection object in
+[`execute_query`](@ref). If none is specified, the default connection
+is used. The same result is achieved by constructing the transaction
+manually:
+
+```julia
+transaction = create_transaction()
+add_query(transaction, "FIND RECORD WITH name Like 'some_pattern' AND id>122")
+execute(transaction)
+results = get_results(transaction)
+```
+
+!!! warning
+
+	SELECT queries haven't been implemented in the C++ client yet and
+	thus cannot be executed from CaosDB.jl right now. SELECT queries
+	will be added in a future release.
+
+### COUNT queries
+
+COUNT queries are different from FIND or SELECT queries in two
+ways. Firstly, they do not return entities but a single number which
+is why, secondly, they do not have a result set that could be
+returned. This is why they are currently not covered by the
+`execute_query` function and have to be executed manually instead. The
+result of the count is obtained using the [`get_count_result`](@ref)
+function:
+
+```julia
+transaction = create_transaction()
+add_query(transaction, "COUNT RECORD person WITH NAME LIKE '*Baggins'")
+execute(transaction)
+count_result = get_count_result(transaction)
+```
+
 ## Insert, update, and delete entities
 
 ## Download a file

src/Authentication.jl

+# ** header v3.0
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+#
+# ** end header
+#
+module Authentication
+
+using ..CaosDB
+
+"""
+Struct containing a pointer to the wrapped cpp authenticator
+class. Meant for internal use; call a
+`CaosDB.Authentication.create_<authenticator>` function to create an
+authenticator object from a configuration.
+"""
+mutable struct _Authenticator
+    wrapped_authenticator::Ptr{Cvoid}
+    _deletable::Bool
+
+    function _Authenticator(managed_by_julia::Bool = false)
+        auth = new()
+        auth._deletable = false
+        if managed_by_julia
+            # Only append a finalizer for this if the object is
+            # actually managed by Julia and not created and destroyed
+            # internally by libcaosdb.
+            function f(t)
+                ccall(
+                    (:caosdb_authentication_delete_authenticator, CaosDB.library_name),
+                    Cint,
+                    (Ref{_Authenticator},),
+                    Ref{_Authenticator}(t),
+                )
+            end
+            finalizer(f, auth)
+        end
+        return auth
+    end
+end
+
+"""
+    create_plain_password_authenticator(
+        username::AbstractString,
+        password::AbstractString,
+    )
+
+Return an authenticator object that contains a wrapped cpp
+plain-password authenticator configured with `username` and
+`password`.
+"""
+function create_plain_password_authenticator(
+    username::AbstractString,
+    password::AbstractString,
+)
+
+    auth = Ref{_Authenticator}(_Authenticator(true))
+
+    err_code = ccall(
+        (:caosdb_authentication_create_plain_password_authenticator, CaosDB.library_name),
+        Cint,
+        (Ref{_Authenticator}, Cstring, Cstring),
+        auth,
+        username,
+        password,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return auth
+
+end
+
+end # Authentication

src/CaosDB.jl

@@ -24,6 +24,74 @@
 
 module CaosDB
 
+# Exports from module Exceptions
+export evaluate_return_code,
+    CaosDBException, ClientException, GenericCaosDBException, CaosDBMessage
+
+# Exports from module Utility
+export get_env_fallback
+
+# Export from module Connection
+export connect, connect_manually
+
+# Exports from module Entity
+# Creators
+export create_entity,
+    create_parent, create_property, create_property_entity, create_record, create_recordtype
+
+# getters
+export get_id,
+    get_role,
+    get_name,
+    get_description,
+    get_datatype,
+    get_unit,
+    get_value,
+    get_version_id,
+    get_property,
+    get_properties,
+    get_parent,
+    get_parents,
+    get_error,
+    get_errors,
+    get_warning,
+    get_warnings,
+    get_info,
+    get_infos,
+    get_importance,
+    get_code
+
+# setters
+export append_parent,
+    append_parents,
+    append_property,
+    append_properties,
+    remove_parent,
+    remove_property,
+    set_id,
+    set_role,
+    set_name,
+    set_description,
+    set_datatype,
+    set_unit,
+    set_value,
+    set_importance
+
+# helper functions
+export has_errors, has_warnings
+
+# Exports from module Transaction
+export create_transaction,
+    add_retrieve_by_id,
+    add_query,
+    execute,
+    get_result_set,
+    get_count_result,
+    get_result_at,
+    get_results,
+    execute_query,
+    retrieve
+
 using Libdl
 
 """
@@ -34,186 +102,22 @@ if isempty(find_library(library_name))
     @error "Could not find $library_name"
 end
 
-module Exceptions
-
-export evaluate_return_code, CaosDBException, GenericCaosDBException, CaosDBMessage
-
-using Logging
-using CaosDB
-
-"""
-The parent type of all CaosDB errors that can also be used for testing.
-"""
-abstract type CaosDBException <: Exception end
-
-"""
-A generic exception that will be raised in case of non-zero return
-values of the calls to libccaosdb. May carry a message string and a
-code.
-"""
-struct GenericCaosDBException <: CaosDBException
-    msg::String
-    code::Cint
-end
-
-Base.showerror(io::IO, e::CaosDBException) =
-    print(io, "CaosDBException: ", e.msg, " (Status code ", e.code, ")")
-
-"""
-Struct containing Messages and codes for status codes<0 that do not
-correspond to errors or success.
-"""
-struct CaosDBMessage
-    msg::String
-    code::Cint
-end
-
-Base.show(io::IO, m::CaosDBMessage) = print(io, m.msg, " (Status code ", m.code, ")")
-
-"""
-    function evaluate_return_code(code::Cint)
-
-Evaluate the return code of a libccaosdb ccall and raise a
-`GenericCaosDBException` in case of a non-zero return code.
-"""
-function evaluate_return_code(code::Cint)
-    if code != 0
-        msg = ccall(
-            (:caosdb_get_status_description, CaosDB.library_name),
-            Cstring,
-            (Cint,),
-            code,
-        )
-        if code > 0
-            throw(GenericCaosDBException(unsafe_string(msg), code))
-        else
-            @info CaosDBMessage(unsafe_string(msg), code)
-        end
-    end
-end
-
-end # Exceptions
-
-module Info
-
-"""
-Struct containing version information of the CaosDB server. Meant
-mainly for internal usage; use `CaosDB.Connection.get_version_info` or
-`CaosDB.Connection.print_version_info` to retrieve the version of the
-connected CaosDB server.
-"""
-mutable struct _VersionInfo
-
-    major::Cint
-    minor::Cint
-    patch::Cint
-    pre_release::Cstring
-    build::Cstring
-
-    _VersionInfo() = new()
-
-end
-
-end # Info
-
-module Utility
-
-export get_env_var
-
-using ..CaosDB
-
-"""
-    get_env_var(var[, default])
-
-Return the environmental variable `var` if it exists, `default`
-otherwise. If no `default` is given an empty string is returned
-instead.
-"""
-function get_env_var(var::AbstractString, default::AbstractString = "")
-
-    ret = ccall(
-        (:caosdb_utility_get_env_var, CaosDB.library_name),
-        Cstring,
-        (Cstring, Cstring),
-        var,
-        default,
-    )
-
-    return unsafe_string(ret)
-
-end
-
-end # Utility
+# include modules from other files
 
-module Authentication
+include("Exceptions.jl")
 
-using ..CaosDB
+include("Info.jl")
 
-"""
-Struct containing a pointer to the wrapped cpp authenticator
-class. Meant for internal use; call a
-`CaosDB.Authentication.create_<authenticator>` function to create an
-authenticator object from a configuration.
-"""
-mutable struct _Authenticator
-    wrapped_authenticator::Ptr{Cvoid}
-
-    function _Authenticator(managed_by_julia::Bool = false)
-        auth = new()
-        if managed_by_julia
-            # Only append a finalizer for this if the object is
-            # actually managed by Julia and not created and destroyed
-            # internally by libcaosdb.
-            function f(t)
-                ccall(
-                    (:caosdb_authentication_delete_authenticator, CaosDB.library_name),
-                    Cint,
-                    (Ref{_Authenticator},),
-                    Ref{_Authenticator}(t),
-                )
-            end
-            finalizer(f, auth)
-        end
-        return auth
-    end
-end
+include("Utility.jl")
 
-"""
-    create_plain_password_authenticator(
-        username::AbstractString,
-        password::AbstractString,
-    )
-
-Return an authenticator object that contains a wrapped cpp
-plain-password authenticator configured with `username` and
-`password`.
-"""
-function create_plain_password_authenticator(
-    username::AbstractString,
-    password::AbstractString,
-)
+include("Authentication.jl")
 
-    auth = Ref{_Authenticator}(_Authenticator(true))
-
-    err_code = ccall(
-        (:caosdb_authentication_create_plain_password_authenticator, CaosDB.library_name),
-        Cint,
-        (Ref{_Authenticator}, Cstring, Cstring),
-        auth,
-        username,
-        password,
-    )
-
-    CaosDB.Exceptions.evaluate_return_code(err_code)
-
-    return auth
-
-end
+include("Connection.jl")
 
-end # Authentication
+include("Entity.jl")
 
-include("Connection.jl")
+include("Transaction.jl")
 
-module Entity end
+using .Exceptions, .Info, .Authentication, .Connection, .Utility, .Entity, .Transaction
 
 end # CaosDB

src/Connection.jl

@@ -34,9 +34,11 @@ to create an connection object from a configuration.
 """
 mutable struct _Connection
     wrapped_connection::Ptr{Cvoid}
+    _deletable::Bool
 
     function _Connection(managed_by_julia::Bool = false)
         conn = new()
+        conn._deletable = false
         if managed_by_julia
             # Only append a finalizer for this if the object is
             # actually managed by Julia and not created and destroyed
@@ -63,9 +65,11 @@ an certificate-provider object from a configuration.
 """
 mutable struct _CertificateProvider
     wrapped_certificate_provider::Ptr{Cvoid}
+    _deletable::Bool
 
     function _CertificateProvider(managed_by_julia::Bool = false)
         prov = new()
+        prov._deletable = false
         if managed_by_julia
             # Only append a finalizer for this if the object is
             # actually managed by Julia and not created and destroyed
@@ -92,9 +96,11 @@ an connection-configuration object from a configuration.
 """
 mutable struct _Configuration
     wrapped_connection_configuration::Ptr{Cvoid}
+    _deletable::Bool
 
     function _Configuration(managed_by_julia::Bool = false)
         config = new()
+        config._deletable = false
         if managed_by_julia
             # Only append a finalizer for this if the object is
             # actually managed by Julia and not created and destroyed
@@ -358,13 +364,13 @@ function connect_manually(;
 
     if host == ""
 
-        host = CaosDB.Utility.get_env_var("CAOSDB_SERVER_HOST", "localhost")
+        host = CaosDB.Utility.get_env_fallback("CAOSDB_SERVER_HOST", "localhost")
 
     end
 
     if port_str == "undefined"
 
-        port_str = CaosDB.Utility.get_env_var("CAOSDB_SERVER_GRPC_PORT_HTTPS", "8443")
+        port_str = CaosDB.Utility.get_env_fallback("CAOSDB_SERVER_GRPC_PORT_HTTPS", "8443")
 
     end
 
@@ -372,19 +378,19 @@ function connect_manually(;
 
     if cacert == ""
 
-        cacert = CaosDB.Utility.get_env_var("CAOSDB_SERVER_CERT")
+        cacert = CaosDB.Utility.get_env_fallback("CAOSDB_SERVER_CERT")
 
     end
 
     if username == ""
 
-        username = CaosDB.Utility.get_env_var("CAOSDB_USER", "admin")
+        username = CaosDB.Utility.get_env_fallback("CAOSDB_USER", "admin")
 
     end
 
     if password == "undefined"
 
-        password = CaosDB.Utility.get_env_var("CAOSDB_PASSWORD", "caosdb")
+        password = CaosDB.Utility.get_env_fallback("CAOSDB_PASSWORD", "caosdb")
 
     end
 

src/Exceptions.jl

+# ** header v3.0
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
+# Copyright (C) 2021 Alexander Kreft
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+#
+# ** end header
+#
+module Exceptions
+
+export evaluate_return_code,
+    CaosDBException, ClientException, GenericCaosDBException, CaosDBMessage
+
+using Logging
+using CaosDB
+
+"""
+The parent type of all CaosDB errors that can also be used for testing.
+"""
+abstract type CaosDBException <: Exception end
+
+"""
+A generic exception that will be raised in case of non-zero return
+values of the calls to libccaosdb. May carry a message string and a
+code.
+"""
+struct GenericCaosDBException <: CaosDBException
+    msg::String
+    code::Cint
+end
+
+"""
+Something went wrong on the client-side or the user is attempting to
+conduct an invalid operation.
+"""
+struct ClientException <: CaosDBException
+    msg::String
+    code::Cint
+
+    function ClientException(message::AbstractString)
+        client_error_code =
+            ccall((:caosdb_status_code_OTHER_CLIENT_ERROR, CaosDB.library_name), Cint, ())
+        new(message, client_error_code)
+    end
+end
+
+
+Base.showerror(io::IO, e::CaosDBException) =
+    print(io, "CaosDBException: ", e.msg, " (Status code ", e.code, ")")
+
+"""
+Struct containing Messages and codes for status codes<0 that do not
+correspond to errors or success.
+"""
+struct CaosDBMessage
+    msg::String
+    code::Cint
+end
+
+Base.show(io::IO, m::CaosDBMessage) = print(io, m.msg, " (Status code ", m.code, ")")
+
+"""
+    function evaluate_return_code(code::Cint)
+
+Evaluate the return code of a libccaosdb ccall and raise a
+`GenericCaosDBException` in case of a non-zero return code.
+"""
+function evaluate_return_code(code::Cint)
+    if code != 0
+        msg = ccall(
+            (:caosdb_get_status_description, CaosDB.library_name),
+            Cstring,
+            (Cint,),
+            code,
+        )
+        if code > 0
+            throw(GenericCaosDBException(unsafe_string(msg), code))
+        else
+            @info CaosDBMessage(unsafe_string(msg), code)
+        end
+    end
+end
+
+end # Exceptions

src/Info.jl

+# ** header v3.0
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+#
+# ** end header
+#
+module Info
+
+"""
+Struct containing version information of the CaosDB server. Meant
+mainly for internal usage; use `CaosDB.Connection.get_version_info` or
+`CaosDB.Connection.print_version_info` to retrieve the version of the
+connected CaosDB server.
+"""
+mutable struct _VersionInfo
+
+    major::Cint
+    minor::Cint
+    patch::Cint
+    pre_release::Cstring
+    build::Cstring
+
+    _VersionInfo() = new()
+
+end
+
+end # Info

src/Transaction.jl

+# ** header v3.0
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+#
+# ** end header
+#
+
+module Transaction
+
+export create_transaction,
+    add_retrieve_by_id,
+    add_query,
+    execute,
+    get_result_set,
+    get_count_result,
+    get_result_at,
+    get_results,
+    execute_query,
+    retrieve
+
+using CaosDB
+
+"""
+Struct containing a pointer to the wrapped cpp transaction
+object. Meant for internal use; call
+`CaosDB.Transaction.create_transaction` to create a transaction
+object.
+"""
+mutable struct _Transaction
+    wrapped_transaction::Ptr{Cvoid}
+    _deletable::Bool
+    function _Transaction(managed_by_julia::Bool = false)
+        trans = new()
+        trans._deletable = false
+        if managed_by_julia
+            function f(t)
+                ccall(
+                    (:caosdb_transaction_delete_transaction, CaosDB.library_name),
+                    Cint,
+                    (Ref{_Transaction},),
+                    Ref{_Transaction}(t),
+                )
+            end
+            finalizer(f, trans)
+        end
+        return trans
+    end
+end
+
+"""
+Struct containing a pointer to the wrapped cpp result set of a
+transaction. The struct is meant for internal use only and shouldn't
+be created directly by the user but is returned by, e.g.,
+`get_result_set`.
+"""
+mutable struct _ResultSet
+    wrapped_result_set::Ptr{Cvoid}
+    _deletable::Bool
+
+    function _ResultSet()
+        rs = new()
+        rs._deletable = false
+        return rs
+    end
+end
+
+"""
+    create_transaction(name::AbstractString = "default")
+
+Return a transaction created with the connection of the given name. If
+none is given, the defult connection is used.
+"""
+function create_transaction(name::AbstractString = "default")
+    connection = CaosDB.Connection.get_connection(name)
+
+    return create_transaction(connection)
+end
+
+"""
+    function create_transaction(connection::Ref{CaosDB.Connection._Connection})
+
+Return a transactioncreated with the given connection object.
+"""
+function create_transaction(connection::Ref{CaosDB.Connection._Connection})
+
+    transaction = Ref{_Transaction}(_Transaction(true))
+
+    err_code = ccall(
+        (:caosdb_connection_connection_create_transaction, CaosDB.library_name),
+        Cint,
+        (Ref{CaosDB.Connection._Connection}, Ref{_Transaction}),
+        connection,
+        transaction,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return transaction
+end
+
+"""
+    function add_retrieve_by_id(transaction::Ref{_Transaction}, id::AbstractString)
+
+Add a sub-request to retrieve a single entity by its `id` to the given
+`transaction`.
+
+!!! info
+
+    This does not execute the transaction.
+"""
+function add_retrieve_by_id(transaction::Ref{_Transaction}, id::AbstractString)
+
+    err_code = ccall(
+        (:caosdb_transaction_transaction_retrieve_by_id, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction}, Cstring),
+        transaction,
+        id,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+end
+
+"""
+    function add_retrieve_by_id(
+        transaction::Ref{_Transaction},
+        ids::Vector{T},
+    ) where {T<:AbstractString}
+
+Add a sub-request to retrieve several entities by their `ids` to the
+given `transaction`.
+
+!!! info
+
+    This does not execute the transaction.
+"""
+function add_retrieve_by_id(
+    transaction::Ref{_Transaction},
+    ids::Vector{T},
+) where {T<:AbstractString}
+
+    len = Cint(length(ids))
+    err_code = ccall(
+        (:caosdb_transaction_transaction_retrieve_by_ids, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction}, Ptr{Ptr{Cchar}}, Cint),
+        transaction,
+        ids,
+        len,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+end
+
+
+"""
+    function add_query(transaction::Ref{_Transaction}, query::AbstractString)
+
+Add a query sub-request to the given `transaction`.
+
+!!! warning
+
+    Only COUNT queris and FIND queries (and no SELECT queries) are
+    currently supported.
+
+!!! info
+
+    This does not execute the transaction
+"""
+function add_query(transaction::Ref{_Transaction}, query::AbstractString)
+
+    err_code = ccall(
+        (:caosdb_transaction_transaction_query, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction}, Cstring),
+        transaction,
+        query,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+end
+
+"""
+    function execute(transaction::Ref{_Transaction})
+
+Send the given `transaction` to the CaosDB server for excecution and
+wait for it to finish.
+"""
+function execute(transaction::Ref{_Transaction})
+
+    err_code = ccall(
+        (:caosdb_transaction_transaction_execute, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction},),
+        transaction,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+end
+
+"""
+    function get_result_set(transaction::Ref{_Transaction})
+
+Return the result set of the given `transaction`.
+"""
+function get_result_set(transaction::Ref{_Transaction})
+
+    results = Ref{_ResultSet}(_ResultSet())
+
+    err_code = ccall(
+        (:caosdb_transaction_transaction_get_result_set, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction}, Ref{_ResultSet}),
+        transaction,
+        results,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return results
+end
+
+"""
+    function get_count_result(transaction::Ref{_Transaction})
+
+Return the number of results of the COUNT query in the given `transaction`.
+
+!!! info
+
+    This is only a meaningful quantity if there actually was a COUNT
+    query in the `transaction`, and it already has been executed. In
+    all other cases the return value will be -1.
+"""
+function get_count_result(transaction::Ref{_Transaction})
+
+    out = Ref{Clong}(0)
+
+    err_code = ccall(
+        (:caosdb_transaction_transaction_get_count_result, CaosDB.library_name),
+        Cint,
+        (Ref{_Transaction}, Ref{Clong}),
+        transaction,
+        out,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return out[]
+end
+
+"""
+    function get_result_size(results::Ref{_ResultSet})
+
+Return the size of the given `results`, i.e., the number of entities
+or other responses returned in this result set.
+"""
+function get_result_size(results::Ref{_ResultSet})
+
+    size = Ref{Cint}(0)
+
+    err_code = ccall(
+        (:caosdb_transaction_result_set_size, CaosDB.library_name),
+        Cint,
+        (Ref{_ResultSet}, Ref{Cint}),
+        results,
+        size,
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return size[]
+end
+
+"""
+    function get_result_at(results::Ref{_ResultSet}, index::Cint)
+
+Return the entity at position `index` of the given `results`.
+"""
+function get_result_at(results::Ref{_ResultSet}, index::Cint)
+
+    size = get_result_size(results)
+
+    if index > size
+        throw(
+            DomainError(
+                index,
+                "You tried to access the result at position $index but the result set only has $size results.",
+            ),
+        )
+    end
+
+    entity = Ref{CaosDB.Entity._Entity}(CaosDB.Entity._Entity())
+
+    err_code = ccall(
+        (:caosdb_transaction_result_set_at, CaosDB.library_name),
+        Cint,
+        (Ref{_ResultSet}, Ref{CaosDB.Entity._Entity}, Cint),
+        results,
+        entity,
+        index - Cint(1),
+    )
+
+    CaosDB.Exceptions.evaluate_return_code(err_code)
+
+    return entity
+end
+
+"""
+    function get_result_at(results::Ref{_ResultSet}, index::Integer)
+
+Convenience wrapper for `get_result_at(::Ref{_ResultSet},
+::Cint)`. Convert the given `index` to Int32 and return the result at
+position `index` of `results`.
+"""
+function get_result_at(results::Ref{_ResultSet}, index::Integer)
+
+    return get_result_at(results, Cint(index))
+end
+
+"""
+    function get_results(result_set::Ref{_ResultSet})
+
+Return al  results of the given `result_set`.
+"""
+function get_results(result_set::Ref{_ResultSet})
+
+    size = get_result_size(result_set)
+
+    return [get_result_at(result_set, Cint(ii)) for ii = 1:size]
+end
+
+"""
+    function get_results(transaction::Ref{_Transaction})
+
+Return all results of the given `transaction`.
+"""
+function get_results(transaction::Ref{_Transaction})
+
+    result_set = get_result_set(transaction)
+
+    return get_results(result_set)
+end
+
+"""
+    function execute_query(
+        query::AbstractString,
+        name::AbstractString = "default"
+    )
+
+Execute the given `query` and return its results. Use the connection
+with the given `name`. If none is given, the default connection is
+used.
+
+!!! info
+
+    Since only the resulting entities are returned, this only makes
+    sense for FIND (and, in the future, SELECT) queries. To get the
+    result of a `COUNT` query, you have to execute the transaction
+    yourself using `create_transaction`, `add_query`, and `execute`,
+    and get the result with `get_count_result`.
+"""
+function execute_query(query::AbstractString, name::AbstractString = "default")
+
+    transaction = create_transaction(name)
+
+    add_query(transaction, query)
+
+    execute(transaction)
+
+    return get_results(transaction)
+end
+
+"""
+    function execute_query(
+        query::AbstractString,
+        connection::Ref{CaosDB.Connection._Connection}
+    )
+
+Execute the given `query` and return its results. Use the given
+`connection`.
+
+!!! info
+
+    Since only the resulting entities are returned, this only makes
+    sense for FIND (and, in the future, SELECT) queries. To get the
+    result of a `COUNT` query, you have to execute the transaction
+    yourself using `create_transaction`, `add_query`, and `execute`,
+    and get the result with `get_count_result`.
+"""
+function execute_query(
+    query::AbstractString,
+    connection::Ref{CaosDB.Connection._Connection},
+)
+
+    transaction = create_transaction(connection)
+
+    add_query(transaction, query)
+
+    execute(transaction)
+
+    return get_results(transaction)
+end
+
+"""
+    function retrieve(id::AbstractString, name::AbstractString = "default")
+
+Retrieve and return the entity with the given `id`. Use the connection
+with the given `name`. If none is provided, the default connection is
+used.
+"""
+function retrieve(id::AbstractString, name::AbstractString = "default")
+
+    transaction = create_transaction(name)
+
+    add_retrieve_by_id(transaction, id)
+
+    execute(transaction)
+
+    return get_results(transaction)[1]
+end
+
+"""
+    function retrieve(id::AbstractString, connection::Ref{CaosDB.Connection._Connection})
+
+Retrieve and return the entity with the given `id`. Use the given
+`connection`.
+"""
+function retrieve(id::AbstractString, connection::Ref{CaosDB.Connection._Connection})
+
+    transaction = create_transaction(connection)
+
+    add_retrieve_by_id(transaction, id)
+
+    execute(transaction)
+
+    return get_results(transaction)[1]
+end
+
+"""
+    function retrieve(
+        ids::Vector{T},
+        name::AbstractString = "default",
+    ) where {T<:AbstractString}
+
+Retrieve and return the entities with the given `ids`. Use the
+connection of the given `name`. If none is provided, the default
+connection is used.
+"""
+function retrieve(
+    ids::Vector{T},
+    name::AbstractString = "default",
+) where {T<:AbstractString}
+
+    transaction = create_transaction(name)
+
+    add_retrieve_by_id(transaction, ids)
+
+    execute(transaction)
+
+    return get_results(transaction)
+end
+
+"""
+    function retrieve(
+        ids::Vector{T},
+        connection::Ref{CaosDB.Connection._Connection},
+    ) where {T<:AbstractString}
+
+Retrieve and return the entities with the given `ids`. Use the given
+`connection`.
+"""
+function retrieve(
+    ids::Vector{T},
+    connection::Ref{CaosDB.Connection._Connection},
+) where {T<:AbstractString}
+
+    transaction = create_transaction(connection)
+
+    add_retrieve_by_id(transaction, ids)
+
+    execute(transaction)
+
+    return get_results(transaction)
+end
+
+end # Transaction

src/Utility.jl

+# ** header v3.0
+# This file is a part of the CaosDB Project.
+#
+# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
+# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as
+# published by the Free Software Foundation, either version 3 of the
+# License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+#
+# ** end header
+#
+module Utility
+
+export get_env_fallback
+
+using ..CaosDB
+
+"""
+    get_env_fallback(var[, default])
+
+Return the environmental variable `var` if it exists, `default`
+otherwise. If no `default` is given an empty string is returned
+instead.
+"""
+function get_env_fallback(var::AbstractString, default::AbstractString = "")
+
+    ret = ccall(
+        (:caosdb_utility_get_env_fallback, CaosDB.library_name),
+        Cstring,
+        (Cstring, Cstring),
+        var,
+        default,
+    )
+
+    return unsafe_string(ret)
+
+end
+
+end # Utility

test/runtests.jl

@@ -30,7 +30,7 @@ using CaosDB
         else
             shell_var = "default"
         end
-        @test CaosDB.Utility.get_env_var("SHELL", "default") == shell_var
+        @test CaosDB.Utility.get_env_fallback("SHELL", "default") == shell_var
     end
 
     @testset "TestExceptions" begin
@@ -63,5 +63,114 @@ using CaosDB
                 Cint(-1),
             ),
         ) CaosDB.Exceptions.evaluate_return_code(Cint(-1))
+
+        # Check whether client errors can be built correctly and
+        # return the correct code (Change it if this ever changes in
+        # libcaosdb)
+        client_err = CaosDB.Exceptions.ClientException("Client error")
+        @test client_err.msg == "Client error"
+        @test client_err.code == 9999
+    end
+
+    @testset "TestTransaction" begin
+        # transactions can be generated from the default connection,
+        # from a connection name, or from a connection object
+        @test CaosDB.Transaction.create_transaction() != nothing
+        @test CaosDB.Transaction.create_transaction("default") != nothing
+        conn = CaosDB.Connection.get_connection()
+        @test CaosDB.Transaction.create_transaction(conn) != nothing
+
+        # Retrieval works by a single id, by a list of ids, or by querying
+        trans1 = CaosDB.Transaction.create_transaction()
+        @test CaosDB.Transaction.add_retrieve_by_id(trans1, "some_id") == nothing
+        trans2 = CaosDB.Transaction.create_transaction()
+        @test CaosDB.Transaction.add_retrieve_by_id(trans2, ["id1", "id2", "id3"]) ==
+              nothing
+        trans3 = CaosDB.Transaction.create_transaction()
+        @test CaosDB.Transaction.add_query(trans3, "FIND ENTITY WITH id=123") == nothing
+
+    end
+
+    @testset "TestEntity" begin
+        ent_with_name = CaosDB.Entity.create_entity("TestEnt")
+        @test CaosDB.Entity.get_name(ent_with_name) == "TestEnt"
+
+        rt_with_name = CaosDB.Entity.create_recordtype("TestRT")
+        @test CaosDB.Entity.get_name(rt_with_name) == "TestRT"
+        @test CaosDB.Entity.get_role(rt_with_name) == "RECORD_TYPE"
+
+        prop_with_name_and_unit =
+            CaosDB.Entity.create_property_entity(name = "TestProp", unit = "m")
+        @test CaosDB.Entity.get_name(prop_with_name_and_unit) == "TestProp"
+        @test CaosDB.Entity.get_role(prop_with_name_and_unit) == "PROPERTY"
+        @test CaosDB.Entity.get_unit(prop_with_name_and_unit) == "m"
+
+        rec_with_parent_and_props = CaosDB.Entity.create_record("TestRec")
+        # cannot set an ID
+        @test_throws CaosDB.Exceptions.ClientException CaosDB.Entity.set_id(
+            rec_with_parent_and_props,
+            "some_id",
+        )
+        # cannot set the datatype of a record
+        @test_throws CaosDB.Exceptions.ClientException CaosDB.Entity.set_datatype(
+            rec_with_parent_and_props,
+            "INTEGER",
+        )
+
+        par1 = CaosDB.Entity.create_parent(name = "Parent1")
+        par2 = CaosDB.Entity.create_parent(name = "Parent2", id = "id_of_parent_2")
+        CaosDB.Entity.append_parents(rec_with_parent_and_props, [par1, par2])
+        @test length(CaosDB.Entity.get_parents(rec_with_parent_and_props)) == 2
+        # Getting has to work with all Integers that can be cast to Cint
+        @test CaosDB.Entity.get_name(
+            CaosDB.Entity.get_parent(rec_with_parent_and_props, Cint(1)),
+        ) == "Parent1"
+        @test CaosDB.Entity.get_name(
+            CaosDB.Entity.get_parent(rec_with_parent_and_props, 2),
+        ) == "Parent2"
+        @test_throws DomainError CaosDB.Entity.get_parent(rec_with_parent_and_props, 3)
+        @test CaosDB.Entity.get_id(
+            CaosDB.Entity.get_parent(rec_with_parent_and_props, Cint(2)),
+        ) == "id_of_parent_2"
+        CaosDB.Entity.remove_parent(rec_with_parent_and_props, Cint(1))
+        @test length(CaosDB.Entity.get_parents(rec_with_parent_and_props)) == 1
+        @test CaosDB.Entity.get_name(
+            CaosDB.Entity.get_parent(rec_with_parent_and_props, Cint(1)),
+        ) == "Parent2"
+
+        prop1 = CaosDB.Entity.create_property(name = "Property1", value = "2")
+        prop2 = CaosDB.Entity.create_property(id = "id_of_property_2")
+        CaosDB.Entity.set_datatype(prop2, "TEXT")
+        prop3 = CaosDB.Entity.create_property(name = "Property3")
+        CaosDB.Entity.append_properties(rec_with_parent_and_props, [prop1, prop2, prop3])
+        @test length(CaosDB.Entity.get_properties(rec_with_parent_and_props)) == 3
+        # properties can be accessed as a list
+
+        # TODO(henrik, daniel)
+        # @test CaosDB.Entity.get_value(
+        #     CaosDB.Entity.get_properties(rec_with_parent_and_props)[1],
+        # ) == "2"
+        type, is_ref, is_list = CaosDB.Entity.get_datatype(
+            CaosDB.Entity.get_properties(rec_with_parent_and_props)[2],
+        )
+        @test type == "TEXT"
+        @test is_ref == false
+        @test is_list == false
+        @test CaosDB.Entity.get_id(
+            CaosDB.Entity.get_properties(rec_with_parent_and_props)[2],
+        ) == "id_of_property_2"
+        CaosDB.Entity.remove_property(rec_with_parent_and_props, Cint(2))
+        @test length(CaosDB.Entity.get_properties(rec_with_parent_and_props)) == 2
+        @test CaosDB.Entity.get_name(
+            CaosDB.Entity.get_properties(rec_with_parent_and_props)[2],
+        ) == "Property3"
+    end
+
+    @testset "Datatype and values" begin
+        # TODO(hernik, daniel) tests for some kinds of datatypes and
+        # values. We probably don't need all since the actual
+        # functions are already being tested in the C++ client but at
+        # least some strings, numbers, and lists thereof should be
+        # tested here, too
     end
 end