Skip to content
Snippets Groups Projects
Commit c67392d8 authored by Florian Spreckelsen's avatar Florian Spreckelsen Committed by Henrik tom Wörden
Browse files

ENH: Add minimal wrapping of libcaosdb's C interface

parent 1ae0102f
No related branches found
No related tags found
1 merge request!2ENH: Add minimal wrapping of libcaosdb's C interface
FROM julia:1.6
RUN echo 'deb http://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/buster-backports.list
RUN apt-get update && apt-get install -y git python3-pip
RUN apt-get install -y cmake/buster-backports
RUN pip3 install conan
#!/bin/bash
git clone https://gitlab.indiscale.com/caosdb/src/caosdb-cpplib.git
cd caosdb-cpplib
git checkout f-extern-c
git submodule update --init --recursive
mkdir build
cd build
conan install .. -s "compiler.libcxx=libstdc++11"
cmake ..
cmake --build .
cmake --install .
cd ../..
......@@ -14,10 +14,13 @@
# [2]: https://docs.julialang.org/en/v1/manual/documentation/index.html
variables:
CI_REGISTRY_IMAGE: $CI_REGISTRY/caosdb/src/caosdb-julialib/testenv:latest
JULIALIB_REGISTRY_IMAGE: $CI_REGISTRY/caosdb/src/caosdb-julialib/testenv:$CI_COMMIT_REF_NAME
image: $JULIALIB_REGISTRY_IMAGE
stages:
- code-style
- setup
- test
- deploy
......@@ -45,12 +48,22 @@ code-style:
end'
allow_failure: true
# Install libcaosdb in docker image
setup:
tags: [ cached-dind ]
image: docker:20.10
stage: setup
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker pull $JULIALIB_REGISTRY_IMAGE || true
- docker build
--file .docker/Dockerfile
--pull
--tag $JULIALIB_REGISTRY_IMAGE .
- docker push $JULIALIB_REGISTRY_IMAGE
# Name a test and select an appropriate image.
# images comes from Docker hub
test:1.6:
test:
stage: test
image: julia:1.6
# Use `docker` runners
tags: [ docker ]
# Uncomment below if you would like to run the tests on specific
......@@ -60,6 +73,8 @@ test:1.6:
# - master
# - development
script:
- .docker/install_cpplib.sh
- export LD_LIBRARY_PATH=/root/.local/lib:$LD_LIBRARY_PATH
# Let's run the tests. Substitute `coverage = false` below, if you
# do not want coverage results.
- julia -e 'using Pkg; Pkg.add(path=pwd());
......@@ -84,11 +99,9 @@ test:1.6:
# Example documentation deployment
pages:
tags: [ cached-dind ]
image: julia:1.6
tags: [ docker ]
stage: deploy
script:
- apt-get update -qq && apt-get install -y git # needed by Documenter
- julia -e 'using Pkg; Pkg.add(path=pwd()); Pkg.build("CaosDB");' # rebuild Julia (can be put somewhere else I'm sure)
- julia -e 'using Pkg; import CaosDB; Pkg.add("Documenter")' # install Documenter
- julia --color=yes docs/make.jl # make documentation
......
# Summary
Insert a meaningful description for this merge request here. What is the
new/changed behavior? Which bug has been fixed? Are there related Issues?
# Focus
Point the reviewer to the core of the code change. Where should they start
reading? What should they focus on (e.g. security, performance,
maintainability, user-friendliness, compliance with the specs, finding more
corner cases, concrete questions)?
# Test Environment
How to set up a test environment for manual testing?
# Check List for the Author
Please, prepare your MR for a review. Be sure to write a summary and a
focus and create gitlab comments for the reviewer. They should guide the
reviewer through the changes, explain your changes and also point out open
questions. For further good practices have a look at [our review
guidelines](https://gitlab.com/caosdb/caosdb/-/blob/dev/REVIEW_GUIDELINES.md)
- [ ] All automated tests pass
- [ ] Reference related Issues
- [ ] Up-to-date CHANGELOG.md
- [ ] Annotations in code (Gitlab comments)
- Intent of new code
- Problems with old code
- Why this implementation?
# Check List for the Reviewer
- [ ] I understand the intent of this MR
- [ ] All automated tests pass
- [ ] Up-to-date CHANGELOG.md
- [ ] The test environment setup works and the intended behavior is
reproducible in the test environment
- [ ] In-code documentation and comments are up-to-date.
- [ ] Check: Are there spezifications? Are they satisfied?
For further good practices have a look at [our review guidelines](https://gitlab.com/caosdb/caosdb/-/blob/dev/REVIEW_GUIDELINES.md).
/assign me
/target_branch dev
......@@ -9,6 +9,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
* Basic functionality to establish connection to a CaosDB server and
retrieve its version (using the Extern C interface of caosdb-cpplib)
### Changed
### Deprecated
......
name = "CaosDB"
uuid = "091fdcf5-a163-4d8f-97f4-9adce40cd04e"
authors = ["florian <f.spreckelsen@inidscale.com>"]
version = "0.1.0"
version = "0.0.1"
# Set-up caosdb-julialib
# Set-up CaosDB.jl
**TODO**: Explain installation (with Pkg.jl), running tests, dependencies,
etc.
## Installation
CaosDB.jl uses CaosDB's C interface which is accessed via `ccall`. So
to us CaosDB.jl, you must first build and install
[caosdb-cpplib](https://gitlab.com/caosdb/caosdb-cpplib) as explained
in its
[documentation](https://docs.indiscale.com/caosdb-cpplib). After
building the path to the shared library `libccaosdb` has to be made
known to Julia's
[`ccall`](https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code),
e.g., by adding its path to the `LD_LIBRARY_PATH`. Then, the CaosDB.jl
package can be accessed by
```julia-repl
julia> ]add "path/to/caosdb-julialib"
julia> using CaosDB
julia> connection = CaosDB.Connection.connect() # Enter host, port, path to SSL certificate, and credentials here
```
## Tests
After installing, the unit tests can be executed by
```julia-repl
julia> ]activate "/path/to/caosdb-julialib"
julia> ]test
```
## Documentation
The documentation can be built locally using
[Documenter.jl](https://github.com/JuliaDocs/Documenter.jl) by
executing
```sh
julia --color=yes docs/make.jl
```
Afterwards the built html files can be found in `docs/build`.
# This file is machine-generated - editing it directly is not advised
[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
[[DocStringExtensions]]
deps = ["LibGit2"]
git-tree-sha1 = "a32185f5428d3986f47c2ab78b1f216d5e6cc96f"
uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
version = "0.8.5"
[[Documenter]]
deps = ["Base64", "Dates", "DocStringExtensions", "IOCapture", "InteractiveUtils", "JSON", "LibGit2", "Logging", "Markdown", "REPL", "Test", "Unicode"]
git-tree-sha1 = "47f13b6305ab195edb73c86815962d84e31b0f48"
uuid = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
version = "0.27.3"
[[IOCapture]]
deps = ["Logging", "Random"]
git-tree-sha1 = "f7be53659ab06ddc986428d3a9dcc95f6fa6705a"
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
version = "0.2.2"
[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "81690084b6198a2e1da36fcfda16eeca9f9f24e4"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.1"
[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"
[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "c8abc88faa3f7a3950832ac5d6e690881590d6dc"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "1.1.0"
[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
[deps]
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
......@@ -6,7 +6,7 @@ library.
Manually add a docstring here:
```@docs
dummy_func(x)
CaosDB.Utility.get_env_var(var, default)
```
......
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public
# License along with this program. If not, see
# <https://www.gnu.org/licenses/>.
#
# ** end header
#
module CaosDB
export dummy_func
module Info
"""
Struct containing version information of the CaosDB server. Meant
mainly for internal usage; use `CaosDB.Connection.get_version_info` or
`CaosDB.Connection.print_version_info` to retrieve the version of the
connected CaosDB server.
"""
mutable struct _VersionInfo
major::Cint
minor::Cint
patch::Cint
pre_release::Cstring
build::Cstring
_VersionInfo() = new()
end
end # Info
module Utility
"""
get_env_var(var[, default])
Return the environmental variable `var` if it exists, `default`
otherwise. If no `default` is given an empty string is returned
instead.
"""
function get_env_var(var::AbstractString, default::AbstractString = "")
ret = ccall(
(:caosdb_utility_get_env_var, "libccaosdb"),
Cstring,
(Cstring, Cstring),
var,
default,
)
return unsafe_string(ret)
end
end # Utility
module Authentication
"""
Struct containing a pointer to the wrapped cpp authenticator
class. Meant for internal use; call a
`CaosDB.Authentication.create_<authenticator>` function to create an
authenticator object from a configuration.
"""
mutable struct _Authenticator
wrapped_authenticator::Ptr{Cvoid}
function _Authenticator()
auth = new()
# force this to point to C_NULL after initialization
auth.wrapped_authenticator = C_NULL
function f(t)
if t.wrapped_authenticator != C_NULL
# Only if pointer was filled after real initialization
ccall(
(:caosdb_authentication_delete_authenticator, "libccaosdb"),
Cint,
(Ref{_Authenticator},),
Ref{_Authenticator}(t),
)
end
end
finalizer(f, auth)
end
end
"""
dummy_func(x)
create_plain_password_authenticator(
username::AbstractString,
password::AbstractString,
)
Returns double the number `x` plus `1`.
Return an authenticator object that contains a wrapped cpp
plain-password authenticator configured with `username` and
`password`.
"""
dummy_func(x) = 2x + 1
function create_plain_password_authenticator(
username::AbstractString,
password::AbstractString,
)
auth = Ref{_Authenticator}(_Authenticator())
err_code = ccall(
(:caosdb_authentication_create_plain_password_authenticator, "libccaosdb"),
Cint,
(Ref{_Authenticator}, Cstring, Cstring),
auth,
username,
password,
)
if err_code != 0
@error "Creating authenticator failed with code $err_code"
end
return auth
end
end # Authentication
module Connection
using ..CaosDB
export connect
"""
Struct containing the actual connection to a CaosDB server. Meant for
internal use; call a `CaosDB.Connection.create_<connection>` function
to create an connection object from a configuration.
"""
mutable struct _Connection
wrapped_connection::Ptr{Cvoid}
function _Connection()
conn = new()
conn.wrapped_connection = C_NULL
function f(t)
if t.wrapped_connection != C_NULL
ccall(
(:caosdb_connection_delete_connection, "libccaosdb"),
Cint,
(Ref{_Connection},),
Ref{_Connection}(t),
)
end
end
finalizer(f, conn)
end
end
"""
Struct containing a pointer to the wrapped cpp class providing the
certificate provider. Meant for internal use; call a
`CaosDB.Connection.create_<certificate_provider>` function to create
an certificate-provider object from a configuration.
"""
mutable struct _CertificateProvider
wrapped_certificate_provider::Ptr{Cvoid}
function _CertificateProvider()
prov = new()
prov.wrapped_certificate_provider = C_NULL
function f(t)
if t.wrapped_certificate_provider != C_NULL
ccall(
(:caosdb_connection_delete_certificate_provider, "libccaosdb"),
Cint,
(Ref{_CertificateProvider},),
Ref{_CertificateProvider}(t),
)
end
end
finalizer(f, prov)
end
end
"""
Struct containing a pointer to the wrapped cpp class for storing the
connection configuration. Meant for internal use; call a
`CaosDB.Connection.create_<configuration>` function to create
an connection-configuration object from a configuration.
"""
mutable struct _Configuration
wrapped_connection_configuration::Ptr{Cvoid}
function _Configuration()
config = new()
config.wrapped_connection_configuration = C_NULL
function f(t)
if t.wrapped_connection_configuration != C_NULL
ccall(
(:caosdb_connection_delete_connection_configuration, "libccaosdb"),
Cint,
(Ref{_Configuration},),
Ref{_Configuration}(t),
)
end
end
finalizer(f, config)
end
end
"""
create_pem_file_certificate_provider(path::AbstractString)
Return a `_CertificateProvider` for the pem certificate located at
`path`.
"""
function create_pem_file_certificate_provider(path::AbstractString)
cert_provider = Ref{_CertificateProvider}(_CertificateProvider())
err_code = ccall(
(:caosdb_connection_create_pem_file_certificate_provider, "libccaosdb"),
Cint,
(Ref{_CertificateProvider}, Cstring),
cert_provider,
path,
)
if err_code != 0
@error "PEM certificate creation returned code $err_code."
end
return cert_provider
end
"""
create_tls_connection_configuration(
host::AbstractString,
port::Cint,
authenticator::Ref{CaosDB.Authentication._Authenticator},
provider::Ref{_CertificateProvider}
)
Return a TLS connection configuration with authentication.
"""
function create_tls_connection_configuration(
host::AbstractString,
port::Cint,
authenticator::Ref{CaosDB.Authentication._Authenticator},
provider::Ref{_CertificateProvider},
)
config = Ref{_Configuration}(_Configuration())
err_code = ccall(
(:caosdb_connection_create_tls_connection_configuration, "libccaosdb"),
Cint,
(
Ref{_Configuration},
Cstring,
Cint,
Ref{CaosDB.Authentication._Authenticator},
Ref{_CertificateProvider},
),
config,
host,
port,
authenticator,
provider,
)
if err_code != 0
@error "TLS-configuration creation returned code $err_code."
end
return config
end
function create_insecure_connection_configuration(host::AbstractString, port::Cint)
config = Ref{_Configuration}(_Configuration())
err_code = ccall(
(:caosdb_connection_create_insecure_connection_configuration, "libccaosdb"),
Cint,
(Ref{_Configuration}, Cstring, Cint),
config,
host,
port,
)
if err_code != 0
@error "Insecure configuration creation returned code $err_code."
end
return config
end
"""
create_connection(config::Ref{_Configuration})
Return a connection based on the given `config`.
"""
function create_connection(config::Ref{_Configuration})
connection = Ref{_Connection}(_Connection())
err_code = ccall(
(:caosdb_connection_create_connection, "libccaosdb"),
Cint,
(Ref{_Connection}, Ref{_Configuration}),
connection,
config,
)
if err_code != 0
@error "Creating connection failed with code $err_code."
end
return connection
end
"""
get_version_info(con::Ref{_Connection})
Return the version of the CaosDB server that `con` is connected
to.
"""
function get_version_info(con::Ref{_Connection})
info = Ref{CaosDB.Info._VersionInfo}(CaosDB.Info._VersionInfo())
err_code = ccall(
(:caosdb_connection_get_version_info, "libccaosdb"),
Cint,
(Ref{CaosDB.Info._VersionInfo}, Ref{_Connection}),
info,
con,
)
# TODO Real error-code handling
if err_code != 0
@error "Version info returned with code $err_code"
end
return info
end
"""
print_version_info(con::Ref{_Connection})
Retrieve the version info for the CaosDB server `con` is connected to,
and print the version in a nice message.
"""
function print_version_info(con::Ref{_Connection})
# Dereference to access the fields
info = get_version_info(con)[]
major = info.major
minor = info.minor
patch = info.patch
pre_release_str = unsafe_string(info.pre_release)
build_str = unsafe_string(info.build)
println(
"Connected to a CaosDB server with version $major.$minor.$patch-$pre_release_str-$build_str.",
)
end
"""
function connect([;
host::AbstractString="",
port_str::AbstractString="undefined",
cacert::AbstractString="",
username::AbstractString="",
password::AbstractString="undefined"]
)
Return a connection object created for the given `host`:`port` with an
SSL certificate located at `cacert` with the given credentials.
# Extended help
!!! info
Because of type-stability, and since an empty string may be a
valid password, the value of `password`, for which it is fetched
from an environmental variable, is "undefined". This means that if
you absolutely must use "undefined" as your password, you have to
specify it via the `CAOSDB_PASSWORD` variable.
# Arguments
- `host::AbstractString=""`: The hostname of the CaosDB server. If
none is provided, the `CAOSDB_SERVER_HOST` environmental variable is
used instead. If that's not defined, "localhost" is used.
- `port_str::AbstractString="undefined"`: The port of the CaosDB
server, given as string. If none is provided, the
`CAOSDB_SERVER_GRPC_PORT_HTTPS` environmental variable is used
instead. If that's not defined, "8443" is used. The default value is
"undefined" rather than an empty string because an empty string
could be a valid port, too, i.e. the CaosDB server is available at
`host` without a port.
- `cacert::AbstractString=""`: The path to the SSL certificate of the
CaosDB server. If none is provided, the `CAOSDB_SERVER_CERT`
environmental variable is used instead.
- `username::AbstractString=""`: The username with which to log in
into the CaosDB server. If none is provided, the `CAOSDB_USER`
environmental variable is used instead. If that's not defined,
"admin" is used.
- `password::AbstractString="undefined"`: The password with which to
log in into the CaosDB server. If none is provided, the
`CAOSDB_PASSWORD` environmental variable is used instead. If that's
not defined, "caosdb" is used. The default value is "undefined"
rather than an empty string to allow an empty password.
"""
function connect(;
host::AbstractString = "",
port_str::AbstractString = "undefined",
cacert::AbstractString = "",
username::AbstractString = "",
password::AbstractString = "undefined",
)
if host == ""
host = CaosDB.Utility.get_env_var("CAOSDB_SERVER_HOST", "localhost")
end
if port_str == "undefined"
port_str = CaosDB.Utility.get_env_var("CAOSDB_SERVER_GRPC_PORT_HTTPS", "8443")
end
port = parse(Cint, port_str)
if cacert == ""
cacert = CaosDB.Utility.get_env_var("CAOSDB_SERVER_CERT")
end
if username == ""
username = CaosDB.Utility.get_env_var("CAOSDB_USER", "admin")
end
if password == "undefined"
password = CaosDB.Utility.get_env_var("CAOSDB_PASSWORD", "caosdb")
end
provider = create_pem_file_certificate_provider(cacert)
authenticator =
CaosDB.Authentication.create_plain_password_authenticator(username, password)
config = create_tls_connection_configuration(host, port, authenticator, provider)
connection = create_connection(config)
print_version_info(connection)
return connection
end
end # Connection
module Entity end
end # module
end # CaosDB
# ** header v3.0
# This file is a part of the CaosDB Project.
#
# Copyright (C) 2021 Indiscale GmbH <info@indiscale.com>
# Copyright (C) 2021 Florian Spreckelsen <f.spreckelsen@indiscale.com>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public
# License along with this program. If not, see
# <https://www.gnu.org/licenses/>.
#
# ** end header
#
using Test
using CaosDB
println("Testing...")
@test CaosDB.dummy_func(1) == 3
if haskey(ENV, "SHELL")
shell_var = ENV["SHELL"]
else
shell_var = "default"
end
@test CaosDB.Utility.get_env_var("SHELL", "default") == shell_var
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment