DOC: added examples to documentation

README_SETUP.md

-# How to Develop and Use Libcaosdb
-
-## Dependencies
-
-* See [DEPENDENCIES.md](Dependencies.md)
-
-## Build
-
-We use [cmake](https://cmake.org) as build tool.
-
-0. clone/update the subrepo `git submodule update --init proto`
-1. `mkdir build && cd build`
-2. `conan install .. -s "compiler.libcxx=libstdc++11"`
-3. `cmake -B . ..`
-4. `cmake --build .`
-
-You may also want to install libcaosdb system-wide to
-`CMAKE_INSTALL_PREFIX/lib` by
-
-5. `cmake --install .`
-
-The default install prefix is `~/.local`. It can be set by adding
-`-DCMAKE_INSTALL_PREFIX=/path/to/install/prefix` to the first cmake
-command (3.).
-
-### How to build on MacOS
-
-Instead of the above conan command (2.) use
-
-2. `conan install .. -s "cppstd=11"`
-
-and continue as you would when building on a Linux system. You may
-have to add `build/lib/` (or, alternatively after installation,
-`CMAKE_INSTALL_PREFIX/lib`) to your `DYLD_LIBRARY_PATH` environmental
-variable.
-
-### How to build on Windows
-
-We use [Visual Studio 2019](https://visualstudio.microsoft.com/de/vs/features/cplusplus/)
-as compiler. We use [cmake](https://cmake.org/download/) as build tool.
-
-0. clone/update the subrepo `git submodule update --init proto`
-1. `mkdir build`
-2. `cd build`
-3. `conan install .. -g visual_studio -s arch=x86_64 -s build_type=Release -s compiler.toolset=v142 -s compiler.version=16 -s compiler.runtime=MD  --build=missing --update`
-4. `cmake -B . ..`
-5. open ` libcaosdb.sln` with Visual Studio, change the buildtype to `Release`
-   and build the project. (You can open Tools/Command Line/Developer Command
-   Prompt and execute `msbuild libcaosdb.sln /property:Configuration=Release`)
-
-### Creating a Local Conan Package ##
-
-Building and installing libcaosdb with Conan is just a single command: `make conan-install`
-
-For MacOS, you probably should adjust the option as mentioned above.
-
-### Troubleshooting
-
-#### `conan install` Fails Due to Missing Prebuilts
-
-When `conan install` fails during the installation of the dependencies because
-a precompiled package is not available for your specific settings, try adding
-the `--build=missing` option: `conan install .. [ other options
-] --build=missing`. This should download and compile the sources of the
-dependencies.
-
-#### cmake fails when using the Debug flag
-Depending on the clang version it might be necessary to use additionally the following flag:
-`-DCMAKE_CXX_FLAGS="-Wno-unused-parameter"`
-
-## Unit Tests
-
-### Build
-
-For the tests there is a slightly different setup required (with option `-D CMAKE_BUILD_TYPE=Debug`)
-
-1. `mkdir build && cd build/`
-2. `conan install .. -s "compiler.libcxx=libstdc++11"`
-3. `cmake -B . -D CMAKE_BUILD_TYPE=Debug ..`
-   * If your clang-format version is too old, formatting, linting etc. can be skipped:  
-     `cmake -B . -D CMAKE_BUILD_TYPE=Debug -D SKIP_LINTING=ON ..`
-4. `cmake --build .`
-
-### Run
-
-In the build directory, run `ctest`
-
-### Framework
-
-We use [GoogleTest](https://google.github.io/googletest/) for unit testing.
-
-### Coverage
-
-We use [gcov](https://gcc.gnu.org/onlinedocs/gcc/Gcov.html) and
-[lcov](https://github.com/linux-test-project/lcov) for generating test coverage
-reports.
-
-In the build directory, generate the coverage report by running
-`cmake --build . --target unit_test_coverage`.
-
-Note that this special target will run the tests again. Thus it is not
-necessary to run `ctest` prior to this target.
-
-The coverage report can be viewed in a browser by opening
-`<build_directory>/unit_test_coverage/index.html`.
-
-## Code Formatting
-
-* install clang-format on your system.
-* `clang-format -i --verbose $(find test/ src/ include/ -type f -iname "*.cpp" -o -iname "*.h" -o -iname "*.h.in")`
-
-## Naming Convention
-
-Please adhere to [Google's C++ naming conventions](https://google.github.io/styleguide/cppguide.html#Naming).
-
-## Client Configuration
-
-You can use a json file for the configuration of the client. See
-`test/test_data/test_caosdb_client.json` for an example. You may use
-`caosdb-client-configuration-schema.json` to validate your schema.
-Typically, you will need to provide the path to your SSL certificate.
-
-The client will load the configuration file from the first existing
-file in the following locations (precedence from highest to lowest):
-
-1. A file specified by the environment variable
-   `$CAOSDB_CLIENT_CONFIGURATION`.
-2. `$PWD/caosdb_client.json`
-3. `$PWD/caosdb-client.json`
-4. `$PWD/.caosdb_client.json`
-5. `$PWD/.caosdb-client.json`
-6. `$HOME/caosdb_client.json`
-7. `$HOME/caosdb-client.json`
-8. `$HOME/.caosdb_client.json`
-9. `$HOME/.caosdb-client.json`
-
-## Documentation
-
-In the build directory, run
-
-* `cmake --build . --target doc-doxygen` (generate Doxygen)
-* `cmake --build . --target doc-sphinx` (generate Sphinx)

README_SETUP.md

+doc/README_SETUP.md
\ No newline at end of file

doc/CMakeLists.txt

@@ -81,10 +81,19 @@ if (DOXYGEN_FOUND)
                 -c ${CMAKE_CURRENT_BINARY_DIR}
                 ${CMAKE_CURRENT_BINARY_DIR}
                 sphinx_out
-            DEPENDS doc-doxygen
+                DEPENDS doc-doxygen
+                Examples.rst
+                README_SETUP.md
             WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
             COMMENT "Generating API documentation with Sphinx"
-            VERBATIM )
+            VERBATIM
+            )
+        # Copying files is necessary: https://stackoverflow.com/a/45808534/232888
+        FILE(COPY
+            Examples.rst
+            README_SETUP.md
+            # Tutorial.rst
+            DESTINATION "${CMAKE_CURRENT_BINARY_DIR}")
     else ()
         message("Sphinx need to be installed to generate the sphinx documentation")
     endif ()

doc/Examples.rst

+Examples
+========
+
+
+Connect to a CaosDB server
+--------------------------
+
+See also the hints on how to :doc:`get started<README_SETUP>`, and set-up of
+libcaosdb. In order to connect to a CaosDB server with libcaosdb you first have to configure the
+connection via a configuration file as explained in the :ref:`configuration section <Client Configuration>`. Once the configuration is set up, connecting to the server is
+as easy as
+
+.. code:: cpp
+
+  const auto &connection =
+      caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+You can print the full version of the server that you are connected to (and 
+therby test the connection) via: 
+
+.. code:: cpp
+  // get version info of the server
+  connection.RetrieveVersionInfo()
+  const auto &v_info = connection->GetVersionInfo();
+  std::cout << "Server Version: " << v_info->GetMajor() << "."
+            << v_info->GetMinor() << "." << v_info->GetPatch() << "-"
+            << v_info->GetPreRelease() << "-" << v_info->GetBuild()
+            << std::endl;
+
+
+Retrieve a Record
+-----------------
+
+With libcaosdb you can create a transaction
+object, fill it with sub-requests, execute it, and retrieve the
+result(s) . This
+is handy when you want to have several requests in the same transaction.
+However, most of the time, e.g., when retrieving one or multiple
+Records, this is not necessary. libcaosdb provides helper functions so
+you don’t have to worry about transactions and their executions. Assume
+you want to retrieve an Entity with id=123. This is done via
+
+.. code:: cpp
+
+   const auto &connection =
+     caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+   auto transaction(connection->CreateTransaction());
+
+   const auto *id = "1231";
+   transaction->RetrieveById(id);
+   auto status = transaction->Execute();
+
+   const auto &result_set = transaction->GetResultSet();
+
+   const auto &entity = result_set.at(0)
+
+
+You can then use the getter methods like :cpp:any:`GetId<caosdb::entity::Entity::GetId>`,
+:cpp:any:`GetParents<caosdb::entity::Entity::GetParents>`, or :cpp:any:`GetProperties`<caosdb::entity::Entity::GetProperties>` to get the
+name, the parents, or the properties of the retrieved entity.
+
+Retrieving multiple entities works in the same way. Type
+
+.. code:: cpp
+
+  const std::vector<std::string> ids = {"1220", "1221", "1222"};
+  transaction->RetrieveById(ids.begin(), ids.end());
+
+to retrieve the entities with ids 1220, 1221 and 1222. They can then be
+accessed using ``result_set.at(1)`` etc. 
+
+The same (and more) can be achieved by building and executing the
+transaction manually. This is done by
+
+.. code:: cpp
+
+   const auto &connection =
+     caosdb::connection::ConnectionManager::GetDefaultConnection();
+   auto transaction(connection->CreateTransaction());
+
+   transaction->RetrieveById("1231");
+   transaction->RetrieveById("1233");
+   auto status = transaction->Execute();
+
+A result set can be obtained
+via :cpp:any:`GetResultSet`<caosdb::transaction::Transaction::GetResultSet>` which contains the resulting entities
+and can, e.g., be checked for length. 
+
+Execute queries
+---------------
+
+Executing queries works very similar to the retrieval of entities via
+ids. 
+
+FIND and SELECT queries
+~~~~~~~~~~~~~~~~~~~~~~~
+
+In general, entities can be found using CaosDB’s query language like
+
+.. code:: cpp
+
+  const auto &connection =
+      caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+  auto query_transaction(connection->CreateTransaction());
+  query_transaction->Query("FIND ENTITY WITH id = 1222");
+  query_transaction->Execute();
+  auto result_set = query_transaction->GetResultSet();
+  std::cout << "Found " << result_set.size() 
+            << "entity with id="
+            <<  result_set.at(0).GetId() << std::endl;
+
+``result_set`` is a (possibly empty) list of entities that can be
+inspected as above.
+
+::
+
+   SELECT queries haven't been implemented in the C++ client yet and
+   thus cannot be executed from libcaosdb right now. SELECT queries
+   will be added in a future release.
+
+COUNT queries
+~~~~~~~~~~~~~
+
+COUNT queries are different from FIND or SELECT queries in two ways.
+Firstly, they do not return entities but a single number which is why,
+secondly, they do not have a result set that could be returned.  
+The result of the count is
+obtained using the :cpp:any:`GetCountResult<caosdb::transaction::Transaction::GetCountResult>` function:
+
+.. code:: cpp
+
+  const auto &connection =
+      caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+  auto query_transaction(connection->CreateTransaction());
+  query_transaction->Query("COUNT RECORD person WITH NAME LIKE '*Baggins'");
+  query_transaction->Execute();
+  std::cout << "Found " << query_transaction->GetCountResult() 
+            << "entities."<< std::endl;
+
+Insert, update, and delete entities
+-----------------------------------
+
+Insert, update and delete  operations function the same way. The respective 
+task is added to a transaction and the transaction is executed.
+
+.. code:: cpp
+  const auto &connection =
+      caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+
+  // ######## INSERT ######## 
+  // create the entity
+  Entity entity;
+  entity.SetRole(Role::RECORD_TYPE);
+  entity.SetName("RT1");
+
+  // create the transaction, add the task and execute the transaction
+  auto insert_transaction(connection->CreateTransaction());
+  insert_transaction->InsertEntity(&entity);
+  auto insert_status = insert_transaction->Execute();
+
+  const auto &insert_result_set = insert_transaction->GetResultSet();
+  // the result is an entity with the new id
+  const auto &new_entity = insert_result_set.at(0);
+  std::cout << "The newly inserted entity has the id "
+            << new_entity.GetId() << std::endl;
+
+
+  // get the inserted entity
+  // RETRIEVE
+  auto retrieve_transaction(connection->CreateTransaction());
+  retrieve_transaction->RetrieveById(new_entity.GetId());
+  retrieve_transaction->Execute();
+  // save the entity from the result set in a new variable
+  auto update_entity(retrieve_transaction->GetResultSet().at(0));
+
+  // ######## UPDATE ######## 
+  // and change it
+  update_entity.SetName("RT1-Update");
+
+  // create update transaction
+  auto update_transaction(connection->CreateTransaction());
+  update_transaction->UpdateEntity(&update_entity);
+  auto update_status = update_transaction->Execute();
+  // same as with insert transaction, the update transaction returns an entity 
+  // with the id
+  const auto &updated_entity = update_transaction->GetResultSet().at(0);
+  std::cout << "The  entity with the id "
+            << update_entity.GetId() << "was updated." << std::endl;
+
+  // ######## DELETE ######## 
+  auto delete_transaction(connection->CreateTransaction());
+  delete_transaction->DeleteById(updated_entity .GetId());
+  auto delete_status = delete_transaction->Execute();
+
+  // again, the delete transaction returns a result set with entities
+  // with the ids that where deleted
+  const auto &delete_result_set = delete_transaction->GetResultSet();
+
+
+Up- and Download a file
+---------------
+
+.. code:: cpp
+
+  const auto &connection =
+      caosdb::connection::ConnectionManager::GetDefaultConnection();
+
+  Entity file;
+  file.SetRole(Role::FILE);
+  file.SetFilePath("test.txt");
+  file.SetLocalPath(test_upload_file_1);
+
+  // in order to upload a file, the corresponding entity simply has to be 
+  // inserted
+  auto insert_transaction(connection->CreateTransaction());
+  insert_transaction->InsertEntity(&file);
+  insert_transaction->Execute();
+
+  // entity in the result set contains the new id
+  const auto &insert_results = insert_transaction->GetResultSet();
+  const auto &inserted_file = insert_results.at(0);
+
+  // for the download you need to use the RetrieveAndDownloadFilesById task and
+  // supply the path where the file shall be stored
+  test_download_file = fs::path("test_download_file_delete_me.dat");
+  auto download_transaction(connection->CreateTransaction());
+  download_transaction->RetrieveAndDownloadFilesById(
+      inserted_file.GetId(), test_download_file.string());
+  download_transaction->ExecuteAsynchronously();
+  download_transaction->WaitForIt().GetCode()
+
+  const auto &download_results = download_transaction->GetResultSet();
+  const auto &downloaded_file = download_results.at(0);

doc/README_SETUP.md

+# How to Develop and Use Libcaosdb
+
+## Dependencies
+
+* See [DEPENDENCIES.md](Dependencies.md)
+
+## Build
+
+We use [cmake](https://cmake.org) as build tool.
+
+0. clone/update the subrepo `git submodule update --init proto`
+1. `mkdir build && cd build`
+2. `conan install .. -s "compiler.libcxx=libstdc++11"`
+3. `cmake -B . ..`
+4. `cmake --build .`
+
+You may also want to install libcaosdb system-wide to
+`CMAKE_INSTALL_PREFIX/lib` by
+
+5. `cmake --install .`
+
+The default install prefix is `~/.local`. It can be set by adding
+`-DCMAKE_INSTALL_PREFIX=/path/to/install/prefix` to the first cmake
+command (3.).
+
+### How to build on MacOS
+
+Instead of the above conan command (2.) use
+
+2. `conan install .. -s "cppstd=11"`
+
+and continue as you would when building on a Linux system. You may
+have to add `build/lib/` (or, alternatively after installation,
+`CMAKE_INSTALL_PREFIX/lib`) to your `DYLD_LIBRARY_PATH` environmental
+variable.
+
+### How to build on Windows
+
+We use [Visual Studio 2019](https://visualstudio.microsoft.com/de/vs/features/cplusplus/)
+as compiler. We use [cmake](https://cmake.org/download/) as build tool.
+
+0. clone/update the subrepo `git submodule update --init proto`
+1. `mkdir build`
+2. `cd build`
+3. `conan install .. -g visual_studio -s arch=x86_64 -s build_type=Release -s compiler.toolset=v142 -s compiler.version=16 -s compiler.runtime=MD  --build=missing --update`
+4. `cmake -B . ..`
+5. open ` libcaosdb.sln` with Visual Studio, change the buildtype to `Release`
+   and build the project. (You can open Tools/Command Line/Developer Command
+   Prompt and execute `msbuild libcaosdb.sln /property:Configuration=Release`)
+
+### Creating a Local Conan Package ##
+
+Building and installing libcaosdb with Conan is just a single command: `make conan-install`
+
+For MacOS, you probably should adjust the option as mentioned above.
+
+### Troubleshooting
+
+#### `conan install` Fails Due to Missing Prebuilts
+
+When `conan install` fails during the installation of the dependencies because
+a precompiled package is not available for your specific settings, try adding
+the `--build=missing` option: `conan install .. [ other options
+] --build=missing`. This should download and compile the sources of the
+dependencies.
+
+#### cmake fails when using the Debug flag
+Depending on the clang version it might be necessary to use additionally the following flag:
+`-DCMAKE_CXX_FLAGS="-Wno-unused-parameter"`
+
+## Unit Tests
+
+### Build
+
+For the tests there is a slightly different setup required (with option `-D CMAKE_BUILD_TYPE=Debug`)
+
+1. `mkdir build && cd build/`
+2. `conan install .. -s "compiler.libcxx=libstdc++11"`
+3. `cmake -B . -D CMAKE_BUILD_TYPE=Debug ..`
+   * If your clang-format version is too old, formatting, linting etc. can be skipped:  
+     `cmake -B . -D CMAKE_BUILD_TYPE=Debug -D SKIP_LINTING=ON ..`
+4. `cmake --build .`
+
+### Run
+
+In the build directory, run `ctest`
+
+### Framework
+
+We use [GoogleTest](https://google.github.io/googletest/) for unit testing.
+
+### Coverage
+
+We use [gcov](https://gcc.gnu.org/onlinedocs/gcc/Gcov.html) and
+[lcov](https://github.com/linux-test-project/lcov) for generating test coverage
+reports.
+
+In the build directory, generate the coverage report by running
+`cmake --build . --target unit_test_coverage`.
+
+Note that this special target will run the tests again. Thus it is not
+necessary to run `ctest` prior to this target.
+
+The coverage report can be viewed in a browser by opening
+`<build_directory>/unit_test_coverage/index.html`.
+
+## Code Formatting
+
+* install clang-format on your system.
+* `clang-format -i --verbose $(find test/ src/ include/ -type f -iname "*.cpp" -o -iname "*.h" -o -iname "*.h.in")`
+
+## Naming Convention
+
+Please adhere to [Google's C++ naming conventions](https://google.github.io/styleguide/cppguide.html#Naming).
+
+## Client Configuration
+
+You can use a json file for the configuration of the client. See
+`test/test_data/test_caosdb_client.json` for an example. You may use
+`caosdb-client-configuration-schema.json` to validate your schema.
+Typically, you will need to provide the path to your SSL certificate.
+
+The client will load the configuration file from the first existing
+file in the following locations (precedence from highest to lowest):
+
+1. A file specified by the environment variable
+   `$CAOSDB_CLIENT_CONFIGURATION`.
+2. `$PWD/caosdb_client.json`
+3. `$PWD/caosdb-client.json`
+4. `$PWD/.caosdb_client.json`
+5. `$PWD/.caosdb-client.json`
+6. `$HOME/caosdb_client.json`
+7. `$HOME/caosdb-client.json`
+8. `$HOME/.caosdb_client.json`
+9. `$HOME/.caosdb-client.json`
+
+## Documentation
+
+In the build directory, run
+
+* `cmake --build . --target doc-doxygen` (generate Doxygen)
+* `cmake --build . --target doc-sphinx` (generate Sphinx)

doc/conf.py.in

@@ -45,7 +45,8 @@ extensions = [
     'sphinx.ext.viewcode',
     'sphinx_sitemap',
     'sphinx.ext.inheritance_diagram',
-    'breathe'
+    'breathe',
+    "recommonmark"            # For markdown files.
 ]
 
 # Add any paths that contain templates here, relative to this directory.

doc/index.rst.in

@@ -25,13 +25,13 @@
 Welcome to |PROJECT_NAME|'s documentation!
 ==========================================
 
-This is work in progress.
-
 .. toctree::
     :maxdepth: 4
     :caption: Contents:
 
     Welcome <self>
+    Getting Started <README_SETUP>
+    Examples.rst
     cppapi/index
     capi/index
 

doc/requirements.txt

@@ -2,6 +2,7 @@ alabaster==0.7.12
 Babel==2.9.1
 breathe==4.30.0
 certifi==2020.12.5
+recommonmark
 chardet==4.0.0
 docutils==0.16
 idna==2.10