From f3e8ec1e68d4d4c274400a78941dd5d1246ef36e Mon Sep 17 00:00:00 2001 From: fspreck <f.spreckelsen@indiscale.com> Date: Fri, 16 Feb 2024 16:14:43 +0100 Subject: [PATCH] DOC: Document the individual HDF5 converter classes --- src/doc/concepts.rst | 2 ++ src/doc/converters.rst | 49 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/src/doc/concepts.rst b/src/doc/concepts.rst index b10deccd..32176b9e 100644 --- a/src/doc/concepts.rst +++ b/src/doc/concepts.rst @@ -18,6 +18,8 @@ Relevant sources in: - ``src/structure_elements.py`` +.. _ConceptConverters: + Converters ++++++++++ diff --git a/src/doc/converters.rst b/src/doc/converters.rst index 26dc287d..8cda5f2c 100644 --- a/src/doc/converters.rst +++ b/src/doc/converters.rst @@ -233,7 +233,54 @@ installed with additional optional dependencies. HDF5 Converters =============== - +For treating `HDF5 Files +<https://docs.hdfgroup.org/hdf5/develop/_s_p_e_c.html>`_, there are in total +four indivdual converters corresponding to the internal structure of HDF5 files: +the :ref:`H5FileConverter` which opens the file itself and creates further +structure elements from HDF5 groups, datasets, and included multi-dimensional +arrays that are in turn treated by the :ref:`H5GroupConverter`, the +:ref:`H5DatasetConverter`, and the :ref:`H5NdarrayConverter`, respectively. You +need to install the LinkAhead crawler with its optional ``h5crawler`` dependency +for using these converters. + + + +H5FileConverter +--------------- + +This is an extension of the +:py:class:`~caoscrawler.converters.SimpleFileConverter` class. It opens the HDF5 +file and creates children for any contained group or dataset. Additionaly, the +root-level attributes of the HDF5 file are accessible as children. + +H5GroupConverter +---------------- + +This is an extension of the +:py:class:`~caoscrawler.converters.DictElementConverter` class. Children are +created for all subgroups and datasets in this HDF5 group. Additionally, the +group-level attributes are accessible as children. + +H5DatasetConverter +------------------ + +This is an extension of the +:py:class:`~caoscrawler.converters.DictElementConverter` class. Most +importantly, it stores the array data in HDF5 dataset into +:py:class:`~caoscrawler.hdf5_converters.H5NdarrayElement` which is added to its +children, as well as the dataset attributes. + +H5NdarrayConverter +------------------ + +This converter creates a wrapper record for the contained dataset. The name of +this record needs to be specified in the cfood definition of this converter via +the ``recordname`` option. The RecordType of this record can be configured with +the ``array_recordtype_name`` option and defaults to ``H5Ndarray``. Via the +given ``recordname``, this record can be used within the cfood. Most +importantly, this record stores the internal path of this array within the HDF5 +file in a text property, the name of which can be configured with the +``internal_path_property_name`` option which defaults to ``internal_hdf5_path``. Custom Converters +++++++++++++++++ -- GitLab