From 0bb772b8ea6a61d0da559f08deb201bd86da2f45 Mon Sep 17 00:00:00 2001
From: Daniel <d.hornung@indiscale.com>
Date: Wed, 22 May 2024 08:48:37 +0200
Subject: [PATCH] DOC: Macros

---
 src/doc/macros.rst | 71 +++++++++++++++++++++++++++++++++-------------
 1 file changed, 52 insertions(+), 19 deletions(-)

diff --git a/src/doc/macros.rst b/src/doc/macros.rst
index d093d9b6..3a234973 100644
--- a/src/doc/macros.rst
+++ b/src/doc/macros.rst
@@ -1,6 +1,9 @@
 Macros
 ------
 
+Introduction
+============
+
 Macros highly facilitate the writing of complex :doc:`CFoods<cfood>`. Consider the following common
 example:
 
@@ -83,16 +86,46 @@ The expanded version of `ExperimentalData` will look like:
 This :ref:`example<example_files_2>` can also be found in the macro unit tests (see :func:`unittests.test_macros.test_documentation_example_2`).
 
 
-Complex Example
-===============
 
-The following, more complex example, demonstrates the use
-of macro variable substitutions that generate crawler variable substitutions:
+Mixing macros and plain definitions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can also mix macros and plain definitions.  Whenever a name cannot be resolved to a macro, a
+plain yaml node definition is used as a fallback:
+
+.. code:: yaml
+
+  ---
+  metadata:
+    macros:
+    - !defmacro
+      name: MarkdownFile
+  # ... Definition here ...
+  ---
+  ExperimentalData:
+    type: Directory
+    match: ExperimentalData
+    subtree: !macro
+      MarkdownFile:
+      - name: README
+        filename: ^README.md$
+      OtherContent:  # There is no macro named "OtherContent", so this is parsed as normal content.
+        type: SimpleFile
+        match: .*txt
+        records:
+          # ... Normal content ...
+
 
-- `$$$nodename` will lead to a macro variable substitution of variable `$nodename` during macro expansion.
-- `$$` will be turned into `$`
-- So in the crawler cfood, the string will appear as `$value` if variable `nodename` would be set to `value` when using the macro.
+Complex example
+===============
+
+Let's try something more complex: what happens to multiple ``$``?  This example demonstrates the use
+of `macro` variable substitutions to generate `crawler` variable substitutions:
 
+- ``$$`` will be converted into ``$``.
+- ``$$$nodename`` will retain a single ``$`` and substitute ``$nodename`` during macro expansion.
+- So in the cfood, if ``nodename: value``, the string ``$$$nodename`` will be converted to
+  ``$value``.
 
 .. _example_1:
 .. code-block:: yaml
@@ -118,7 +151,8 @@ of macro variable substitutions that generate crawler variable substitutions:
              Simulation:
                $recordtype: +$File
 
-The expanded version of :ref:`example<example_1>` can be seen in :ref:`example<example_1_expanded>`.
+The expanded version of the :ref:`example above<example_1>` (with ``nodename: Dataset``) can be seen
+:ref:`here<example_1_expanded>`:
 
 
 .. _example_1_expanded:
@@ -141,11 +175,11 @@ The expanded version of :ref:`example<example_1>` can be seen in :ref:`example<e
         type: SimpleFile
     type: Directory
 
-This :ref:`example<example_1>` can also be found in the macro unit tests (see :func:`unittests.test_macros.test_documentation_example_1`).
-
+This example can also be found in the macro unit tests (see
+:func:`unittests.test_macros.test_documentation_example_1`).
 
 
-Using Macros Multiple Times
+Using macros multiple times
 ===========================
 
 To use the same macro multiple times in the same yaml node, lists can be used:
@@ -198,11 +232,11 @@ use the same top level key. Because later versions would overwrite previous
 ones. Here we used ``$macro_name`` to prevent that.
 
 
-Limitation
-==========
+Limitations
+===========
 
-Currently it is not possible to use the same macro twice in the same yaml node, but in different
-positions. Consider:
+Currently it is not possible to use the same macro twice in the same yaml node, if it occurs in
+different positions. Consider:
 
 .. _example_multiple_limitation:
 .. code-block:: yaml
@@ -227,14 +261,13 @@ positions. Consider:
       Other_node:
         type: test
         
-      test_twice:  # This is NOT possible as each
-                   #  dictionary element can only appear once in a yaml node.
+      test_twice:  # This is NOT possible as each key
+                   # can only appear once in a yaml node.
       - macro_name: twice # <- This is the second one, with different arguments
         a: 5
       - {}                # <- This is the third one, just using default arguments
 
-However, this should not be a real limitation, as the crawler is designed in a way,
-that the order of the nodes in the same level should not matter.
+This should not be a real limitation however, as the order of nodes does not matter for the crawler.
 
 
 Using macros within macro definitions
-- 
GitLab