Draft: ENH: file system: directory

Summary

Insert a meaningful description for this merge request here: What is the new/changed behavior? Which bug has been fixed? Are there related issues?

Focus

Point the reviewer to the core of the code change. Where should they start reading? What should they focus on (e.g. security, performance, maintainability, user-friendliness, compliance with the specs, finding more corner cases, concrete questions)?

Test Environment

How to set up a test environment for manual testing?

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

patches/patch20240127-8.0-SNAPSHOT/patch.sql

 -- USE _caosdb_schema_unit_tests;
 
-
--- REMOVE SQLITE datatype (obsolete)
-DELETE FROM name_data WHERE entity_id = 50;
-DELETE FROM entities WHERE id = 50;
-
 /*
 -- Create file_hashes table.
 -- This table is necessary because not all FSO objects have hashes (e.g.
@@ -40,8 +35,8 @@ ALTER TABLE archive_files DROP COLUMN hash;
 
 -- Add new fields to files table...
 ALTER TABLE files ADD COLUMN IF NOT EXISTS (
-    hash_algorithm VARBINARY(255) NULL DEFAULT "SHA-512",
-    -- parent_directory INT UNSIGNED DEFAULT NULL,
+    hash_algorithm VARBINARY(255) NULL DEFAULT 'SHA-512',
+    parent_directory INT UNSIGNED DEFAULT NULL,
     mimetype VARBINARY(255) DEFAULT NULL,
     file_storage_id VARBINARY(255) NOT NULL DEFAULT "DEFAULT",
     file_key VARBINARY(16000) DEFAULT NULL
@@ -50,8 +45,8 @@ ALTER TABLE files ADD COLUMN IF NOT EXISTS (
 -- ... and to the corresponding archive_files table.
 ALTER TABLE archive_files ADD COLUMN IF NOT EXISTS (
     checked_timestamp BIGINT NOT NULL DEFAULT 0,
-    hash_algorithm VARBINARY(255) NULL DEFAULT "SHA-512",
-    -- parent_directory INT UNSIGNED DEFAULT NULL,
+    hash_algorithm VARBINARY(255) NULL DEFAULT 'SHA-512',
+    parent_directory INT UNSIGNED DEFAULT NULL,
     mimetype VARBINARY(255) DEFAULT NULL,
     file_storage_id VARBINARY(255) NOT NULL DEFAULT "DEFAULT",
     file_key VARBINARY(16000) DEFAULT NULL
@@ -60,15 +55,16 @@ ALTER TABLE archive_files ADD COLUMN IF NOT EXISTS (
 -- no two files at the same path are allowed.
 ALTER TABLE files ADD UNIQUE (`path`);
 -- directories cannot be deleted without deleting all the content first.
--- ALTER TABLE files ADD FOREIGN KEY (parent_directory) REFERENCES entities (id);
--- ALTER TABLE archive_files ADD FOREIGN KEY (parent_directory) REFERENCES entities (id) ON DELETE CASCADE;
+ALTER TABLE files ADD FOREIGN KEY (parent_directory) REFERENCES entities (id);
+ALTER TABLE archive_files ADD FOREIGN KEY (parent_directory) REFERENCES entities (id) ON DELETE CASCADE;
 
 
--- ALTER TABLE entities MODIFY COLUMN `role` enum('RECORDTYPE','RECORD','FILE','DOMAIN','PROPERTY','DATATYPE','ROLE','QUERYTEMPLATE', 'DIRECTORY') COLLATE utf8_unicode_ci NOT NULL;
+ALTER TABLE entities MODIFY COLUMN `role` enum('RECORDTYPE','RECORD','FILE','_REPLACEMENT','PROPERTY','DATATYPE','ROLE','QUERYTEMPLATE', 'DIRECTORY') COLLATE utf8_unicode_ci NOT NULL;
 
 -- Add a new special role "DIRECTORY"
--- INSERT IGNORE INTO entities (id, description, role, acl) VALUES (9, "The directory role.", "ROLE", 0);
--- INSERT IGNORE INTO name_data (domain_id, entity_id, property_id, value, status, pidx) VALUES (0, 9, 20, "DIRECTORY", "FIX", 0);
+INSERT IGNORE INTO entities (id, description, role, acl) VALUES (9, "The directory role.", "ROLE", 0);
+INSERT IGNORE INTO name_data (domain_id, entity_id, property_id, value, status, pidx) VALUES (0, 9, 20, "DIRECTORY", "FIX", 0);
+INSERT IGNORE INTO entity_ids (id, internal_id) VALUES (9,9);
 
 -- In the default file storage back-end the key is just the path.
 UPDATE files SET file_key=path;
@@ -81,8 +77,8 @@ ALTER TABLE files MODIFY COLUMN file_key VARBINARY(16000) NOT NULL;
 ALTER TABLE archive_files MODIFY COLUMN file_key VARBINARY(16000) NOT NULL;
 
 -- Size may be NULL for directories
--- ALTER TABLE files MODIFY COLUMN size BIGINT UNSIGNED NULL DEFAULT NULL;
--- ALTER TABLE archive_files MODIFY COLUMN size BIGINT UNSIGNED NULL DEFAULT NULL;
+ALTER TABLE files MODIFY COLUMN size BIGINT UNSIGNED NULL DEFAULT NULL;
+ALTER TABLE archive_files MODIFY COLUMN size BIGINT UNSIGNED NULL DEFAULT NULL;
 
 -- More characters for long names
 ALTER TABLE files MODIFY COLUMN path VARCHAR(16000) NOT NULL;
@@ -90,11 +86,12 @@ ALTER TABLE archive_files MODIFY COLUMN path VARCHAR(16000) NOT NULL;
 
 -- Add a special nameless directory which is used as the root directory of the
 -- internal file system.
--- INSERT IGNORE INTO entities (id, description, role, acl) VALUES (51, "The root directory of the internal files system", "DIRECTORY", 0);
---INSERT IGNORE INTO files
---    (file_id, path, checked_timestamp, mimetype, file_storage_id,
---        file_key, parent_directory)
---    VALUES (51, "", 0, "inode/directory", "DEFAULT", "", NULL);
+INSERT IGNORE INTO entities (id, description, role, acl) VALUES (51, "The root directory of the internal files system", "DIRECTORY", 0);
+INSERT IGNORE INTO entity_ids (id, internal_id) VALUES (51,51);
+INSERT IGNORE INTO files
+    (file_id, path, checked_timestamp, mimetype, file_storage_id,
+        file_key, parent_directory)
+    VALUES (51, "", 0, "inode/directory", "DEFAULT", "", NULL);
 
 DROP PROCEDURE IF EXISTS _create_dirs;
 DELIMITER //
@@ -146,9 +143,7 @@ BEGIN
                 -- this is the case when all parent directories of the current file have been processed.
                 SET dir_exists = FALSE;
                 SET done = FALSE;
-                LEAVE loop2;
-            END IF;
-
+                LEAVE loop2; END IF;
             -- remove the trailing slash for convenience.
             SET dir_path_no_trailing_slash = LEFT(dir_path, CHAR_LENGTH(dir_path)-1);
 
@@ -188,7 +183,7 @@ END //
 
 DELIMITER ;
 
--- CALL _create_dirs();
+CALL _create_dirs();
 
 -- not to be used after this patch has been applied.
 -- DROP PROCEDURE _create_dirs;