Draft: ENH: file system: import

Summary

Insert a meaningful description for this merge request here: What is the new/changed behavior? Which bug has been fixed? Are there related issues?

Focus

Point the reviewer to the core of the code change. Where should they start reading? What should they focus on (e.g. security, performance, maintainability, user-friendliness, compliance with the specs, finding more corner cases, concrete questions)?

Test Environment

How to set up a test environment for manual testing?

Check List for the Author

Please, prepare your MR for a review. Be sure to write a summary and a focus and create gitlab comments for the reviewer. They should guide the reviewer through the changes, explain your changes and also point out open questions. For further good practices have a look at our review guidelines

• All automated tests pass
• Reference related issues
• Up-to-date CHANGELOG.md (or not necessary)
• Annotations in code (Gitlab comments)
  • Intent of new code
  • Problems with old code
  • Why this implementation?

Check List for the Reviewer

• I understand the intent of this MR
• All automated tests pass
• Up-to-date CHANGELOG.md (or not necessary)
• The test environment setup works and the intended behavior is reproducible in the test environment
• In-code documentation and comments are up-to-date.
• Check: Are there specifications? Are they satisfied?

For further good practices have a look at our review guidelines.

src/caosdb/common/models.py

@@ -1858,7 +1858,8 @@ class Link(Entity):
 class Directory(Record):
     """This class represents CaosDB's directory entities."""
 
-    def __init__(self, name=None, id=None, description=None, path=None):
+    def __init__(self, name=None, id=None, description=None, path=None,
+                 import_file=False, recursive_import=False):
         Record.__init__(self, id=id, name=name, description=description)
         self.role = "Directory"
         self.datatype = None
@@ -1866,6 +1867,9 @@ class Directory(Record):
         # location in the fileserver
         self.path = path
 
+        self.recursive_import = recursive_import
+        self.import_file = import_file or recursive_import
+
     def to_xml(self, xml=None, add_properties=ALL, local_serialization=False):
         """Convert this Directory to an xml element.
 
@@ -1913,11 +1917,13 @@ class File(Record):
         Deprecated.
     thumbnail : str
         Deprecated.
+    import_file : bool
+        Import the file (don't upload it, its already there). Default: `False`
     """
 
     def __init__(self, name=None, id=None, description=None,
                  path=None, file=None, pickup=None,
-                 thumbnail=None):
+                 thumbnail=None, import_file=False):
         Record.__init__(self, id=id, name=name, description=description)
         self.role = "File"
         self.datatype = None
@@ -1927,6 +1933,9 @@ class File(Record):
 
         # local file path or pointer to local file
         self.file = file
+
+        self.import_file = import_file
+
         if thumbnail is not None:
             warn(DeprecationWarning(
                 "The thumbnail feature has been removed from the CaosDB "
@@ -3455,6 +3464,8 @@ class Container(list):
 
             if hasattr(entity, '_upload') and entity._upload is not None:
                 entity_xml.set("upload", entity._upload)
+            elif hasattr(entity, 'import_file') and entity.import_file is True:
+                entity_xml.set("import", "true")
 
             insert_xml.append(entity_xml)
 
@@ -3495,7 +3506,7 @@ class Container(list):
     def _process_file_if_present_and_add_to_http_parts(http_parts, entity):
         if isinstance(entity, File) and hasattr(
                 entity, 'file') and entity.file is not None:
-            new_checksum = File._get_checksum(entity.file)
+            new_checksum = File.sha512(entity.file)
 
             # do not transfer unchanged files.
 
@@ -3601,6 +3612,10 @@ class Container(list):
 
             if hasattr(entity, '_upload') and entity._upload is not None:
                 entity_xml.set("upload", entity._upload)
+            elif hasattr(entity, 'recursive_import') and entity.recursive_import is True:
+                entity_xml.set("recursive_import", "true")
+            elif hasattr(entity, 'import_file') and entity.import_file is True:
+                entity_xml.set("import", "true")
             insert_xml.append(entity_xml)
 
         if len(self) > 0 and len(insert_xml) < 1: