CHANGELOG.md

# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased] ##

### Added ###

### Changed ###

### Deprecated ###

### Removed ###

### Fixed ###

### Security ###

## [0.6.0] - 2022-10-11 ##
(Florian Spreckelsen)

### Added ###

- Unauthorized inserts can now be cached. Note that the Crawler cannot postpone
  inserts but the Cache has the functionality now.
- caosdbignore; You can add one or more `.caosdbignore` files to the directory
  structure that you want to make available in CaosDB and the run loadFiles.
  The syntax is that of `.gitignore` files. For more information see `loadFiles`
  section of the Crawler in the documentation.

## [0.5.0] - 2022-09-05 ##
(Florian Spreckelsen)

### Added ###

- You can now use `python -m caosadvancedtools.models.parser model_file` to
  parse and potentially synchronize data models.

### Deprecated ###

- [#36](https://gitlab.com/caosdb/caosdb-advanced-user-tools/-/issues/36)
  `parent` keyword in yaml datamodel definition (replaced by
  `inherit_from_{obligatory|recommended|suggested}` keywords).

## [0.4.1] - 2022-05-03 ##
(Henrik tom Wörden)

### Changed ###

- `JsonSchemaParser` now identifies `name` properties in the schema with the
  CaosDB name property.

### Fixed ###

- [#40](https://gitlab.com/caosdb/caosdb-advanced-user-tools/-/issues/40)
  `assure_object_is_in_list` now handles adding objects to an initially empty list correctly.

## [0.4.0] - 2022-04-05  ##

### Added ###

- CFood that creates a Record for each line in a csv file
- `generic_analysis.py` allows to easily call scripts to perform analyses in
  server side scripting [EXPERIMENTAL]
- **EXPERIMENTAL:** Models parser can import from Json Schema files now:
  `models.parser.parse_model_from_json_schema(...)`. See the documentation of
  `models.parser.JsonSchemaParser` for the limitations of the current
  implementation.
- New keyword "role" in yaml data model that allows creation of Records and Files.
- It is now possible to set values of properties and default values of properties
  directly in the yaml model.

### Changed ###

- `TableConverter` now converts int to float and vice versa to match the desired dtype.

### Deprecated ###

### Removed ###

### Fixed ###

- CaosDB internal properties `name`, `unit` and `description` can now be used via the `extern`
  keyword in YAML datamodel specifications.

### Security ###

## [0.3.1] - 2021-12-06  ##

### Added ###
- `check_reference_field` function to check whether entities with provided ids
  exits (for example when importing data from a table)
- added the `datatypes` argument to `TableImporter` for columns that do not
  need a special conversion function

## [0.3.0] - 2021-11-02 ##

### Added ###

- Error handling for missing files when using the crawler
- included the scifolder module
- included the caosmodels module
- `send_mail` function in `caosadvancedtools.serverside.helper` module
- New class to collect possible problems with the data model
- New class for checking and importing tables
- Function to get a file path to a shared resource directory
- Function to setup logging appropriate for server side scripts with webui
  output
- New class for collecting information for exporting tables, e.g., to
  metadata repositories
- new name parsing
- new test for software folder structure
- new assure_name_is function
- two utility functions when working with files: NameCollector and
  get_file_via_download
- Automated documentation builds: `make doc`
- Crawler documentation
- Proof-of-concept integration with Bloxberg.
- Introduce a cfood that can create a Record structure based on the contents of a hdf5 file
  h5py is now an optional dependency
- table importer implementations for csv and tsv
- string-in-list check for table imports
- AbtractCFood has new property, `update_flags`.

### Changed ###

- identifiables of single CFoods are now treated one after the other. This
  allows them to have dependencies among each other if they are ordered
  correctly
- identifiables must have at least one property or a name
- `caosadvancedtools.serverside.helper.init_data_model` also checks the role
  and data type of entities.
- The `caosadvancedtools.table_importer.date_converter` now actually returns
  `datetime.date` instance. A new
  `caosadvancedtools.table_importer.datetime_converter` replaces the old
  `date_converter` and returns a `datetime.datetime` instance.
- The suppression module is now a logging filter.
- The WebUIHandler is now a python logging formatter.
- instead of `get_entity`, type-specific functions are used in
  `cfood.py` when the type of the entity in question is known.
- Logger is used instead of `print` for errors in `crawler.py`.
- complies with new exception handling, i.e., TransactionErros with
  children being raised in all cases of EntityErrors (see
  [#32](https://gitlab.com/caosdb/caosdb-pylib/-/issues/32) in
  caosdb-pylib)
- `caosadvancedtools.cfood.assure_object_is_in_list` conducts in-place
  updates if no `to_be_updated` object is supplied.

### Fixed ###
- An exception in collect_information does no longer lead to a break down.
- Removed dependency on discontiued xlrd version
- Fixed an issue where `caosadvancedtools.cache.UpdateCache` would
  cause an `sqlite3.IntegrityError` if more than one change was cached
  for the same entity.
- #40 Insertion of identifiables with missing obligatory properties
- Before, a Property with the datatype "LIST(TEXT)" would lead to the creation
  of a RecordType. This is fixed now.
- #52 `XLSimporter.read_xls` throwed a wrong error when reading from a file with a wrong ending.
  Now, a `DataInconsistencyError` is raised instead of a ValueError.
- List properties are no longer updated unnecessarily by the crawler.

## [0.2.0] - 2020-05-28 ##

### Added
- everything