Dump loader#

The dump loader is in charge to restore data and storages from a Diskette archive.

class diskette.core.loader.Loader(logger=None)[source]#

Dump loader opens a Diskette archive to deploy its data and storage contents.

Keyword Arguments:

logger (object) – Instance of a logger object to use. Logger object must implement common logging message methods (like error, info, etc..). See diskette.utils.loggers for available loggers. If not given, a dummy logger will be used that ignores any messages and won’t output anything.

download_archive(url, destination=None)[source]#

Download archive from given URL into destination directory.

Parameters:

url (string) – The archive URL to download.

Keyword Arguments:

destination (Path) – A path where to write downloaded archive file. If not given, the archive file will be written as diskette_downloaded_archive.tar.gz into the current working directory.

Returns:

Path to downloaded archive file.

Return type:

Path

open(source, download_destination=None, keep=False, checksum=None)[source]#

Extract archive files in a temporary directory.

Warning

Using this method, you are responsible to remove the temporary directory once you are done with it. Your code must be safe about it and remove it even when your code fails or you will produce a lot of remaining temporary directories.

Parameters:

source (Path) – A Path object to the archive to open.

Keyword Arguments:

  • download_destination (Path) – A path where to write downloaded archive file. If not given, the archive file will be written as diskette_downloaded_archive.tar.gz into the current working directory. This argument is useless with local archive file.

  • keep (boolean) – Archive won’t be removed from filesystem if True, else the archive file is removed once it have been extracted.

  • checksum (object) –

    Manage if archive is checksumed or not depending value:

    • If None: Checksum is done and just output to logs;

    • If True: Checksum is done and just output to logs;

    • If False: No checksum are done or compared;

    • Any other value is assumed to be a string for a checksum to compare. Then a checksum is done on archive and compared to the given one, if comparaison fails it results to a critical error.

Returns:

The temporary directory where archive files have been extracted.

Return type:

Path

get_manifest(path)[source]#

Search for manifest file in given path, validate it and return it.

This raises an exception if manifest is invalid, the used exception class will depends from used logger.

Parameters:

path (Path) – Path object to the directory where to search for the manifest file.

Returns:

The manifest data.

Return type:

dict

validate_datas()[source]#

Call validators from all enabled data dumps.

Note

There is currently no validator needed.

Returns:

Always return True since there is nothing to validate actually.

Return type:

boolean

validate_storages()[source]#

Call validators from all enabled storages.

Note

There is currently no validator needed.

Returns:

Always return True since there is nothing to validate actually.

Return type:

boolean

validate()[source]#

Call all validators

deploy_storages(archive_dir, manifest, destination)[source]#

Deploy storages directories in given destination

Parameters:

  • archive_dir (Path) – Path to directory where archive has been exracted.

  • manifest (dict) – The manifest data.

  • destination (Path) – Path to directory where to deploy storages.

Returns:

List of tuples for deployed storage with respectively source and

destination paths.

Return type:

list

check_data_dump(dump, excludes)[source]#

Check if data dump is to be loaded or not.

When dump is not to be loaded, a INFO log message will be output.

This check dump file against filename exclusions and minimal file size.

Returns:

True if dump is to be loaded, else False.

Return type:

boolean

deploy_datas(archive_dir, manifest, excludes=None)[source]#

Deploy storages directories in given destination

Parameters:

  • archive_dir (Path) – Path to directory where archive has been exracted.

  • manifest (dict) – The manifest data.

Keyword Arguments:

excludes (list) – List of dump filenames to exclude from loading.

Returns:

List of tuples for deployed dumps with respectively source and

loaddata output.

Return type:

list

deploy(archive, storages_destination, data_exclusions=None, with_data=True, with_storages=True, download_destination=None, keep=False, checksum=None)[source]#

Load archive and deploy its content.

Parameters:

  • archive (Path or string) – The tarball archive to open and extract dumps. It may be either a Path to a local archive file or a string for an URL to download the archive.

  • storages_destination (Path) – Destination where to deploy all storage directories.

Keyword Arguments:

  • data_exclusions (list) – List of dump filenames to exclude from loading.

  • with_data (boolean) – Enable application datas loading.

  • with_storages (boolean) – Enabled media storages loading.

  • download_destination (Path) – A path where to write downloaded archive file. If not given, the archive file will be written as diskette_downloaded_archive.tar.gz into the current working directory. This argument is useless with local archive file.

  • keep (boolean) – Archive won’t be removed from filesystem if True, else the archive file is removed once it have been extracted.

  • checksum (object) –

    Manage if archive is checksumed or not depending value:

    • If None: Checksum is done and just output to logs;

    • If True: Checksum is done and just output to logs;

    • If False: No checksum are done or compared;

    • Any other value is assumed to be a string for a checksum to compare. Then a checksum is done on archive and compared to the given one, if comparaison fails it results to a critical error.

Returns:

Statistics of deployed storages and datas.

Return type:

dict