filesystem

dirsize(path)

Recursively compute the size of the contents of a directory.

Parameters:path – Returns:size in bytes

format_file_size(bytes)

copy_dir_with_progress(source_dir, target_dir, move=False)

Utility for moving/copying a large directory and displaying a progress bar showing how much is copied.

Note that the directory is first copied, then the old directory is removed, if move=True.

Parameters:

• source_dir –
• target_dir –

Returns:

move_dir_with_progress(source_dir, target_dir)

new_filename(directory, initial_filename='tmp_file')

Generate a filename that doesn’t already exist.

retry_open(filename, errnos=[13], retry_schedule=[2, 10, 30, 120, 300], **kwargs)

Try opening a file, using the builtin open() function (Py3, or io.open on Py2). If an IOError is raised and its errno is in the given list, wait a moment then retry. Keeps doing this, waiting a bit longer each time, hoping that the problem will go away.

Once too many attempts have been made, outputs a message and waits for user input. This means the user can fix the problem (e.g. renew credentials) and pick up where execution left off. If they choose not to, the original error will be raised

Default list of errnos is just [13] – permission denied.

Use retry_schedule to customize the lengths of time waited between retries. Default: 2s, 10s, 30s, 2m, 5m, then give up.

Additional kwargs are pass on to open().

extract_from_archive(archive_filename, members, target_dir, preserve_dirs=True)

Extract a file or files from an archive, which may be a tarball or a zip file (determined by the file extension).

extract_archive(archive_filename, target_dir, preserve_dirs=True)

Extract all files from an archive, which may be a tarball or a zip file (determined by the file extension).