Documentation

This is a helper for saving and loading data from files, with a given encoding and decoding function. It can automatically detect the appropriate serializer based on the file ending, and saves in an atomic way by first saving to a temporary file and then moving it.

To define an encoding, an instance of SerializerDispatch is created, which has methods for saving and loading.

A default encoding which can handle common numpy and built-in types is also given.

Serializer Dispatch – saving and loading data

class fsc.iohelper.SerializerDispatch(encoding)

Defines functions for saving and loading for a given encoding.

Parameters:encoding – Object with encode and decode members, to be passed as the default and object_hook parameters, respectively (for JSON and msgpack, see e.g. json.dump() and json.load()).

Basic usage:

import fsc.iohelper

IO_HANDLER = fsc.iohelper.SerializerDispatch(fsc.iohelper.encoding.default)
IO_HANDLER.save([1, 2, 3], 'filename.json')
x = IO_HANDLER.load('filename.json')

load(file_path, serializer='auto')

Loads the object that was saved to file_path.

Parameters:

• file_path (str) – Path to the file.
• serializer (module) – The serializer which should be used to load the result. By default, is deduced from the file extension. If no serializer is given and it cannot be deduced from the file ending, a ValueError is raised, to avoid loading corrupted data.

save(obj, file_path, serializer='auto')

Saves an object to the file given in file_path. The saving is made atomic (on systems where os.replace() is atomic) by first creating a temporary file and then moving to the file_path.

Parameters:

• obj – Object to be saved.
• file_path (str) – Path to the file.
• serializer (module) – The serializer to be used. Valid options are msgpack json and pickle. By default, the serializer is determined from the file extension. If this does not work, json is used to avoid data loss.

Default encoding – encoding.default

This is an example implementation of an encoding, to be used as input for SerializerDispatch.

fsc.iohelper.encoding.default.encode(obj)

Encodes numpy and built-in types into JSON / msgpack - compatible types.

fsc.iohelper.encoding.default.decode(obj)

Decodes JSON / msgpack objects into the corresponding built-in types.