Io

coola.io

Provide loaders and savers to read and write data to files.

A loader reads one value from a file (load(path)) and a saver writes one value to a file (save(data, path, exist_ok=False)), with the same interface for every format: JSON, pickle, text and PyTorch. load and save pick the loader or the saver from the suffix of the path. Each format has a class (JsonLoader, JsonSaver, ...) and a function shortcut (load_json, save_json, ...).

Properties of a file saver:

• Atomic write: the data is written to a temporary file that replaces the target in one operation, so a reader never sees a partial file and a failed save leaves the target untouched.
• No silent overwrite: save raises FileExistsError if the target exists, unless exist_ok=True.
• Opt-in durability: fsync=True flushes the file and its directory.

resolve_loader and resolve_saver build a loader or a saver from a configuration. A new format is added by subclassing BaseLoader and BaseFileSaver. TorchLoader and TorchSaver import PyTorch the first time they are used.

coola.io.BaseFileSaver

Bases: BaseSaver[T]

Define the base class to implement a file saver.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.BaseFileSaver.save

save(
    to_save: T,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	str | PathLike[str]	The path where to save the data. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file. Note: the exist_ok=False guarantee holds against a concurrent creation of path between the initial existence check and the commit step (the commit uses os.link which atomically fails with FileExistsError in that case). Note: concurrent save calls targeting the same path, from this process or from other processes, never produce a mixed file: each writes its own hidden temporary file (.<name>-<uuid><ext>) and commits it atomically, so the file is the complete content of one of them. With exist_ok=True which one is unspecified (the last commit wins); with exist_ok=False exactly one wins and the others raise FileExistsError. save takes no lock: a caller that needs mutual exclusion around a read-modify-write of one file uses its own lock.	False
fsync	bool	If True, the data is flushed to the disk before the file is committed, and the directory is flushed after, so the file is not empty or missing after a power failure or a system crash. It is slower. If False (the default), the data is only written to the operating system cache: the file is still replaced atomically (a reader never sees a partially written file), but its content can be lost if the machine crashes right after the call. This is fine for caches and most logs.	False

Note

The caller needs write access to the directory of path, not only to the file: the data is first written to a hidden temporary file next to path. A process killed between the write and the commit leaves this file; it is safe to delete. The saved file is committed with a hard link, or with a rename if the filesystem does not support hard links (e.g. FAT/exFAT, and some network and container volumes); in that case two processes that create the same file at the same instant can both succeed (the last one wins, no file is torn).

Raises:

Type	Description
FileExistsError	if the file already exists and exist_ok is False.
IsADirectoryError	If path is a directory.
NotADirectoryError	If the parent of path is a file.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.BaseLoader

Bases: InlineDisplayMixin, ABC, Generic[T]

Define the base class to implement a data loader.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.BaseLoader.equal

equal(other: Any, equal_nan: bool = False) -> bool

Indicate if two objects are equal or not.

Two objects are equal if they have the same type and the same configuration, i.e. the fields returned by _display_fields (the fields shown by repr) are equal. A format does not override this method.

Parameters:

Name	Type	Description	Default
other	Any	The object to compare with.	required
equal_nan	bool	If True, then two NaNs will be considered equal.	False

Returns:

Type	Description
bool	True if the two objects are equal, otherwise False.

Example

>>> from coola.io import JsonLoader, TextLoader
>>> JsonLoader().equal(JsonLoader())
True
>>> JsonLoader().equal(TextLoader())
False

coola.io.BaseLoader.load

load(path: str | PathLike[str]) -> T

Load the data from the given path.

The path is converted to a Path (~ and file:// URIs are expanded) before _load is called, so a format always receives a Path.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path with the data to load. A str or any path-like object is converted to a Path.	required

Returns:

Type	Description
T	The data

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.BaseSaver

Bases: InlineDisplayMixin, ABC, Generic[T]

Define the base class to implement a data saver.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.BaseSaver.equal

equal(other: Any, equal_nan: bool = False) -> bool

Indicate if two objects are equal or not.

Two objects are equal if they have the same type and the same configuration, i.e. the fields returned by _display_fields (the fields shown by repr) are equal. A format does not override this method.

Parameters:

Name	Type	Description	Default
other	Any	The object to compare with.	required
equal_nan	bool	If True, then two NaNs will be considered equal.	False

Returns:

Type	Description
bool	True if the two objects are equal, otherwise False.

Example

>>> from coola.io import JsonSaver, TextSaver
>>> JsonSaver().equal(JsonSaver())
True
>>> JsonSaver().equal(TextSaver())
False

coola.io.BaseSaver.save abstractmethod

save(
    to_save: T,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	str | PathLike[str]	The path where to save the data. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), an exception is raised if the target path already exists.	False

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.JsonLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a JSON file.

Parameters:

Name	Type	Description	Default
encoding	str | None	The encoding of the file, which should be the one used to save it. If None (default), the file is read in binary mode and json detects the encoding, which only works for UTF-8, UTF-16 and UTF-32.	None

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.JsonSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a JSON file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
allow_nan	bool	If False (the default), NaN and infinite floats raise a ValueError, because the file would not be valid JSON. If True, they are written as the non-standard NaN/Infinity tokens.	False
jsonable	bool	If True, the data is converted with coola.jsonable.to_jsonable before it is written, so the objects that json does not support (tuples are lists, NumPy arrays, ...) are converted. Defaults to False.	False
**kwargs	Any	Additional arguments passed to json.dump, e.g. indent or sort_keys.	{}

Note

JSON has only string keys: int keys are loaded back as str.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.PickleLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a pickle file.

Warning

load calls pickle.load on the given file. Unpickling can execute arbitrary code as a side effect of deserialization. Only load pickle files that come from a trusted source; never load a pickle file that comes from an untrusted or unauthenticated source.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.PickleSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a pickle file.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Additional arguments passed to pickle.dump.	{}

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import PickleSaver, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     PickleSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.TextLoader

Bases: BaseLoader[str]

Implement a data loader to load data from a text file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when reading the file. Defaults to "utf-8".	DEFAULT_ENCODING

Note

The line endings are returned as they are in the file (no universal newline translation), so \\r\\n and \\r are preserved.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

coola.io.TextSaver

Bases: BaseFileSaver[str]

Implement a file saver to save data to a text file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING

Note

The line endings are written as they are in the string (no translation of \\n, even on Windows).

Raises:

Type	Description
TypeError	if the data to save is not a str. Call str on the data to save its text representation.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import TextSaver, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     TextSaver().save("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

coola.io.TorchLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a PyTorch file.

Parameters:

Name	Type	Description	Default
weights_only	bool	If True (the default), only tensors and simple containers are loaded. Pass False only for trusted files.	True
**kwargs	Any	Additional arguments passed to torch.load.	{}

Security

Loading with weights_only=False can execute arbitrary code (it uses pickle). Only do so for files from a trusted source.

Raises:

Type	Description
MissingPackageError	If torch is not installed.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.TorchSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a PyTorch file.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Additional arguments passed to torch.save.	{}

Raises:

Type	Description
MissingPackageError	If torch is not installed.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import TorchSaver, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     TorchSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.is_loader_config

is_loader_config(
    config: dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    import_target: bool = False,
) -> bool

Indicate if the input configuration is a configuration for a BaseLoader.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict[Any, Any]	The configuration to check.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
import_target	bool	If True, the module of "_target_" is imported if it is not imported yet, even without allowed_prefixes. By default, a "_target_" whose module is not imported yet and that is not restricted by allowed_prefixes is not a configuration for a BaseLoader (see coola.factory.is_object_config).	False

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseLoader object.

Security

The module of the "_target_" is imported to check its class only with allowed_prefixes or import_target=True: see coola.factory.is_object_config.

Example

>>> from coola.io import is_loader_config
>>> is_loader_config({"_target_": "coola.io.JsonLoader"})
True

coola.io.is_saver_config

is_saver_config(
    config: dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    import_target: bool = False,
) -> bool

Indicate if the input configuration is a configuration for a BaseSaver.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict[Any, Any]	The configuration to check.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
import_target	bool	If True, the module of "_target_" is imported if it is not imported yet, even without allowed_prefixes. By default, a "_target_" whose module is not imported yet and that is not restricted by allowed_prefixes is not a configuration for a BaseSaver (see coola.factory.is_object_config).	False

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseSaver object.

Security

The module of the "_target_" is imported to check its class only with allowed_prefixes or import_target=True: see coola.factory.is_object_config.

Example

>>> from coola.io import is_saver_config
>>> is_saver_config({"_target_": "coola.io.JsonSaver"})
True

coola.io.load

load(path: str | PathLike[str], **kwargs: Any) -> Any

Load the data from a file, with the loader of its suffix.

The suffixes are .json (JsonLoader), .pkl and .pickle (PickleLoader), .pt and .pth (TorchLoader) and .txt (TextLoader). The suffix is case-insensitive.

Warning

A .pkl file is loaded with pickle.load, which can execute arbitrary code. Only load a pickle file that comes from a trusted source.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the file.	required
**kwargs	Any	The arguments of the loader, for example encoding for .json and .txt, or weights_only for .pt.	{}

Returns:

Type	Description
Any	The data from the file.

Raises:

Type	Description
ValueError	If the suffix of path is not supported.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import load, save
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save({"key": [1, 2, 3]}, path)
...     load(path)
...
{'key': [1, 2, 3]}

coola.io.load_json

load_json(
    path: str | PathLike[str], encoding: str | None = None
) -> Any

Load the data from a given JSON file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the JSON file. A str or any path-like object is converted to a Path.	required
encoding	str | None	The encoding of the file. If None, the encoding is detected (UTF-8, UTF-16 and UTF-32 only).	None

Returns:

Type	Description
Any	The data from the JSON file.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.load_pickle

load_pickle(path: str | PathLike[str]) -> Any

Load the data from a given pickle file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the pickle file. A str or any path-like object is converted to a Path.	required

Returns:

Type	Description
Any	The data from the pickle file.

Warning

This function unpickles the file's content, which can execute arbitrary code as a side effect of deserialization. Only load pickle files that come from a trusted source; never load a pickle file that comes from an untrusted or unauthenticated source.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.load_text

load_text(
    path: str | PathLike[str],
    encoding: str = DEFAULT_ENCODING,
) -> str

Load the data from a given text file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the text file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when reading the file. Defaults to "utf-8".	DEFAULT_ENCODING

Returns:

Type	Description
str	The text content of the file as a string.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

coola.io.load_torch

load_torch(path: str | PathLike[str], **kwargs: Any) -> Any

Load the data from a given PyTorch file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the PyTorch file. A str or any path-like object is converted to a Path.	required
**kwargs	Any	Additional arguments passed to torch.load.	{}

Returns:

Type	Description
Any	The data from the PyTorch file.

Security

weights_only defaults to True, which restricts the unpickler to tensors and simple containers. Pass weights_only=False only for files from a trusted source: it uses pickle, which can execute arbitrary code.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.resolve_loader

resolve_loader(
    loader: BaseLoader[T] | dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    strict: bool = False,
) -> BaseLoader[T]

Set up a data loader.

The data loader is instantiated from its configuration by using the BaseLoader factory function.

Parameters:

Name	Type	Description	Default
loader	BaseLoader[T] | dict[Any, Any]	The data loader or its configuration.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
strict	bool	If True, a "_target_" that is not a class and has no usable return type hint is refused before it is called (see resolve_object).	False

Returns:

Type	Description
BaseLoader[T]	The instantiated data loader.

Security

When loader is a configuration dict, its "_target_" value is imported and instantiated with no allowlist by default (see coola.factory.factory). Only call this function with a configuration that comes from a trusted source, or restrict it with allowed_prefixes; never resolve a loader configuration derived from untrusted input without it, as this is a remote-code-execution vector.

Example

>>> from coola.io import resolve_loader
>>> loader = resolve_loader({"_target_": "coola.io.JsonLoader"})
>>> loader
JsonLoader(encoding=None)

coola.io.resolve_saver

resolve_saver(
    saver: BaseSaver[T] | dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    strict: bool = False,
) -> BaseSaver[T]

Set up a data saver.

The data saver is instantiated from its configuration by using the BaseSaver factory function.

Parameters:

Name	Type	Description	Default
saver	BaseSaver[T] | dict[Any, Any]	The data saver or its configuration.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
strict	bool	If True, a "_target_" that is not a class and has no usable return type hint is refused before it is called (see resolve_object).	False

Returns:

Type	Description
BaseSaver[T]	The instantiated data saver.

Security

When saver is a configuration dict, its "_target_" value is imported and instantiated with no allowlist by default (see coola.factory.factory). Only call this function with a configuration that comes from a trusted source, or restrict it with allowed_prefixes; never resolve a saver configuration derived from untrusted input without it, as this is a remote-code-execution vector.

Example

>>> from coola.io import resolve_saver
>>> saver = resolve_saver({"_target_": "coola.io.JsonSaver"})
>>> saver
JsonSaver(encoding='utf-8', allow_nan=False, jsonable=False)

coola.io.save

save(
    to_save: Any,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
    **kwargs: Any,
) -> None

Save the data in a file, with the saver of its suffix.

The suffixes are .json (JsonSaver), .pkl and .pickle (PickleSaver), .pt and .pth (TorchSaver) and .txt (TextSaver). The suffix is case-insensitive.

Parameters:

Name	Type	Description	Default
to_save	Any	The data to write.	required
path	str | PathLike[str]	The path where to write the file.	required
exist_ok	bool	If False (the default), FileExistsError is raised if the file already exists.	False
fsync	bool	If True, flush the data to the disk before committing the file. Defaults to False.	False
**kwargs	Any	The arguments of the saver, for example indent for .json or encoding for .txt.	{}

Raises:

Type	Description
ValueError	If the suffix of path is not supported.
FileExistsError	If the file already exists.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import load, save
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save("abc", path)
...     load(path)
...
'abc'

coola.io.save_json

save_json(
    to_save: Any,
    path: str | PathLike[str],
    *,
    encoding: str = DEFAULT_ENCODING,
    exist_ok: bool = False,
    fsync: bool = False,
    allow_nan: bool = False,
    jsonable: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a JSON file.

The file is what json.dump writes: the keys that are not strings (for example int) are written as strings, so they are strings after load_json. NaN and infinite floats are not valid JSON, so they raise a ValueError unless allow_nan=True (they are then written as the non-standard tokens NaN and Infinity). The data is not converted unless jsonable=True: the objects that json does not support (NumPy arrays, ...) raise a TypeError.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a JSON file.	required
path	str | PathLike[str]	The path where to write the JSON file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
allow_nan	bool	If False (the default), NaN and infinite floats raise a ValueError. If True, they are written as the non-standard NaN/Infinity tokens.	False
jsonable	bool	If True, convert the data with coola.jsonable.to_jsonable before it is written. Defaults to False.	False
**kwargs	Any	Additional arguments passed to json.dump, e.g. indent or sort_keys.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.
ValueError	if the data has a NaN or an infinite float and allow_nan is False.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.save_pickle

save_pickle(
    to_save: Any,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a pickle file.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a pickle file.	required
path	str | PathLike[str]	The path where to write the pickle file. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
**kwargs	Any	Additional arguments passed to pickle.dump.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.save_text

save_text(
    to_save: str,
    path: str | PathLike[str],
    *,
    encoding: str = DEFAULT_ENCODING,
    exist_ok: bool = False,
    fsync: bool = False,
) -> None

Save the given data to a text file.

Parameters:

Name	Type	Description	Default
to_save	str	The string to write to the text file.	required
path	str | PathLike[str]	The path where to write the text file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False

Raises:

Type	Description
FileExistsError	if the file already exists.
TypeError	if the data to save is not a str. Call str on the data to save its text representation.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

coola.io.save_torch

save_torch(
    to_save: Any,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a PyTorch file.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a PyTorch file.	required
path	str | PathLike[str]	The path where to write the PyTorch file. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
**kwargs	Any	Additional arguments passed to torch.save.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base

Contain the base class to implement a data loader or saver object.

coola.io.base.BaseFileSaver

Bases: BaseSaver[T]

Define the base class to implement a file saver.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.BaseFileSaver.save

save(
    to_save: T,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	str | PathLike[str]	The path where to save the data. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file. Note: the exist_ok=False guarantee holds against a concurrent creation of path between the initial existence check and the commit step (the commit uses os.link which atomically fails with FileExistsError in that case). Note: concurrent save calls targeting the same path, from this process or from other processes, never produce a mixed file: each writes its own hidden temporary file (.<name>-<uuid><ext>) and commits it atomically, so the file is the complete content of one of them. With exist_ok=True which one is unspecified (the last commit wins); with exist_ok=False exactly one wins and the others raise FileExistsError. save takes no lock: a caller that needs mutual exclusion around a read-modify-write of one file uses its own lock.	False
fsync	bool	If True, the data is flushed to the disk before the file is committed, and the directory is flushed after, so the file is not empty or missing after a power failure or a system crash. It is slower. If False (the default), the data is only written to the operating system cache: the file is still replaced atomically (a reader never sees a partially written file), but its content can be lost if the machine crashes right after the call. This is fine for caches and most logs.	False

Note

The caller needs write access to the directory of path, not only to the file: the data is first written to a hidden temporary file next to path. A process killed between the write and the commit leaves this file; it is safe to delete. The saved file is committed with a hard link, or with a rename if the filesystem does not support hard links (e.g. FAT/exFAT, and some network and container volumes); in that case two processes that create the same file at the same instant can both succeed (the last one wins, no file is torn).

Raises:

Type	Description
FileExistsError	if the file already exists and exist_ok is False.
IsADirectoryError	If path is a directory.
NotADirectoryError	If the parent of path is a file.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.BaseLoader

Bases: InlineDisplayMixin, ABC, Generic[T]

Define the base class to implement a data loader.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.BaseLoader.equal

equal(other: Any, equal_nan: bool = False) -> bool

Indicate if two objects are equal or not.

Two objects are equal if they have the same type and the same configuration, i.e. the fields returned by _display_fields (the fields shown by repr) are equal. A format does not override this method.

Parameters:

Name	Type	Description	Default
other	Any	The object to compare with.	required
equal_nan	bool	If True, then two NaNs will be considered equal.	False

Returns:

Type	Description
bool	True if the two objects are equal, otherwise False.

Example

>>> from coola.io import JsonLoader, TextLoader
>>> JsonLoader().equal(JsonLoader())
True
>>> JsonLoader().equal(TextLoader())
False

coola.io.base.BaseLoader.load

load(path: str | PathLike[str]) -> T

Load the data from the given path.

The path is converted to a Path (~ and file:// URIs are expanded) before _load is called, so a format always receives a Path.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path with the data to load. A str or any path-like object is converted to a Path.	required

Returns:

Type	Description
T	The data

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.BaseSaver

Bases: InlineDisplayMixin, ABC, Generic[T]

Define the base class to implement a data saver.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.BaseSaver.equal

equal(other: Any, equal_nan: bool = False) -> bool

Indicate if two objects are equal or not.

Two objects are equal if they have the same type and the same configuration, i.e. the fields returned by _display_fields (the fields shown by repr) are equal. A format does not override this method.

Parameters:

Name	Type	Description	Default
other	Any	The object to compare with.	required
equal_nan	bool	If True, then two NaNs will be considered equal.	False

Returns:

Type	Description
bool	True if the two objects are equal, otherwise False.

Example

>>> from coola.io import JsonSaver, TextSaver
>>> JsonSaver().equal(JsonSaver())
True
>>> JsonSaver().equal(TextSaver())
False

coola.io.base.BaseSaver.save abstractmethod

save(
    to_save: T,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	str | PathLike[str]	The path where to save the data. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), an exception is raised if the target path already exists.	False

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.base.is_loader_config

is_loader_config(
    config: dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    import_target: bool = False,
) -> bool

Indicate if the input configuration is a configuration for a BaseLoader.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict[Any, Any]	The configuration to check.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
import_target	bool	If True, the module of "_target_" is imported if it is not imported yet, even without allowed_prefixes. By default, a "_target_" whose module is not imported yet and that is not restricted by allowed_prefixes is not a configuration for a BaseLoader (see coola.factory.is_object_config).	False

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseLoader object.

Security

The module of the "_target_" is imported to check its class only with allowed_prefixes or import_target=True: see coola.factory.is_object_config.

Example

>>> from coola.io import is_loader_config
>>> is_loader_config({"_target_": "coola.io.JsonLoader"})
True

coola.io.base.is_saver_config

is_saver_config(
    config: dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    import_target: bool = False,
) -> bool

Indicate if the input configuration is a configuration for a BaseSaver.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict[Any, Any]	The configuration to check.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
import_target	bool	If True, the module of "_target_" is imported if it is not imported yet, even without allowed_prefixes. By default, a "_target_" whose module is not imported yet and that is not restricted by allowed_prefixes is not a configuration for a BaseSaver (see coola.factory.is_object_config).	False

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseSaver object.

Security

The module of the "_target_" is imported to check its class only with allowed_prefixes or import_target=True: see coola.factory.is_object_config.

Example

>>> from coola.io import is_saver_config
>>> is_saver_config({"_target_": "coola.io.JsonSaver"})
True

coola.io.base.resolve_loader

resolve_loader(
    loader: BaseLoader[T] | dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    strict: bool = False,
) -> BaseLoader[T]

Set up a data loader.

The data loader is instantiated from its configuration by using the BaseLoader factory function.

Parameters:

Name	Type	Description	Default
loader	BaseLoader[T] | dict[Any, Any]	The data loader or its configuration.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
strict	bool	If True, a "_target_" that is not a class and has no usable return type hint is refused before it is called (see resolve_object).	False

Returns:

Type	Description
BaseLoader[T]	The instantiated data loader.

Security

When loader is a configuration dict, its "_target_" value is imported and instantiated with no allowlist by default (see coola.factory.factory). Only call this function with a configuration that comes from a trusted source, or restrict it with allowed_prefixes; never resolve a loader configuration derived from untrusted input without it, as this is a remote-code-execution vector.

Example

>>> from coola.io import resolve_loader
>>> loader = resolve_loader({"_target_": "coola.io.JsonLoader"})
>>> loader
JsonLoader(encoding=None)

coola.io.base.resolve_saver

resolve_saver(
    saver: BaseSaver[T] | dict[Any, Any],
    *,
    allowed_prefixes: Sequence[str] | None = None,
    strict: bool = False,
) -> BaseSaver[T]

Set up a data saver.

The data saver is instantiated from its configuration by using the BaseSaver factory function.

Parameters:

Name	Type	Description	Default
saver	BaseSaver[T] | dict[Any, Any]	The data saver or its configuration.	required
allowed_prefixes	Sequence[str] | None	An optional allow-list of module prefixes that the "_target_" must match, e.g. ["coola.io"]. If None (default), any "_target_" is allowed.	None
strict	bool	If True, a "_target_" that is not a class and has no usable return type hint is refused before it is called (see resolve_object).	False

Returns:

Type	Description
BaseSaver[T]	The instantiated data saver.

Security

When saver is a configuration dict, its "_target_" value is imported and instantiated with no allowlist by default (see coola.factory.factory). Only call this function with a configuration that comes from a trusted source, or restrict it with allowed_prefixes; never resolve a saver configuration derived from untrusted input without it, as this is a remote-code-execution vector.

Example

>>> from coola.io import resolve_saver
>>> saver = resolve_saver({"_target_": "coola.io.JsonSaver"})
>>> saver
JsonSaver(encoding='utf-8', allow_nan=False, jsonable=False)

coola.io.json

Contain JSON-based data loaders and savers.

coola.io.json.JsonLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a JSON file.

Parameters:

Name	Type	Description	Default
encoding	str | None	The encoding of the file, which should be the one used to save it. If None (default), the file is read in binary mode and json detects the encoding, which only works for UTF-8, UTF-16 and UTF-32.	None

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.json.JsonSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a JSON file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
allow_nan	bool	If False (the default), NaN and infinite floats raise a ValueError, because the file would not be valid JSON. If True, they are written as the non-standard NaN/Infinity tokens.	False
jsonable	bool	If True, the data is converted with coola.jsonable.to_jsonable before it is written, so the objects that json does not support (tuples are lists, NumPy arrays, ...) are converted. Defaults to False.	False
**kwargs	Any	Additional arguments passed to json.dump, e.g. indent or sort_keys.	{}

Note

JSON has only string keys: int keys are loaded back as str.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.json.load_json

load_json(
    path: str | PathLike[str], encoding: str | None = None
) -> Any

Load the data from a given JSON file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the JSON file. A str or any path-like object is converted to a Path.	required
encoding	str | None	The encoding of the file. If None, the encoding is detected (UTF-8, UTF-16 and UTF-32 only).	None

Returns:

Type	Description
Any	The data from the JSON file.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.json.save_json

save_json(
    to_save: Any,
    path: str | PathLike[str],
    *,
    encoding: str = DEFAULT_ENCODING,
    exist_ok: bool = False,
    fsync: bool = False,
    allow_nan: bool = False,
    jsonable: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a JSON file.

The file is what json.dump writes: the keys that are not strings (for example int) are written as strings, so they are strings after load_json. NaN and infinite floats are not valid JSON, so they raise a ValueError unless allow_nan=True (they are then written as the non-standard tokens NaN and Infinity). The data is not converted unless jsonable=True: the objects that json does not support (NumPy arrays, ...) raise a TypeError.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a JSON file.	required
path	str | PathLike[str]	The path where to write the JSON file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
allow_nan	bool	If False (the default), NaN and infinite floats raise a ValueError. If True, they are written as the non-standard NaN/Infinity tokens.	False
jsonable	bool	If True, convert the data with coola.jsonable.to_jsonable before it is written. Defaults to False.	False
**kwargs	Any	Additional arguments passed to json.dump, e.g. indent or sort_keys.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.
ValueError	if the data has a NaN or an infinite float and allow_nan is False.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.pickle

Contain pickle-based data loaders and savers.

coola.io.pickle.PickleLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a pickle file.

Warning

load calls pickle.load on the given file. Unpickling can execute arbitrary code as a side effect of deserialization. Only load pickle files that come from a trusted source; never load a pickle file that comes from an untrusted or unauthenticated source.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.pickle.PickleSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a pickle file.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Additional arguments passed to pickle.dump.	{}

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import PickleSaver, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     PickleSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.pickle.load_pickle

load_pickle(path: str | PathLike[str]) -> Any

Load the data from a given pickle file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the pickle file. A str or any path-like object is converted to a Path.	required

Returns:

Type	Description
Any	The data from the pickle file.

Warning

This function unpickles the file's content, which can execute arbitrary code as a side effect of deserialization. Only load pickle files that come from a trusted source; never load a pickle file that comes from an untrusted or unauthenticated source.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.pickle.save_pickle

save_pickle(
    to_save: Any,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a pickle file.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a pickle file.	required
path	str | PathLike[str]	The path where to write the pickle file. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
**kwargs	Any	Additional arguments passed to pickle.dump.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.text

Contain text-based data loaders and savers.

coola.io.text.TextLoader

Bases: BaseLoader[str]

Implement a data loader to load data from a text file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when reading the file. Defaults to "utf-8".	DEFAULT_ENCODING

Note

The line endings are returned as they are in the file (no universal newline translation), so \\r\\n and \\r are preserved.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

coola.io.text.TextSaver

Bases: BaseFileSaver[str]

Implement a file saver to save data to a text file.

Parameters:

Name	Type	Description	Default
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING

Note

The line endings are written as they are in the string (no translation of \\n, even on Windows).

Raises:

Type	Description
TypeError	if the data to save is not a str. Call str on the data to save its text representation.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import TextSaver, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     TextSaver().save("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

coola.io.text.load_text

load_text(
    path: str | PathLike[str],
    encoding: str = DEFAULT_ENCODING,
) -> str

Load the data from a given text file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the text file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when reading the file. Defaults to "utf-8".	DEFAULT_ENCODING

Returns:

Type	Description
str	The text content of the file as a string.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

coola.io.text.save_text

save_text(
    to_save: str,
    path: str | PathLike[str],
    *,
    encoding: str = DEFAULT_ENCODING,
    exist_ok: bool = False,
    fsync: bool = False,
) -> None

Save the given data to a text file.

Parameters:

Name	Type	Description	Default
to_save	str	The string to write to the text file.	required
path	str | PathLike[str]	The path where to write the text file. A str or any path-like object is converted to a Path.	required
encoding	str	The file encoding to use when writing the file. Defaults to "utf-8".	DEFAULT_ENCODING
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False

Raises:

Type	Description
FileExistsError	if the file already exists.
TypeError	if the data to save is not a str. Call str on the data to save its text representation.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

coola.io.torch

Contain torch-based data loaders and savers.

coola.io.torch.TorchLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a PyTorch file.

Parameters:

Name	Type	Description	Default
weights_only	bool	If True (the default), only tensors and simple containers are loaded. Pass False only for trusted files.	True
**kwargs	Any	Additional arguments passed to torch.load.	{}

Security

Loading with weights_only=False can execute arbitrary code (it uses pickle). Only do so for files from a trusted source.

Raises:

Type	Description
MissingPackageError	If torch is not installed.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.torch.TorchSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a PyTorch file.

Parameters:

Name	Type	Description	Default
**kwargs	Any	Additional arguments passed to torch.save.	{}

Raises:

Type	Description
MissingPackageError	If torch is not installed.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import TorchSaver, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     TorchSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.torch.load_torch

load_torch(path: str | PathLike[str], **kwargs: Any) -> Any

Load the data from a given PyTorch file.

Parameters:

Name	Type	Description	Default
path	str | PathLike[str]	The path to the PyTorch file. A str or any path-like object is converted to a Path.	required
**kwargs	Any	Additional arguments passed to torch.load.	{}

Returns:

Type	Description
Any	The data from the PyTorch file.

Security

weights_only defaults to True, which restricts the unpickler to tensors and simple containers. Pass weights_only=False only for files from a trusted source: it uses pickle, which can execute arbitrary code.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

coola.io.torch.save_torch

save_torch(
    to_save: Any,
    path: str | PathLike[str],
    *,
    exist_ok: bool = False,
    fsync: bool = False,
    **kwargs: Any,
) -> None

Save the given data in a PyTorch file.

Parameters:

Name	Type	Description	Default
to_save	Any	Anyhe data to write in a PyTorch file.	required
path	str | PathLike[str]	The path where to write the PyTorch file. A str or any path-like object is converted to a Path.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
fsync	bool	If True, flush the data to the disk before committing the file, so it survives a power failure or a system crash. It is slower. Defaults to False: the file is replaced atomically, but its content may be lost if the machine crashes right after the call.	False
**kwargs	Any	Additional arguments passed to torch.save.	{}

Raises:

Type	Description
FileExistsError	if the file already exists.

Example

>>> import tempfile
>>> from pathlib import Path
>>> from coola.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}