iden.io

iden.io

Contain data loaders and savers.

iden.io.AutoFileLoader

Bases: BaseLoader[Any]

Implement a data loader to load data based on the file extension.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, AutoFileLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     loader = AutoFileLoader()
...     data = loader.load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.AutoFileLoader.add_loader classmethod

add_loader(
    extension: str,
    loader: BaseLoader,
    exist_ok: bool = False,
) -> None

Add a loader for a given file extension.

Parameters:

Name	Type	Description	Default
extension	str	The file extension.	required
loader	BaseLoader	The loader to use for the given file extension.	required
exist_ok	bool	If False, RuntimeError is raised if the extension already exists. This parameter should be set to True to overwrite the loader for an extension.	False

Raises:

Type	Description
RuntimeError	if a loader is already registered for the extension and exist_ok=False.

Example usage:

>>> from iden.io import AutoFileLoader, TextLoader
>>> AutoFileLoader.add_loader("text", TextLoader())

iden.io.AutoFileLoader.find_loader classmethod

find_loader(extension: str) -> BaseLoader

Find the loader associated to the file extension.

Parameters:

Name	Type	Description	Default
extension	str	The file extension.	required

Returns:

Type	Description
BaseLoader	The loader for the given file extension.

Raises:

Type	Description
TypeError	if the file extension is not registered.

Example usage:

>>> from iden.io import AutoFileLoader
>>> AutoFileLoader.find_loader("txt")
TextLoader()
>>> AutoFileLoader.find_loader("json")
JsonLoader()

iden.io.AutoFileLoader.has_loader classmethod

has_loader(extension: str) -> bool

Indicate if a loader is registered for the given file extension.

Parameters:

Name	Type	Description	Default
extension	str	The file extension.	required

Returns:

Type	Description
bool	True if a loader comparator is registered, otherwise False.

Example usage:

>>> from iden.io import AutoFileLoader
>>> AutoFileLoader.has_loader("txt")
True
>>> AutoFileLoader.has_loader("newtxt")
False

iden.io.BaseFileSaver

Bases: BaseSaver[T]

Define the base class to implement a file saver.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.BaseFileSaver.save

save(
    to_save: T, path: Path, *, exist_ok: bool = False
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	Path	Specifies the path where to save the data.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False

Raises:

Type	Description
FileExistsError	if the file already exists.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.BaseLoader

Bases: Generic[T], ABC

Define the base class to implement a data loader.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.BaseLoader.load abstractmethod

load(path: Path) -> T

Save the data into the given path.

Parameters:

Name	Type	Description	Default
path	Path	The path with the data to load.	required

Returns:

Type	Description
T	The data

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.BaseSaver

Bases: Generic[T], ABC

Define the base class to implement a data saver.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.BaseSaver.save abstractmethod

save(
    to_save: T, path: Path, *, exist_ok: bool = False
) -> None

Save the data into the given path.

Parameters:

Name	Type	Description	Default
to_save	T	The data to save. The data should be compatible with the saving engine.	required
path	Path	Specifies the path where to save the data.	required
exist_ok	bool	If exist_ok is False (the default), an exception is raised if the target path already exists.	False

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.JsonLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.JsonSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import JsonSaver, JsonLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     JsonSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = JsonLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.PickleLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a pickle file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_pickle, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.PickleSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a pickle file.

Parameters:

Name	Type	Description	Default
protocol	int	The pickle protocol. By default, it uses the highest protocol available.	HIGHEST_PROTOCOL

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import PickleSaver, PickleLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     PickleSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = PickleLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.TextLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a text file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_text, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

iden.io.TextSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a text file.

Note

If the data to save is not a string, it is converted to a string before to be saved by using str.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import TextSaver, TextLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     TextSaver().save("hello", path)
...     data = TextLoader().load(path)
...     data
...
'hello'

iden.io.TorchLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_torch, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.TorchSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import TorchSaver, TorchLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     TorchSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = TorchLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.YamlLoader

Bases: BaseLoader[Any]

Implement a data loader to load data in a YAML file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_yaml, YamlLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.yaml")
...     save_yaml({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = YamlLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.YamlSaver

Bases: BaseFileSaver[Any]

Implement a file saver to save data with a YAML file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import YamlSaver, YamlLoader
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.yaml")
...     YamlSaver().save({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = YamlLoader().load(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.is_loader_config

is_loader_config(config: dict) -> bool

Indicate if the input configuration is a configuration for a BaseLoader.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict	Specifies the configuration to check.	required

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseLoader object.

Example usage:

>>> from iden.io import is_loader_config
>>> is_loader_config({"_target_": "iden.io.JsonLoader"})
True

iden.io.is_saver_config

is_saver_config(config: dict) -> bool

Indicate if the input configuration is a configuration for a BaseSaver.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict	Specifies the configuration to check.	required

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseSaver object.

Example usage:

>>> from iden.io import is_saver_config
>>> is_saver_config({"_target_": "iden.io.JsonSaver"})
True

iden.io.load_json

load_json(path: Path) -> Any

Load the data from a given JSON file.

Parameters:

Name	Type	Description	Default
path	Path	Specifies the path to the JSON file.	required

Returns:

Type	Description
Any	The data from the JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.load_pickle

load_pickle(path: Path) -> Any

Load the data from a given pickle file.

Parameters:

Name	Type	Description	Default
path	Path	Specifies the path to the pickle file.	required

Returns:

Type	Description
Any	The data from the pickle file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.load_text

load_text(path: Path) -> str

Load the data from a given text file.

Parameters:

Name	Type	Description	Default
path	Path	Specifies the path where to the text file.	required

Returns:

Type	Description
str	The data from the text file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

iden.io.load_torch

load_torch(path: Path) -> Any

Load the data from a given JSON file.

Parameters:

Name	Type	Description	Default
path	Path	Specifies the path to the JSON file.	required

Returns:

Type	Description
Any	The data from the JSON file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.load_yaml

load_yaml(path: Path) -> Any

Load the data from a given YAML file.

Parameters:

Name	Type	Description	Default
path	Path	Specifies the path to the YAML file.	required

Returns:

Type	Description
Any	The data from the YAML file.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import load_yaml, save_yaml
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.yaml")
...     save_yaml({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_yaml(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.save_json

save_json(
    to_save: Any, path: Path, *, exist_ok: bool = False
) -> None

Save the given data in a JSON file.

Parameters:

Name	Type	Description	Default
to_save	Any	Specifies the data to write in a JSON file.	required
path	Path	Specifies the path where to write the JSON file.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False

Raises:

Type	Description
FileExistsError	if the file already exists.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_json, load_json
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.json")
...     save_json({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_json(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.save_pickle

save_pickle(
    to_save: Any,
    path: Path,
    *,
    exist_ok: bool = False,
    protocol: int = pickle.HIGHEST_PROTOCOL
) -> None

Save the given data in a pickle file.

Parameters:

Name	Type	Description	Default
to_save	Any	Specifies the data to write in a pickle file.	required
path	Path	Specifies the path where to write the pickle file.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False
protocol	int	Specifies the pickle protocol. By default, it uses the highest protocol available.	HIGHEST_PROTOCOL

Raises:

Type	Description
FileExistsError	if the file already exists.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_pickle, load_pickle
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pkl")
...     save_pickle({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_pickle(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.save_text

save_text(
    to_save: Any, path: Path, *, exist_ok: bool = False
) -> None

Save the given data in a text file.

Parameters:

Name	Type	Description	Default
to_save	Any	Specifies the data to write in a text file.	required
path	Path	Specifies the path where to write the text file.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False

Raises:

Type	Description
FileExistsError	if the file already exists.

Note

If the data to save is not a string, it is converted to a string before to be saved by using str.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_text, load_text
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.txt")
...     save_text("hello", path)
...     data = load_text(path)
...     data
...
'hello'

iden.io.save_torch

save_torch(
    to_save: Any, path: Path, *, exist_ok: bool = False
) -> None

Save the given data in a JSON file.

Parameters:

Name	Type	Description	Default
to_save	Any	Specifies the data to write in a JSON file.	required
path	Path	Specifies the path where to write the JSON file.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False

Raises:

Type	Description
FileExistsError	if the file already exists.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import save_torch, load_torch
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.pt")
...     save_torch({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_torch(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.save_yaml

save_yaml(
    to_save: Any, path: Path, *, exist_ok: bool = False
) -> None

Save the given data in a YAML file.

Parameters:

Name	Type	Description	Default
to_save	Any	Specifies the data to write in a YAML file.	required
path	Path	Specifies the path where to write the YAML file.	required
exist_ok	bool	If exist_ok is False (the default), FileExistsError is raised if the target file already exists. If exist_ok is True, FileExistsError will not be raised unless the given path already exists in the file system and is not a file.	False

Raises:

Type	Description
FileExistsError	if the file already exists.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> from iden.io import load_yaml, save_yaml
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     path = Path(tmpdir).joinpath("data.yaml")
...     save_yaml({"key1": [1, 2, 3], "key2": "abc"}, path)
...     data = load_yaml(path)
...     data
...
{'key1': [1, 2, 3], 'key2': 'abc'}

iden.io.setup_loader

setup_loader(
    loader: BaseLoader[T] | dict,
) -> BaseLoader[T]

Set up a data loader.

The data loader is instantiated from its configuration by using the BaseLoader factory function.

Parameters:

Name	Type	Description	Default
loader	BaseLoader[T] | dict	Specifies the data loader or its configuration.	required

Returns:

Type	Description
BaseLoader[T]	The instantiated data loader.

Example usage:

>>> from iden.io import setup_loader
>>> loader = setup_loader({"_target_": "iden.io.JsonLoader"})
>>> loader
JsonLoader()

iden.io.setup_saver

setup_saver(saver: BaseSaver[T] | dict) -> BaseSaver[T]

Set up a data saver.

The data saver is instantiated from its configuration by using the BaseSaver factory function.

Parameters:

Name	Type	Description	Default
saver	BaseSaver[T] | dict	Specifies the data saver or its configuration.	required

Returns:

Type	Description
BaseSaver[T]	The instantiated data saver.

Example usage:

>>> from iden.io import setup_saver
>>> saver = setup_saver({"_target_": "iden.io.JsonSaver"})
>>> saver
JsonSaver()