Nested

coola.nested

Helpers to reshape and query nested mapping data structures.

coola.nested.add_prefix_suffix_to_keys

add_prefix_suffix_to_keys(
    mapping: Mapping[Any, Any],
    prefix: str = "",
    suffix: str = "",
    recursive: bool = False,
) -> dict[Any, Any]

Add a prefix and/or a suffix to the keys of a mapping.

Only keys of type str are renamed; every other key is kept as-is because a prefix/suffix cannot be concatenated to it.

Parameters:

Name	Type	Description	Default
mapping	Mapping[Any, Any]	The input mapping.	required
prefix	str	The prefix to prepend to the keys.	''
suffix	str	The suffix to append to the keys.	''
recursive	bool	If True, the prefix and suffix are also added to the keys of every nested mapping. If False, only the keys of the top-level mapping are renamed.	False

Returns:

Type	Description
dict[Any, Any]	A new dict with the renamed keys.

Example

>>> from coola.nested import add_prefix_suffix_to_keys
>>> add_prefix_suffix_to_keys({"key1": 1, "key2": 2}, prefix="prefix_")
{'prefix_key1': 1, 'prefix_key2': 2}
>>> add_prefix_suffix_to_keys(
...     {"key1": 1, "key2": {"key3": 3}}, suffix="_suffix", recursive=True
... )
{'key1_suffix': 1, 'key2_suffix': {'key3_suffix': 3}}

coola.nested.convert_to_dict_of_lists

convert_to_dict_of_lists(
    seq_of_mappings: Sequence[Mapping[Any, Any]],
) -> dict[Any, list[Any]]

Convert a sequence of mappings to a dictionary of lists.

All the mappings must have the same keys as the first mapping in the sequence, which is used to find the keys.

Parameters:

Name	Type	Description	Default
seq_of_mappings	Sequence[Mapping[Any, Any]]	The sequence of mappings to convert.	required

Returns:

Type	Description
dict[Any, list[Any]]	A dictionary of lists.

Raises:

Type	Description
ValueError	If a mapping's keys differ from the first mapping's keys.

Example

>>> from coola.nested import convert_to_dict_of_lists
>>> convert_to_dict_of_lists(
...     [{"key1": 1, "key2": 10}, {"key1": 2, "key2": 20}, {"key1": 3, "key2": 30}]
... )
{'key1': [1, 2, 3], 'key2': [10, 20, 30]}

coola.nested.convert_to_jsonable

convert_to_jsonable(data: Any) -> Any

Recursively convert a nested data structure to a JSON-compatible representation.

This function walks through nested containers (e.g. list, tuple, dict) and applies coola.utils.conversion.to_jsonable to every object, converting pydantic.BaseModel and dataclass objects found at any depth. Use to_jsonable directly if the data is a single, non-nested object.

Parameters:

Name	Type	Description	Default
data	Any	The nested data structure to convert.	required

Returns:

Type	Description
Any	The converted data, with the same structure as the input.

Example

>>> from dataclasses import dataclass
>>> from coola.nested import convert_to_jsonable
>>> @dataclass
... class Point:
...     x: int
...     y: int
...
>>> convert_to_jsonable([Point(x=1, y=2), {"key": Point(x=3, y=4)}])
[{'x': 1, 'y': 2}, {'key': {'x': 3, 'y': 4}}]

coola.nested.convert_to_list_of_dicts

convert_to_list_of_dicts(
    mapping_of_seqs: Mapping[Any, Sequence[Any]],
) -> list[dict[Any, Any]]

Convert a mapping of sequences to a list of dictionaries.

All the sequences must have the same length.

Parameters:

Name	Type	Description	Default
mapping_of_seqs	Mapping[Any, Sequence[Any]]	The mapping of sequences to convert.	required

Returns:

Type	Description
list[dict[Any, Any]]	A list of dictionaries.

Raises:

Type	Description
ValueError	If the sequences do not all have the same length.

Example

>>> from coola.nested import convert_to_list_of_dicts
>>> convert_to_list_of_dicts({"key1": [1, 2, 3], "key2": [10, 20, 30]})
[{'key1': 1, 'key2': 10}, {'key1': 2, 'key2': 20}, {'key1': 3, 'key2': 30}]

coola.nested.flatten_mapping

flatten_mapping(
    mapping: Mapping[Any, Mapping[Any, Any]],
    on_duplicate: str = "raise",
    always_prefix: bool = False,
    separator: str = ".",
) -> dict[Any, Any]

Flatten a mapping of mappings into a single dict.

Each outer key is used as a prefix for its inner keys. If an inner key appears in more than one outer mapping with the same value in every occurrence, the plain inner key is kept and on_duplicate is not triggered. on_duplicate only applies when an inner key appears more than once with different values.

Parameters:

Name	Type	Description	Default
mapping	Mapping[Any, Mapping[Any, Any]]	The mapping of mappings to flatten.	required
on_duplicate	str	The strategy used to manage duplicate inner keys that have different values across outer mappings. The valid values are: - 'raise': raise an exception if an inner key appears more than once with different values. - 'first': keep the value from the first occurrence, under the plain inner key. - 'last': keep the value from the last occurrence, under the plain inner key. - 'prefix': keep all the values, renaming every occurrence of a conflicting key as '{outer_key}{separator}{inner_key}', including the first occurrence. This differs from :func:merge_mappings's on_duplicate='suffix' strategy, which leaves the first occurrence under the plain key.	'raise'
always_prefix	bool	If True, every key in the output is named '{outer_key}{separator}{inner_key}', regardless of whether it is a duplicate. If False, only keys involved in a genuine conflict are prefixed (only relevant when on_duplicate='prefix').	False
separator	str	The separator used to join the outer and inner keys when prefixing.	'.'

Returns:

Type	Description
dict[Any, Any]	The flattened dict.

Raises:

Type	Description
ValueError	if on_duplicate is not a valid value.
KeyError	if on_duplicate='raise' and an inner key appears more than once with different values.

Example

>>> from coola.nested import flatten_mapping
>>> flatten_mapping(
...     {"module1": {"a": 1, "b": 2}, "module2": {"b": 3, "c": 4}},
...     on_duplicate="prefix",
... )
{'a': 1, 'module1.b': 2, 'module2.b': 3, 'c': 4}

coola.nested.from_flat_dict

from_flat_dict(
    data: dict[str, Any], separator: str = "."
) -> dict[str, Any]

Return a nested dict from a flat dict produced by to_flat_dict.

Each key in data is split on separator to reconstruct the nesting depth. Keys whose segments are all decimal integers are not converted to lists; the output is always a plain dict so the round-trip is lossless for the dict-of-dicts case and predictable for the list-originated case (integer string keys are preserved as strings).

Parameters:

Name	Type	Description	Default
data	dict[str, Any]	A flat dictionary whose keys use separator to encode nesting depth, as produced by :func:to_flat_dict.	required
separator	str	The separator that was used when the flat dict was created. Defaults to ".".	'.'

Returns:

Type	Description
dict[str, Any]	A nested dict.

Raises:

Type	Description
ValueError	If separator is empty, which would make key splitting ambiguous.
ValueError	If two keys in data imply conflicting types at the same path (e.g. one key treats a node as a dict and another treats it as a scalar leaf).

Example

>>> from coola.nested import from_flat_dict
>>> from_flat_dict({"str": "def", "module.component.float": 3.5, "module.component.int": 2})
{'str': 'def', 'module': {'component': {'float': 3.5, 'int': 2}}}
>>> # Integer-string keys are preserved as strings
>>> from_flat_dict({"module.0.0": 1, "module.0.1": 2, "module.1.bool": True, "str": "abc"})
{'module': {'0': {'0': 1, '1': 2}, '1': {'bool': True}}, 'str': 'abc'}

coola.nested.get_first_value

get_first_value(data: Mapping[Any, T]) -> T

Get the first value of a mapping.

Parameters:

Name	Type	Description	Default
data	Mapping[Any, T]	The input mapping.	required

Returns:

Type	Description
T	The first value in the mapping.

Raises:

Type	Description
ValueError	if the mapping is empty.

Example

>>> from coola.nested import get_first_value
>>> get_first_value({"key1": 1, "key2": 2})
1

coola.nested.merge_mappings

merge_mappings(
    mappings: Iterable[Mapping[Any, Any]],
    on_duplicate: str = "raise",
) -> dict[Any, Any]

Merge an iterable of mappings into a single dict.

If a key appears more than once with the same value in every occurrence, the value is kept as-is and on_duplicate is not triggered. on_duplicate only applies when a key appears more than once with different values.

Parameters:

Name	Type	Description	Default
mappings	Iterable[Mapping[Any, Any]]	The mappings to merge.	required
on_duplicate	str	The strategy used to manage duplicate keys that have different values across mappings. The valid values are: - 'raise': raise an exception if a key appears more than once with different values. - 'first': keep the value from the first occurrence. - 'last': keep the value from the last occurrence. - 'suffix': keep all the values by adding a '_n' suffix to the key name, where n is incremented for each new occurrence.	'raise'

Note

With on_duplicate='suffix', only the later occurrences of a conflicting key are renamed ('{key}_1', '{key}_2', ...); the first occurrence keeps the plain key. This differs from :func:flatten_mapping's on_duplicate='prefix' strategy, which renames every occurrence, including the first, once a genuine conflict is detected for a key.

Returns:

Type	Description
dict[Any, Any]	The merged dict.

Raises:

Type	Description
ValueError	if on_duplicate is not a valid value.
KeyError	if on_duplicate='raise' and a key appears more than once with different values.

Example

>>> from coola.nested import merge_mappings
>>> merge_mappings([{"key1": 1, "key2": 2}, {"key2": 3, "key3": 4}], on_duplicate="last")
{'key1': 1, 'key2': 3, 'key3': 4}

coola.nested.remove_keys_containing

remove_keys_containing(
    mapping: Mapping[Any, Any], substring: str
) -> dict[Any, Any]

Recursively remove the keys that contain a given substring.

Parameters:

Name	Type	Description	Default
mapping	Mapping[Any, Any]	The original mapping.	required
substring	str	The substring used to filter the keys.	required

Returns:

Type	Description
dict[Any, Any]	A new dict without the removed keys.

Example

>>> from coola.nested import remove_keys_containing
>>> remove_keys_containing(
...     {"key": 1, "key.abc": 2, "abc": 3, "abc.key": 4, 1: 5, (2, 3): 6},
...     "key",
... )
{'abc': 3, 1: 5, (2, 3): 6}
>>> remove_keys_containing(
...     {"abc": {"key": 1, "abc": 2}, "list": [{"key": 1, "abc": 2}]},
...     "key",
... )
{'abc': {'abc': 2}, 'list': [{'abc': 2}]}

coola.nested.remove_keys_if

remove_keys_if(
    mapping: Mapping[Any, Any],
    predicate: Callable[[Any], bool],
) -> dict[Any, Any]

Recursively remove the keys that satisfy a given predicate.

The mapping is traversed recursively using the same type-dispatch machinery as :func:coola.recursive.recursive_apply, so it works on arbitrary nested combinations of mappings (dict, OrderedDict, ...), sequences (list, tuple, named tuples, ...), and sets/frozensets, not just plain dict/ list/tuple. Every other value is kept as-is.

Parameters:

Name	Type	Description	Default
mapping	Mapping[Any, Any]	The original mapping.	required
predicate	Callable[[Any], bool]	A function that takes a key and returns True if the key (and its value) should be removed.	required

Returns:

Type	Description
dict[Any, Any]	A new dict without the removed keys.

Example

>>> from coola.nested import remove_keys_if
>>> remove_keys_if(
...     {"key": 1, "key.abc": 2, "abc": 3, "abc.key": 4, 1: 5, (2, 3): 6},
...     lambda key: isinstance(key, str) and "key" in key,
... )
{'abc': 3, 1: 5, (2, 3): 6}
>>> remove_keys_if(
...     {"abc": {"key": 1, "abc": 2}, "list": [{"key": 1, "abc": 2}]},
...     lambda key: isinstance(key, str) and "key" in key,
... )
{'abc': {'abc': 2}, 'list': [{'abc': 2}]}

coola.nested.remove_keys_starting_with

remove_keys_starting_with(
    mapping: Mapping[Any, Any], prefix: str
) -> dict[Any, Any]

Recursively remove the keys that start with a given prefix.

Parameters:

Name	Type	Description	Default
mapping	Mapping[Any, Any]	The original mapping.	required
prefix	str	The prefix used to filter the keys.	required

Returns:

Type	Description
dict[Any, Any]	A new dict without the removed keys.

Example

>>> from coola.nested import remove_keys_starting_with
>>> remove_keys_starting_with(
...     {"key": 1, "key.abc": 2, "abc": 3, "abc.key": 4, 1: 5, (2, 3): 6},
...     "key",
... )
{'abc': 3, 'abc.key': 4, 1: 5, (2, 3): 6}

coola.nested.to_flat_dict

to_flat_dict(
    data: object,
    prefix: str | None = None,
    separator: str = ".",
    to_str: type | tuple[type, ...] | None = None,
) -> dict[str, Any]

Return a flat representation of a nested structure as a dict using dotted keys.

Parameters:

Name	Type	Description	Default
data	object	The nested structure to flatten. Any :class:Mapping or non-string :class:Sequence is recursed into; everything else is treated as a leaf value.	required
prefix	str | None	The prefix prepended to each key. None means no prefix; only valid when data is a mapping or sequence — a ValueError is raised if a bare scalar is passed without a prefix.	None
separator	str	The separator used to join nested keys.	'.'
to_str	type | tuple[type, ...] | None	A type or tuple of types that should be converted to their string representation instead of being recursed into. None (default) recurses into all supported containers.	None

Returns:

Type	Description
dict[str, Any]	A flat dictionary whose keys are the separator-joined
dict[str, Any]	paths to every leaf value.

Raises:

Type	Description
ValueError	If data is a scalar (non-mapping, non-sequence) and prefix is None, since this would produce a None key.

Example

>>> from coola.nested import to_flat_dict
>>> data = {
...     "str": "def",
...     "module": {
...         "component": {
...             "float": 3.5,
...             "int": 2,
...         },
...     },
... }
>>> to_flat_dict(data)
{'str': 'def', 'module.component.float': 3.5, 'module.component.int': 2}

>>> # Lists and tuples are also supported
>>> data = {
...     "module": [[1, 2, 3], {"bool": True}],
...     "str": "abc",
... }
>>> to_flat_dict(data)
{'module.0.0': 1, 'module.0.1': 2, 'module.0.2': 3, 'module.1.bool': True, 'str': 'abc'}

>>> # Use to_str to prevent recursion into specific types
>>> to_flat_dict(data, to_str=list)
{'module': "[[1, 2, 3], {'bool': True}]", 'str': 'abc'}