Nested

coola.nested ¶

Helpers to reshape and query nested mapping data structures.

coola.nested.convert_to_dict_of_lists ¶

convert_to_dict_of_lists(
    seq_of_mappings: Sequence[Mapping[Any, Any]],
) -> dict[Any, list[Any]]

Convert a sequence of mappings to a dictionary of lists.

All the dictionaries should have the same keys. The first mapping in the sequence is used to find the keys.

Parameters:

Name	Type	Description	Default
`seq_of_mappings`	`Sequence[Mapping[Any, Any]]`	The sequence of mappings to convert.	required

Returns:

Type	Description
`dict[Any, list[Any]]`	A dictionary of lists.

Example

>>> from coola.nested import convert_to_dict_of_lists
>>> convert_to_dict_of_lists(
...     [{"key1": 1, "key2": 10}, {"key1": 2, "key2": 20}, {"key1": 3, "key2": 30}]
... )
{'key1': [1, 2, 3], 'key2': [10, 20, 30]}

coola.nested.convert_to_list_of_dicts ¶

convert_to_list_of_dicts(
    mapping_of_seqs: Mapping[Any, Sequence[Any]],
) -> list[dict[Any, Any]]

Convert a mapping of sequences to a list of dictionaries.

All the sequences should have the same length.

Parameters:

Name	Type	Description	Default
`mapping_of_seqs`	`Mapping[Any, Sequence[Any]]`	The mapping of sequences to convert.	required

Returns:

Type	Description
`list[dict[Any, Any]]`	A list of dictionaries.

Example

>>> from coola.nested import convert_to_list_of_dicts
>>> convert_to_list_of_dicts({"key1": [1, 2, 3], "key2": [10, 20, 30]})
[{'key1': 1, 'key2': 10}, {'key1': 2, 'key2': 20}, {'key1': 3, 'key2': 30}]

coola.nested.flatten_mapping ¶

flatten_mapping(
    mapping: Mapping[Any, Mapping[Any, Any]],
    on_duplicate: str = "raise",
    always_prefix: bool = False,
    separator: str = ".",
) -> dict[Any, Any]

Flatten a mapping of mappings into a single dict.

Each outer key is used as a prefix for its inner keys. If an inner key appears in more than one outer mapping with the same value in every occurrence, the plain inner key is kept and on_duplicate is not triggered. on_duplicate only applies when an inner key appears more than once with different values.

Parameters:

Name	Type	Description	Default
`mapping`	`Mapping[Any, Mapping[Any, Any]]`	The mapping of mappings to flatten.	required
`on_duplicate`	`str`	The strategy used to manage duplicate inner keys that have different values across outer mappings. The valid values are: - `'raise'`: raise an exception if an inner key appears more than once with different values. - `'first'`: keep the value from the first occurrence, under the plain inner key. - `'last'`: keep the value from the last occurrence, under the plain inner key. - `'prefix'`: keep all the values, renaming every occurrence of a conflicting key as `'{outer_key}{separator}{inner_key}'`.	`'raise'`
`always_prefix`	`bool`	If `True`, every key in the output is named `'{outer_key}{separator}{inner_key}'`, regardless of whether it is a duplicate. If `False`, only keys involved in a genuine conflict are prefixed (only relevant when `on_duplicate='prefix'`).	`False`
`separator`	`str`	The separator used to join the outer and inner keys when prefixing.	`'.'`

Returns:

Type	Description
`dict[Any, Any]`	The flattened dict.

Raises:

Type	Description
`ValueError`	if `on_duplicate` is not a valid value.
`KeyError`	if `on_duplicate='raise'` and an inner key appears more than once with different values.

Example

>>> from coola.nested import flatten_mapping
>>> flatten_mapping(
...     {"module1": {"a": 1, "b": 2}, "module2": {"b": 3, "c": 4}},
...     on_duplicate="prefix",
... )
{'a': 1, 'module1.b': 2, 'module2.b': 3, 'c': 4}

coola.nested.from_flat_dict ¶

from_flat_dict(
    data: dict[str, Any], separator: str = "."
) -> dict[str, Any]

Return a nested dict from a flat dict produced by to_flat_dict.

Each key in data is split on separator to reconstruct the nesting depth. Keys whose segments are all decimal integers are not converted to lists; the output is always a plain dict so the round-trip is lossless for the dict-of-dicts case and predictable for the list-originated case (integer string keys are preserved as strings).

Parameters:

Name	Type	Description	Default
`data`	`dict[str, Any]`	A flat dictionary whose keys use `separator` to encode nesting depth, as produced by :func:`to_flat_dict`.	required
`separator`	`str`	The separator that was used when the flat dict was created. Defaults to `"."`.	`'.'`

Returns:

Type	Description
`dict[str, Any]`	A nested `dict`.

Raises:

Type	Description
`ValueError`	If `separator` is empty, which would make key splitting ambiguous.
`ValueError`	If two keys in `data` imply conflicting types at the same path (e.g. one key treats a node as a dict and another treats it as a scalar leaf).

Example

>>> from coola.nested import from_flat_dict
>>> from_flat_dict({"str": "def", "module.component.float": 3.5, "module.component.int": 2})
{'str': 'def', 'module': {'component': {'float': 3.5, 'int': 2}}}
>>> # Integer-string keys are preserved as strings
>>> from_flat_dict({"module.0.0": 1, "module.0.1": 2, "module.1.bool": True, "str": "abc"})
{'module': {'0': {'0': 1, '1': 2}, '1': {'bool': True}}, 'str': 'abc'}

coola.nested.get_first_value ¶

get_first_value(data: Mapping[Any, T]) -> T

Get the first value of a mapping.

Parameters:

Name	Type	Description	Default
`data`	`Mapping[Any, T]`	The input mapping.	required

Returns:

Type	Description
`T`	The first value in the mapping.

Raises:

Type	Description
`ValueError`	if the mapping is empty.

Example

>>> from coola.nested import get_first_value
>>> get_first_value({"key1": 1, "key2": 2})
1

coola.nested.merge_mappings ¶

merge_mappings(
    mappings: Iterable[Mapping[Any, Any]],
    on_duplicate: str = "raise",
) -> dict[Any, Any]

Merge an iterable of mappings into a single dict.

If a key appears more than once with the same value in every occurrence, the value is kept as-is and on_duplicate is not triggered. on_duplicate only applies when a key appears more than once with different values.

Parameters:

Name	Type	Description	Default
`mappings`	`Iterable[Mapping[Any, Any]]`	The mappings to merge.	required
`on_duplicate`	`str`	The strategy used to manage duplicate keys that have different values across mappings. The valid values are: - `'raise'`: raise an exception if a key appears more than once with different values. - `'first'`: keep the value from the first occurrence. - `'last'`: keep the value from the last occurrence. - `'suffix'`: keep all the values by adding a `'_n'` suffix to the key name, where `n` is incremented for each new occurrence.	`'raise'`

Returns:

Type	Description
`dict[Any, Any]`	The merged dict.

Raises:

Type	Description
`ValueError`	if `on_duplicate` is not a valid value.
`KeyError`	if `on_duplicate='raise'` and a key appears more than once with different values.

Example

>>> from coola.nested import merge_mappings
>>> merge_mappings([{"key1": 1, "key2": 2}, {"key2": 3, "key3": 4}], on_duplicate="last")
{'key1': 1, 'key2': 3, 'key3': 4}

coola.nested.remove_keys_starting_with ¶

remove_keys_starting_with(
    mapping: Mapping[Any, Any], prefix: str
) -> dict[Any, Any]

Remove the keys that start with a given prefix.

Parameters:

Name	Type	Description	Default
`mapping`	`Mapping[Any, Any]`	The original mapping.	required
`prefix`	`str`	The prefix used to filter the keys.	required

Returns:

Type	Description
`dict[Any, Any]`	A new dict without the removed keys.

Example

>>> from coola.nested import remove_keys_starting_with
>>> remove_keys_starting_with(
...     {"key": 1, "key.abc": 2, "abc": 3, "abc.key": 4, 1: 5, (2, 3): 6},
...     "key",
... )
{'abc': 3, 'abc.key': 4, 1: 5, (2, 3): 6}

coola.nested.to_flat_dict ¶

to_flat_dict(
    data: object,
    prefix: str | None = None,
    separator: str = ".",
    to_str: type | tuple[type, ...] | None = None,
) -> dict[str, Any]

Return a flat representation of a nested structure as a dict using dotted keys.

Parameters:

Name	Type	Description	Default
`data`	`object`	The nested structure to flatten. Any :class:`Mapping` or non-string :class:`Sequence` is recursed into; everything else is treated as a leaf value.	required
`prefix`	`str \| None`	The prefix prepended to each key. `None` means no prefix; only valid when `data` is a mapping or sequence — a `ValueError` is raised if a bare scalar is passed without a prefix.	`None`
`separator`	`str`	The separator used to join nested keys.	`'.'`
`to_str`	`type \| tuple[type, ...] \| None`	A type or tuple of types that should be converted to their string representation instead of being recursed into. `None` (default) recurses into all supported containers.	`None`

Returns:

Type	Description
`dict[str, Any]`	A flat dictionary whose keys are the separator-joined
`dict[str, Any]`	paths to every leaf value.

Raises:

Type	Description
`ValueError`	If `data` is a scalar (non-mapping, non-sequence) and `prefix` is `None`, since this would produce a `None` key.

Example

>>> from coola.nested import to_flat_dict
>>> data = {
...     "str": "def",
...     "module": {
...         "component": {
...             "float": 3.5,
...             "int": 2,
...         },
...     },
... }
>>> to_flat_dict(data)
{'str': 'def', 'module.component.float': 3.5, 'module.component.int': 2}

>>> # Lists and tuples are also supported
>>> data = {
...     "module": [[1, 2, 3], {"bool": True}],
...     "str": "abc",
... }
>>> to_flat_dict(data)
{'module.0.0': 1, 'module.0.1': 2, 'module.0.2': 3, 'module.1.bool': True, 'str': 'abc'}

>>> # Use to_str to prevent recursion into specific types
>>> to_flat_dict(data, to_str=list)
{'module': "[[1, 2, 3], {'bool': True}]", 'str': 'abc'}