Jsonable

coola.jsonable

Utilities to convert nested data to JSON data.

to_jsonable turns any nested data into the values that json.loads returns, with one function, one registry and one transformer per type.

Properties:

• JSON data model: the output contains only dict with str keys, list, str, int, float, bool and None. A subclass of str, int or float (an IntEnum member) is kept: json writes it as its base value.
• Accepted by json: with on_unsupported="raise", json.dumps(to_jsonable(x)) never raises, and the function is idempotent.
• Strict and loud: a value whose type has no transformer raises a TypeError that names the type. "warn" and "keep" are explicit opt-outs.
• Type dispatch: the transformer is chosen from the type of the data, along the Method Resolution Order.
• Registry-extensible: a new type is supported by registering a transformer; the plugin group is coola.jsonable.transformers.
• Keys are strings: a key is converted by the registry, then written like json.dumps writes it. A key that cannot be a string raises a TypeError and two keys that give the same string raise a ValueError.
• Deterministic: a set becomes a sorted list, dictionaries keep their order, so the output is the same in every process.
• Pure: the input is never modified and the output shares no container with it.
• Cycle-safe: a container that contains itself raises a ValueError.
• Any depth: no RecursionError, the traversal uses an explicit stack.
• Consistent with siblings: dataclasses are entered, text and bytes are leaves, options are keyword-only.

Not guaranteed: a round trip to the Python types, strict JSON (NaN and the infinities are kept), the sharing of a container reachable by several paths (it is converted once per path).

Example

>>> from coola.jsonable import to_jsonable
>>> to_jsonable({"a": (1, 2), "b": {3, 1, 2}})
{'a': [1, 2], 'b': [1, 2, 3]}

coola.jsonable.ConvertTransformer

Bases: InlineDisplayMixin, BaseTransformer[T]

Transformer that replaces the data by func(data), which is converted in turn.

The result of func is converted by the engine like any other value, until only JSON scalars and containers remain. A function does not call the registry and does not convert its own result: it only maps a value to a simpler one (a path to a str, an enum member to its value, an array to a list).

Parameters:

Name	Type	Description	Default
func	Callable[[T], Any]	The function that maps the data to the value to convert. Its result must not be of the type of the data, because it would be converted by this transformer again, without end.	required

Raises:

Type	Description
TypeError	From expand, if func returns a value of the type of the data.

Example

>>> from pathlib import PurePosixPath
>>> from coola.jsonable import ConvertTransformer
>>> transformer = ConvertTransformer(str)
>>> children, rebuild = transformer.expand(PurePosixPath("a/b"))
>>> children
['a/b']
>>> rebuild(children)
'a/b'

coola.jsonable.DataclassTransformer

Bases: InlineDisplayMixin, BaseTransformer[Any]

Transformer for dataclass instances that rebuilds a dict.

The values of the fields are the children, so they are converted like the other values, and the converted values are collected in a dict that maps the names of the fields to them. The fields are read one by one: dataclasses.asdict is not used because it deep-copies the values.

Example

>>> from dataclasses import dataclass
>>> from coola.jsonable import DataclassTransformer
>>> @dataclass
... class Point:
...     x: int
...     y: int
...
>>> children, rebuild = DataclassTransformer().expand(Point(1, 2))
>>> children
[1, 2]
>>> rebuild(["1", "2"])
{'x': '1', 'y': '2'}

coola.jsonable.MappingTransformer

Bases: InlineDisplayMixin, BaseTransformer['Mapping[Any, Any]']

Transformer for mappings that rebuilds a dict with str keys.

The keys and the values are the children, so a key is converted by the registry like any other value (a Path gives a str). Then a str key is kept, and a bool, int, float or None key is written like json.dumps writes it ("true", "1", "1.5", "null"). The order of the mapping is kept.

Raises:

Type	Description
TypeError	From the function that rebuilds the dictionary, if a converted key is not a str, bool, int, float or None (a tuple gives a list).
ValueError	From the function that rebuilds the dictionary, if two keys give the same string.

Example

>>> from coola.jsonable import MappingTransformer
>>> transformer = MappingTransformer()
>>> children, rebuild = transformer.expand({"a": 1, 2: 3})
>>> children
['a', 2, 1, 3]
>>> rebuild(children)
{'a': 1, '2': 3}

coola.jsonable.SequenceTransformer

Bases: InlineDisplayMixin, BaseTransformer['Sequence[Any]']

Transformer for sequences (list, tuple, deque, range, ...) that rebuilds a list.

The elements are the children, so they are converted like the other values.

Example

>>> from coola.jsonable import SequenceTransformer
>>> transformer = SequenceTransformer()
>>> children, rebuild = transformer.expand((1, 2, 3))
>>> children
[1, 2, 3]
>>> rebuild(["1", "2", "3"])
['1', '2', '3']

coola.jsonable.SetTransformer

Bases: InlineDisplayMixin, BaseTransformer['AbstractSet[Any]']

Transformer for sets that rebuilds a sorted list.

The elements are the children, so they are converted like the other values, and the converted elements are collected in a list sorted with _sorted_elements. The sort does not depend on the iteration order of the set, so the output is the same in every process.

Example

>>> from coola.jsonable import SetTransformer
>>> transformer = SetTransformer()
>>> children, rebuild = transformer.expand({3, 1, 2})
>>> rebuild([3, 1, 2])
[1, 2, 3]

coola.jsonable.get_default_registry

get_default_registry() -> TransformerRegistry

Get or create the default global registry of the JSON transformers.

Returns a singleton registry that is pre-configured with the transformers of the built-in containers, of the standard library types and of the plugins of the group coola.jsonable.transformers.

Returns:

Type	Description
TransformerRegistry	A singleton TransformerRegistry.

Notes

The singleton pattern means modifications to the returned registry affect all future calls to this function. Use registry.copy() for an isolated registry.

Example

>>> from coola.jsonable import get_default_registry
>>> get_default_registry().transform({"a": (1, 2)}, lambda x: x)
{'a': [1, 2]}

coola.jsonable.register_transformers

register_transformers(
    mapping: Mapping[type, Transformer[Any]],
    *,
    exist_ok: bool = False,
) -> None

Register custom transformers to the default global registry.

This function extends the singleton registry returned by get_default_registry.

Parameters:

Name	Type	Description	Default
mapping	Mapping[type, Transformer[Any]]	Mapping of Python types to transformer instances.	required
exist_ok	bool	If False, raise an error when a type is already registered.	False

Raises:

Type	Description
AlreadyRegisteredError	If any type is already registered and exist_ok is False.

Example

>>> from coola.jsonable import ConvertTransformer, register_transformers, to_jsonable
>>> class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...
>>> register_transformers({Point: ConvertTransformer(lambda p: {"x": p.x, "y": p.y})})
>>> to_jsonable([Point(1, 2)])
[{'x': 1, 'y': 2}]

coola.jsonable.to_jsonable

to_jsonable(
    data: object,
    *,
    registry: TransformerRegistry | None = None,
    on_unsupported: str = "raise",
) -> Any

Convert nested data to JSON data.

The output contains only dict with str keys, list, str, int, float, bool and None: the values that json.loads returns. Tuples, deques, sets and ranges become lists (a set becomes a sorted list), mappings and dataclasses become dictionaries. The keys are converted to strings like json.dumps writes them. The conversion is one way and is not a round trip to the Python types. NaN and the infinite floats are kept.

The options must be passed by keyword.

Parameters:

Name	Type	Description	Default
data	object	The data to convert.	required
registry	TransformerRegistry | None	The registry that finds the transformer of each type. If None, the default registry is used.	None
on_unsupported	str	What happens to a value whose type has no transformer. "raise" raises a TypeError, "warn" issues a UserWarning and keeps the value, "keep" keeps the value silently. With "warn" and "keep" the result may contain objects that json rejects.	'raise'

Returns:

Type	Description
Any	The JSON data.

Raises:

Type	Description
ValueError	If on_unsupported is not "raise", "warn" or "keep", if a container contains itself, or if two keys give the same string.
TypeError	If a value has no transformer (with on_unsupported="raise") or if a key cannot be converted to a string.

Example

>>> from pathlib import PurePosixPath
>>> from coola.jsonable import to_jsonable
>>> to_jsonable({"path": PurePosixPath("a/b"), 1: (1, 2), "s": {3, 1, 2}})
{'path': 'a/b', '1': [1, 2], 's': [1, 2, 3]}