Skip to content

Migration Guide: v1.1 to v2.0

This guide explains how to migrate your code from coola version 1.1.x to version 2.0.x.

Version 2.0 is a large release. Most code that only calls objects_are_equal, hash_object or summarize keeps working, but the results of some calls change (hashes, summaries, a few comparisons). Code that extends coola (custom testers, hashers, summarizers, transformers, child finders, loaders or registries) must be ported.

How to use this guide

  1. Read Before you start and check the requirements.
  2. Go through the checklist and keep the items that match what you use.
  3. Open the section of each item: every section starts with who is affected, then lists the changes in a table (Before / After) and shows a code example when a port is needed.

At a glance

Area Main change Section
Python and extras Python 3.11 or later; new scipy and testing extras 1
Exceptions AlreadyRegisteredError, MissingPackageError and MaxDepthExceededError replace RuntimeError / RecursionError 2
coola.registry One storage class (Registry) and one dispatch class (TypeRegistry), fallbacks, pickling 3
coola.equality One engine; a tester returns a verdict or child pairs; compare replaces show_difference 4
coola.hashing One engine and encode; every hash changes 5
coola.summary One engine, one option object (SummaryConfig); new layout 6
coola.recursive Transformers implement expand; the engine recurses 7
coola.iterator One child finder (find_children) for dfs_iterate and bfs_iterate 8
coola.factory Strict contracts: one way to import, call and check a target 9
coola.random One seed domain, a strict set_rng_state, one registry 10
coola.reducer BaseReducer owns the checks; strict input 11
coola.nested Built on the engines; keyword-only options; stricter conversions 12
coola.utils Leaf layer: formatting and registry helpers moved; import helpers replaced 13
coola.display, coola.testing Optional rich; marks and fixtures modules 14, 15
coola.io, coola.identifier, coola.rounding, coola.validation New packages (nothing to migrate from the release 1.1.10) 16 to 19

Before you start

Requirements. coola 2.0 requires Python 3.11 or later (section 1).

Baseline. The baseline is the release 1.1.10. Some tables also compare with states of the development branch between 1.1.10 and 2.0, so a "Before" cell can name something the release never had. If you only used the release 1.1.10, skip these parts:

  • The packages coola.io (section 16), coola.identifier (17), coola.rounding (18), coola.validation (19), coola.formatting and coola.display.rich did not exist. They are new (see section 20).
  • These names were not in 1.1.10: DataclassMarker, BaseTypeDispatchRegistry, resolve_or, register_fallback, register_lazy, enter_dataclasses, SupportsHash, HashableHasher, _allowed_prefixes_, is_object_config, sample_torch_seed, LazySingleton, add_prefix_suffix_to_keys, coola.testing.registries, coola.testing.marks.requires and plural=, smallest_items and int_repr.

The rows that name a class, a function or an argument of 1.1.10 are the ones to apply.

Migration checklist

Everyone

  • [ ] Use Python 3.11 or later (section 1).
  • [ ] Regenerate every stored hash and stable identifier: the hash format changed (section 5).
  • [ ] Pass the options of objects_are_equal, hash_object, recursive_apply, random_seed, dfs_iterate, bfs_iterate and the register_* functions by keyword (sections 3 to 10).
  • [ ] Replace show_difference=True by compare(...) or assert_objects_equal(...) (section 4).
  • [ ] Catch AlreadyRegisteredError, MaxDepthExceededError and MissingPackageError instead of RuntimeError / RecursionError, and do not rely on max_depth=1000 (section 2).
  • [ ] Read the new summary layout if you compare or parse summary text (section 6).

If you extend coola

  • [ ] Equality: rewrite testers and handlers as testers that return a verdict or child pairs (section 4).
  • [ ] Hashing: port hashers to encode, replace SupportsHash.hash by __coola_hash__, move the secret policy to the registry (section 5).
  • [ ] Summary: port summarizers to summarize(data, config, *, compact) (section 6).
  • [ ] Recursive: port transformers to expand (section 7).
  • [ ] Iterator: port iterators and child finders to find_children (section 8).
  • [ ] Registry: replace DataclassMarker, find, resolve_or and BaseTypeDispatchRegistry (section 3).
  • [ ] Reducer: subclass BaseReducer instead of BaseBasicReducer (section 11).
  • [ ] I/O: rename load to _load in custom loaders and drop their equal (section 16).

If you use a specific package

  • [ ] Factory: replace _init_: __new__ and the _init_ names that start with _, and resolve nested configurations before passing them under an allow-list (section 9).
  • [ ] Random: replace the removed names and give set_rng_state a state with the registered keys (section 10).
  • [ ] Reducer: pass descending by keyword and create TorchReducer() explicitly for tensors (section 11).
  • [ ] Nested: pass the options by keyword, import get_first_value from coola.utils.mapping, and read the stricter conversions (section 12).
  • [ ] Utils: update the import paths, replace the import decorators and the raise_<pkg>_missing_error functions, and read the smaller behaviour changes (section 13).
  • [ ] Display: pass force of configure_colorlog_logging by keyword and read the new pydantic output (section 14).
  • [ ] Testing: import the marks from coola.testing.marks and load the fixture with pytest_plugins (section 15); install the testing extra (section 1).
  • [ ] SciPy sparse matrices: install the scipy extra to compare them (section 1).

Detailed changes

Sections 1 to 3 apply to the whole library. Sections 4 to 11 follow the packages in order of use, section 12 covers coola.nested, sections 13 to 15 the helpers, and sections 16 to 19 the new packages.

Part 1: Whole library (sections 1 to 3)

1. Packaging: Python 3.11, New Extras

Affects: Everyone who installs coola.

Before After
requires-python = ">=3.10" >=3.11 (Python 3.15 is supported)
No extra for SciPy pip install coola[scipy] (scipy>=1.13): the equality tester of the sparse matrices
coola.testing imports pytest with no declared dependency pip install coola[testing] (pytest>=8.0)
polars>=1.0,<2.0 polars>=1.0,<3.0
torch extra on every Python version Not installed on Python 3.15 (python_full_version < '3.15' marker)
dependencies = [] Unchanged: coola still has no required dependency

The jax, numpy, packaging, pandas, pyarrow, pydantic, xarray, colorlog and rich extras are unchanged. The development tools changed (ty replaces pyright; black and mutmut are removed), which matters to contributors only.

2. Exceptions That Changed Type

Affects: Code that catches RuntimeError, KeyError or RecursionError raised by coola.

Before After
registry.register(...), register_many(...) and the register_* functions raise RuntimeError for a duplicate AlreadyRegisteredError, a ValueError: except RuntimeError no longer catches it (exist_ok=True is unchanged)
registry[key], unregister(key) raise a KeyError KeyNotRegisteredError, or TypeNotRegisteredError for a type: both are KeyError (the message is no longer escaped by repr)
raise_package_missing_error, check_package and raise_<pkg>_missing_error raise RuntimeError MissingPackageError, a subclass of ModuleNotFoundError and RuntimeError: both except clauses still work
objects_are_equal and objects_are_allclose raise RecursionError beyond max_depth=1000 MaxDepthExceededError, an Exception, beyond max_depth=100: raise max_depth for a deeper structure
EmptySequenceError is an Exception A ValueError
timeblock("no placeholder") raises RuntimeError ValueError (the message must contain {time})
A rename that gives two equal keys keeps one of them DuplicateKeyError, a KeyError and a ValueError (section 12.3)
RecursionError for a deep structure in recursive_apply, dfs_iterate, summarize and the coola.nested functions No limit: the engines use an explicit stack (a cycle raises ValueError)

The messages of the validations changed too (section 19): a test that matches a message must be updated.

3. coola.registry: Two Classes, Fallbacks, Pickling

Affects: Code that builds, subclasses or registers into a registry, or catches its errors.

Before After
registry.register(DataclassMarker, handler) registry.register_fallback(dataclasses.is_dataclass, handler)
from coola.registry import DataclassMarker Removed; the dataclass handler is in registry.fallbacks(), not in items() / len()
BaseTypeDispatchRegistry, registry.find(t), registry.resolve_or(t) Removed; the registries of the packages are TypeRegistry subclasses: use registry.resolve(t)
registry.register(key, value, True), register_many(mapping, True) exist_ok is keyword-only: register(key, value, exist_ok=True)
registry.equal(other, equal_nan=True) registry.equal(other): the entries are compared with ==, and a registry equals its copy
A duplicate registration raises RuntimeError; a missing key raises KeyError AlreadyRegisteredError (a ValueError, not a RuntimeError); KeyNotRegisteredError (a KeyError)
A hasher for object makes the hashing fallbacks unreachable object is the last resort everywhere: type > fallback > object
HasherRegistry.register_fallback(predicate, hasher) accepts a duplicate predicate A duplicate predicate raises AlreadyRegisteredError (exist_ok=True replaces it, keeping its place)
HasherRegistry.unregister_fallback(predicate) returns None It returns the hasher and raises KeyNotRegisteredError for an unknown predicate
repr(registry) printed (state): TypeRegistry(...); a hashing fallbacks list One display: one (key): value line per entry, then (fallback) lines and the pending lazy groups
register_lazy(["builtins", "decimal"], f) then register_lazy("builtins", g) failed Both groups stay and load; register_lazy is a method of TypeRegistry only
pickle.dumps(registry) and copy.deepcopy(registry) raised TypeError They work; a spawn worker can receive a registry

Before:

from coola.registry import DataclassMarker

registry.register(DataclassMarker, MyDataclassHandler())

After:

from dataclasses import is_dataclass

registry.register_fallback(is_dataclass, MyDataclassHandler())

A handler registered for the type of a dataclass still has priority over the fallback. A fallback also supports the virtual subclasses of an abstract base class (the MRO lookup does not call issubclass): registry.register_fallback(lambda t: issubclass(t, Sequence), handler).

Part 2: Core packages (sections 4 to 11)

4. coola.equality: One Engine, Testers That Return a Verdict or Child Pairs

Affects: Custom equality testers and handlers, code that uses show_difference=True, and tests that depend on comparison edge cases.

The handler chain is replaced by testers that use an engine. A tester never recurses and never checks the identity, the type, the depth or the cycles: the engine does. The tester receives two objects of the same type and the engine (ctx), and returns either a verdict or the child pairs (actual, expected, path part) that must all be equal.

Example (v2.0):

>>> from coola.equality import AttributeName, BaseEqualityTester, compare, objects_are_equal
>>> class Point:
...     def __init__(self, x, y):
...         self.x = x
...         self.y = y
>>> class PointEqualityTester(BaseEqualityTester[Point]):
...     def objects_are_equal(self, actual, expected, ctx):
...         return (
...             (actual.x, expected.x, AttributeName("x")),
...             (actual.y, expected.y, AttributeName("y")),
...         )
>>> from coola.equality import get_default_registry
>>> registry = get_default_registry().copy()
>>> registry.register(Point, PointEqualityTester())  # or register_equality_testers({...})
>>> objects_are_equal(Point(1, 2.0), Point(1, 2.0), registry=registry)
True
>>> result = compare(Point(1, 2.0), Point(1, 3.0), registry=registry)
>>> result.path
('y',)

A leaf tester returns a verdict and gives the reason with ctx.fail, which returns False. The reason can be a function: it is called only when the difference is explained.

class VersionEqualityTester(BaseEqualityTester[Version]):
    def objects_are_equal(self, actual, expected, ctx):
        return actual.number == expected.number or ctx.fail(
            lambda: f"versions differ: {actual.number} vs {expected.number}"
        )

Migration of the v1.1 API (handlers and chain testers):

Before After
BaseChainEqualityTester, HandlerEqualityTester, create_chain A BaseEqualityTester with one method objects_are_equal(a, e, ctx)
BaseEqualityHandler, SameObjectHandler, SameTypeHandler, ... Removed: the engine checks the identity and the type
config.registry.objects_are_equal(a, b, config) (recursion) Return the child pair (a, b, path part)
report_difference(config, message, path=...) return ctx.fail(reason, *path)
check_recursion_depth, track_comparison, HandlerCycleError Removed: the engine owns the depth and the cycles
tester.equal(other) tester == other (same type and same attributes)
EqualityConfig(atol=..., rtol=...), config.reports EqualityConfig(tolerance=Tolerance(...)); the config is frozen
Engine(registry, config) Engine(config, explaining=False)
PandasDataFrameEqualityTester, PolarsSeriesEqualityTester, ... TableEqualityTester(adapter), registered by the default registry
from coola.equality.handler import ... Private (coola.equality._helpers); use coola.equality
RecursionError when max_depth is exceeded (default 1000) MaxDepthExceededError, a plain Exception (default max_depth=100)
objects_are_equal(..., show_difference=True) Removed (TypeError): see "Explaining a difference" below
BaseEqualityHandler and the handlers: SameObjectHandler, SameTypeHandler, SameLengthHandler, SameAttributeHandler, SameShapeHandler, SameDTypeHandler, SameDataHandler, TrueHandler, FalseHandler, EqualHandler, EqualNanHandler, ScalarEqualHandler, NanEqualHandler, ObjectEqualHandler, TolerantEqualHandler, AllCloseNanHandler, MappingSameKeysHandler, MappingSameValuesHandler, SequenceSameValuesHandler and the handlers of NumPy, JAX, torch, pandas, polars and PyArrow Removed: a tester returns a verdict or child pairs
The protocols SupportsEqual, SupportsEqualNan, SupportsAllCloseNan, SupportsTolerantEqual, SupportsShape, SupportsDType, SupportsData Removed: the objects keep the methods (equal(other), equal(other, equal_nan), allclose(other, rtol, atol, equal_nan)) and EqualEqualityTester, EqualNanEqualityTester and TolerantEqualEqualityTester call them (a protocol is not needed to register a type)
PandasDataFrameEqualityTester, PandasSeriesEqualityTester, PolarsDataFrameEqualityTester, PolarsSeriesEqualityTester, PolarsLazyFrameEqualityTester, PyarrowEqualityTester TableEqualityTester(adapter) (a polars LazyFrame is compared like any other table)
frame_equal, series_equal, handlers_are_equal, get_array_impl_class Removed

Changes of the results:

The rules are the same for every container and every library. A comparison can change in these cases:

  • bool and the members of an enum.Enum are labels: the tolerance never applies to them (also IntEnum, numpy.bool_ and pandas booleans).
  • An integer difference is computed in integers: 2**62 and 2**62 + 1 are different with atol=0.5 in an array, as they are in a list.
  • float16, bfloat16 and float32 are compared in double precision (no wrong True at the edge of the range).
  • MappingProxyType and array.array are compared like a mapping and a sequence.
  • pandas, polars and PyArrow numbers follow the tolerance; the nested and object cells are compared by the library (equals), without tolerance. The tolerance does not apply to an index.
  • A table (data frame, series, index, Arrow table) is one leaf of the explanation: the path stops at the table and the reason names the first column that differs.
  • OrderedDict keeps its order, a dict does not. A cycle is equal. A nesting deeper than max_depth raises MaxDepthExceededError.

Explaining a difference: show_difference removed.

show_difference=True (a log message) is replaced by two functions that explain the first difference only, on demand: compare returns a ComparisonResult and assert_objects_equal / assert_objects_allclose raise an AssertionError. objects_are_equal and objects_are_allclose format nothing.

Before:

import logging

logging.basicConfig(level=logging.INFO)
objects_are_equal(actual, expected, show_difference=True)

After:

>>> from coola.equality import compare, assert_objects_equal
>>> result = compare({"a": [1, 2, 3]}, {"a": [1, 2, 4]})
>>> result.equal
False
>>> result.path
('a', 2)
>>> print(result.format_message())
objects are not equal at actual['a'][2]
numbers are different:
  actual   : 3
  expected : 4
>>> assert_objects_equal({"a": 1}, {"a": 2})
Traceback (most recent call last):
    ...
AssertionError: objects are not equal at actual['a']
numbers are different:
  actual   : 1
  expected : 2

compare(actual, expected, rtol=0.0, atol=0.0, ...) takes the same keywords as objects_are_allclose (exact by default). ComparisonResult has equal, path (keys, positions, AttributeName, Element) and reason, and no actual / expected fields.

5. coola.hashing: One Engine, encode, a New Format

Affects: Everyone who stores a hash or a stable identifier, custom hashers, and code that handles secrets.

Every hash value changes (new encoding, format version coola-hash-2, short leaves inlined in the preimage of their parent), so every stable identifier changes too: generate_stable_content_id and generate_stable_uuid5 return other values. Regenerate any stored hash or identifier.

Before After
Hasher.hash(data, registry, *, length, ignore_unhashable) Hasher.encode(data): returns tagged(tag, payload) (a leaf) or Node(tag, children)
A hasher calls registry.hash(child, length=..., ignore_unhashable=...) The children are in the Node; the engine hashes them
hash_string, hash_bytes, hash_tagged, hash_enum, hash_path Removed: use hash_object; tagged and digest are the building blocks of a hasher
SupportsHash.hash(registry, length, ignore_unhashable), HashableHasher __coola_hash__(self) -> object (the content, not a hash), SupportsCoolaHash, ContentHasher
PydanticModelHasher(on_secret=...), hash_pydantic_model, unwrap_secrets SecretHasher(on_secret) and get_secret_hashers(on_secret), registered in a registry
"exclude" removed the key or the item of a secret "exclude" hashes a constant: the position of a secret is kept
A secret in a dict or a dataclass raised, or was TypeNotRegisteredError The policy of the registry applies wherever the secret is
hash_object(data, registry, 32), register_hashers(mapping, True) Keyword-only: hash_object(data, registry=registry, length=32), register_hashers(mapping, exist_ok=True)
HasherRegistry.hash(data, 16) HasherRegistry.hash(data, length=16)
ValueError("... nested too deeply") at about 500 levels No limit: the traversal uses an explicit stack
A DAG costs one visit per path A shared container is hashed once
length applied to every node: hash(x, length=16) was not a prefix of length=128 One digest cut at the end: a hash is the start of the longer ones
A dataclass field with compare=False was hashed Only the compare=True fields are hashed
memoryview hashed like bytes; UserString and MappingProxyType failed Own tag (memoryview), UserString and MappingProxyType are hashed
A pandas object named NaN and one named None had different hashes Same hash

Before:

class PointHasher(BaseHasher[Point]):
    def hash(self, data, registry, *, length=64, ignore_unhashable=False):
        return registry.hash((data.x, data.y), length=length, ignore_unhashable=ignore_unhashable)

After:

class PointHasher(BaseHasher[Point]):
    def encode(self, data):
        return Node("Point", [data.x, data.y])

To reveal or exclude secrets, build a registry:

registry = get_default_registry().copy()
registry.register_many(get_secret_hashers("reveal"), exist_ok=True)
hash_object(model, registry=registry)

6. coola.summary: One Engine, One Option Object

Affects: Custom summarizers, code that passes options to a summarizer, and code that parses summary text.

Before After
Summarizer.summarize(data, registry, depth, max_depth) returns a str and recurses Summarizer.summarize(data, config, *, compact) returns a str (a leaf) or a Node (a container)
registry.summarize(data, depth, max_depth) registry.summarize(data, config=None) with a SummaryConfig
SequenceSummarizer(max_items=3), DefaultSummarizer(max_characters=50), num_spaces=4 summarize(data, max_items=3, max_characters=50); the indentation is 2 spaces and is not an option
NDArraySummarizer(show_data=True), PandasDataFrameSummarizer(max_fields=2) summarize(data, show_data=True), summarize(data, max_items=2)
max_items=-1 and max_characters=-1 for "no limit" max_items=None and max_characters=None
summarize(data, 2) and register_summarizers(mapping, True) TypeError: summarize(data, max_depth=2) and register_summarizers(mapping, exist_ok=True)
max_depth=True and max_depth=1.5 accepted TypeError
BaseCollectionSummarizer, BaseFieldsSummarizer, MAX_FIELDS Removed. format_fields(fields, max_fields) has no default for max_fields
BaseMetadataSummarizer(show_data) with _describe(data) BaseMetadataSummarizer() with _describe(data) returning fields; a table returns ("columns", pairs)
repr and equal to write in every summarizer BaseSummarizer gives Name() and "same type"
RecursionError for a nesting of a few hundred levels No RecursionError: the engine uses an explicit stack
A self-referencing container is repeated up to max_depth <cycle> where it comes back
A __len__, a property or a __iter__ that raises makes summarize raise <class ...> <summary failed: ExcType> and the rest of the summary is intact
field(repr=False) fields of a dataclass are printed They are not shown
UserString is a list of characters, MappingProxyType is printed whole A leaf, and a mapping
A named tuple has the labels (0), (1) The field names
A container at the display depth is [0, 1, 2, 3, 4] ..., its str is computed in full [0, 1, 2, 3, 4, ...], bounded work (three levels, max_items items)
Hidden entries: ...; empty container: <class 'list'> []; nested levels indented by 2 then 4 ... (+N more); <class 'list'> (length=0); 2 spaces per level
A frame, a table or a dataset lists 10 columns It lists max_items (5) columns

A custom summarizer that returned a string only needs the new signature:

from coola.summary import BaseSummarizer, Node, SummaryConfig


class PointSummarizer(BaseSummarizer[Point]):
    def summarize(self, data: Point, config: SummaryConfig, *, compact: bool) -> str:
        return f"Point(x={data.x}, y={data.y})"  # a leaf


class TreeSummarizer(BaseSummarizer[Tree]):
    def summarize(self, data: Tree, config: SummaryConfig, *, compact: bool) -> Node:
        # a container: the engine recurses, summarizes the children with the summarizer of their
        # type and applies max_items, max_depth and the cycle check
        return Node(
            f"Tree (children={len(data.children)})", enumerate(data.children), len(data.children)
        )

A summarizer must be pure and cheap: it does not call the registry, does not recurse and returns the children lazily. A plugin of the entry-point group coola.summary.summarizers that implemented summarize(data, registry, depth, max_depth) must be ported. The options that were per type (max_items=3 for lists and 10 for dicts) do not exist any more: register a summarizer that returns a Node with a smaller total if you need it. The text of a summary is meant to be read, not parsed, and its layout changes in this release.

7. coola.recursive: the expand Protocol

Affects: Custom transformers, and code that relied on dataclasses being leaves.

Before After
Transformer.transform(data, func, registry) Transformer.expand(data): returns None (a leaf) or (children, rebuild)
A transformer recurses through registry.transform The engine recurses: a transformer cannot forget the cycle check or the depth limit
A dataclass is a leaf; enter_dataclasses() enters it A dataclass is entered by default; enter_dataclasses is removed
dfs_iterate and bfs_iterate: a dataclass is a leaf; enter_dataclasses() enters it A dataclass is entered by default; enter_dataclasses is removed from coola.iterator
dataclasses.replace rebuilds a dataclass A shallow copy with the fields set: init=False fields are transformed, no __post_init__
ConditionalTransformer Removed: guard func, or register IdentityTransformer for the type
recursive_apply(data, func, registry) recursive_apply(data, func, registry=registry)
KeyFilterTransformer() with func as the key predicate Removed (see section 12.3): use coola.nested.remove_keys_if
register_transformers(mapping, True) register_transformers(mapping, exist_ok=True)
RecursionError for a deep structure No limit: the traversal uses an explicit stack

Before:

class PointTransformer(BaseTransformer[Point]):
    def transform(self, data, func, registry):
        return Point(registry.transform(data.x, func), registry.transform(data.y, func))

After:

class PointTransformer(BaseTransformer[Point]):
    def expand(self, data):
        return [data.x, data.y], lambda xs: Point(*xs)

A plugin of the group coola.recursive.transformers that implements transform must be ported to expand. To keep a dataclass as a leaf, register DefaultTransformer() for its type (DefaultChildFinder() for dfs_iterate and bfs_iterate).

8. coola.iterator: One Child Finder for Both Orders

Affects: Custom iterators and child finders, and plugins of the groups coola.iterator.dfs and coola.iterator.bfs.

Before After
coola.iterator.dfs and coola.iterator.bfs Removed: import from coola.iterator
BaseIterator.iterate(data, registry) (DFS) and BaseChildFinder.find_children (BFS) BaseChildFinder.find_children(data): returns the children, or None for a leaf
IteratorRegistry, PathRegistry Removed: ChildFinderRegistry serves both orders
register_iterators and register_child_finders register_child_finders(mapping, *, exist_ok=False)
get_default_iterator_registry() get_default_registry() (alias get_default_child_finder_registry())
DefaultIterator, IterableIterator, MappingIterator, DataclassIterator DefaultChildFinder, IterableChildFinder, MappingChildFinder, DataclassChildFinder
A leaf overrides is_leaf (BFS) A leaf returns None from find_children; an empty container returns an empty iterable
registry.iterate(data) and registry.find_children(data) Removed: use dfs_iterate(data, registry=registry) or bfs_iterate(data, registry=registry)
dfs_iterate(data, registry) and bfs_iterate(data, registry) (deprecated) registry is keyword-only
Plugin groups coola.iterator.dfs and coola.iterator.bfs One group: coola.iterator.child_finders
RecursionError for a dfs_iterate structure nested about a thousand levels No limit: both orders use an explicit stack or queue
range, a user Collection and a user collections.abc.Set are entered They are leaves, like in recursive_apply (register a child finder to enter them)
bfs_iterate could raise a false Cycle detected for objects built by a child finder Fixed: the ancestors are compared by identity

Before:

class ReversedListIterator(BaseIterator[list]):
    def iterate(self, data, registry):
        for item in reversed(data):
            yield from registry.iterate(item)

After:

class ReversedListChildFinder(BaseChildFinder[list]):
    def find_children(self, data):
        return reversed(data)

A child finder returns the children in the order they are visited, and the same order is used by dfs_iterate and bfs_iterate: a custom order for one traversal only is no longer possible. A plugin of the group coola.iterator.dfs or coola.iterator.bfs must register a ChildFinder in coola.iterator.child_finders instead.

9. coola.factory: Strict Contracts

Affects: Configurations that use _init_ or _allowed_prefixes_, and code that nests configurations.

Before After
A target cannot receive an argument named obj or cls It receives every key that is not reserved, whatever its name
_init_="__new__" or a name that starts with _ TypeError: use a public class method or static method
_allowed_prefixes_ in the configuration of resolve_object (used when the caller has none) Ignored: the allow-list always comes from the caller
factory(..., _allowed_prefixes_=[...]) with an argument that is {"_target_": ...} ValueError: resolve the nested configuration first
resolve_object with a nested configuration and allowed_prefixes ValueError (same reason)
A dict with _target_ and a dict subclass cls Built (the key _target_ is reserved)
A dict without _target_ Returned if it is an instance of cls, else TypeError
Counter({"_target_": 2}) passed to resolve_object Resolved as a configuration
Missing module or attribute of a target ImportError: The target object does not exist: ... everywhere
A target that returns A \| B with no subclass of cls Refused before the call
is_object_config({"_target_": 5}, int) False (was TypeError)
Explicit _init_="__init__" with resolve_object The target is checked, like the default

Before:

resolve_object({"_target_": "mypkg.Model", "_init_": "__new__"})

After:

resolve_object({"_target_": "mypkg.Model", "_init_": "from_config", "config": {...}})

An allow-list is a filter on a path, not a sandbox: never allow a prefix that contains a function that resolves its arguments ("coola", "coola.factory", pickle, eval, ...).

10. coola.random: One Seed Domain, One Registry

Affects: Code that seeds NumPy or PyTorch through coola.random, and saved RNG states.

Before After
RandomRandomManager (coola.random.random) PythonRandomManager (coola.random.python)
numpy_seed(seed) and torch_seed(seed) NumpyRandomManager().seed(seed) and TorchRandomManager().seed(seed)
random_seed(seed, manager) random_seed(seed, *, registry=None)
register_managers(mapping, True) register_managers(mapping, exist_ok=True)
manager.manual_seed(-1), 1.5, True, 2**63 accepted or folded TypeError / ValueError: every entry validates the seed
NumPy: manual_seed(2**32 + 1) gives the numbers of manual_seed(1) A seed >= 2**32 is split in two 32-bit words: its numbers change
registry.has_manager(key) and registry.find_manager(key) key in registry (or registry.has(key)) and registry[key]
set_rng_state with different keys logs a warning and restores a part of the state ValueError, nothing restored; all the managers are restored before an error is raised
get_random_seed(seed) in [-2**63, 2**63 - 1) In [0, 2**63 - 1), valid for manual_seed: the value for a given seed changes
get_torch_generator(random_seed=1, device) (not exported) make_torch_generator(seed, *, device="cpu") from coola.random; seed is required
repr(RandomManagerRegistry) has a (state): Registry(...) line The managers are listed directly

A state saved with torch and restored on a machine without torch now raises a ValueError. Restore it into a registry with the same keys.

11. coola.reducer: One Base Class That Owns the Checks

Affects: Custom reducers, and code that calls auto_reducer() for tensors.

The changes of coola.reducer:

  • auto_reducer() returns NumpyReducer if NumPy is installed, else NativeReducer. It no longer returns TorchReducer when PyTorch is installed (it is slower on a list and slow to import): create TorchReducer() explicitly for tensors that live on a device.
  • BaseBasicReducer is removed. Subclass BaseReducer, which now owns the checks (empty input, iterator, options, shape) and calls the private computations _max, _mean, _median, _min, _quantile, _std and _sort. _is_empty and _check_values are removed; implement _prepare to convert or validate the input if needed.
  • sort(values, *, descending=False): descending is keyword-only.
  • EmptySequenceError is a ValueError.
  • The input is a sequence of real numbers (list, tuple, range) or a one-dimensional array / tensor for the backend. An iterator, a set, a str, an n-d or 0-d array, a complex number, an integer beyond 64 bits (NumPy and PyTorch) or a Decimal raise TypeError or ValueError with the same text in every backend (they used to return a value or raise AttributeError, RuntimeError, ...). NativeReducer rejects arrays and tensors: use NumpyReducer or TorchReducer.
  • quantile raises ValueError for a quantile outside [0, 1] (or NaN) in the three backends, before the values are read.
  • NativeReducer.mean uses statistics.fmean: it is much faster and the last bit of a float mean can differ.

Part 3: Nested data (section 12)

12. coola.nested: Built on the Engines

Affects: Users of coola.nested (flattening, key filtering, mapping helpers) and of the JSON conversion.

12.1 to_flat_dict on dfs_items

to_flat_dict joins the paths given by coola.iterator.dfs_items instead of walking the data with its own recursion.

Before After
A dataclass was a leaf of the flat dict It is entered: one entry per field ("p.x", "p.y")
A range was entered ("r.0", "r.1", ...) A leaf, like in the other packages
RecursionError for a deep structure No limit on the depth
Its own cycle check The cycle check of the iterator: ValueError
to_flat_dict(data, prefix, separator, to_str) (positional) Keyword-only options

12.2 Keyword-only options, smaller public surface

The options of the coola.nested functions are keyword-only, get_first_value moves next to sort_by_keys, and the helpers that only served one function are private.

Before After
to_flat_dict(data, "prefix", "/", str) to_flat_dict(data, prefix="prefix", separator="/", to_str=str)
flatten_mapping(mapping, "prefix", True, "/") flatten_mapping(mapping, on_duplicate="prefix", always_prefix=True, separator="/")
merge_mappings(mappings, "last") merge_mappings(mappings, on_duplicate="last")
add_prefix_suffix_to_keys(mapping, "x_", "_y", True) add_prefix_suffix_to_keys(mapping, "x_", "_y", recursive=True)
from_flat_dict(data, ".") from_flat_dict(data, separator="."); a key that is not a str raises TypeError
merge_mappings(..., on_duplicate="x"), flatten_mapping(..., on_duplicate="x") with a string Same call; the annotation is a Literal, and a new keyword equal_nan decides whether two NaN are equal duplicates
from coola.nested import get_first_value from coola.utils.mapping import get_first_value
coola.nested.try_sorted_list, coola.nested.conversion.SetToListTransformer Private: sort with sorted, or use coola.jsonable.to_jsonable for a set
coola.nested.polars.{is_nested_struct, unnest_with_separator, unnest_one_level, expand_list_columns} Private: use flatten_frame

12.3 Keys on the engine

remove_keys_* and add_prefix_suffix_to_keys(recursive=True) use the engine of coola.recursive and its default registry, instead of a traversal of their own.

Before After
coola.recursive.KeyFilterTransformer(key_predicate) Removed: coola.nested.remove_keys_if(mapping, predicate)
A dataclass was a leaf of remove_keys_* and add_prefix_suffix_to_keys It is entered, and rebuilt with the filtered or renamed fields
The nested OrderedDict, defaultdict were rebuilt as a dict (rename) The type of every container is kept, at every depth
remove_keys_if([1], ...) returned the list; the annotation said dict TypeError for a non-mapping; the result has the type of the input
RecursionError for a cycle or a deep structure ValueError for a cycle; no limit on the depth
A rename that gives two equal keys silently kept the last entry DuplicateKeyError

12.4 from_flat_dict without recursion

Before After
from_flat_dict({"a": {"0": 1}}, restore_lists=True) gave {"a": [1]} {"a": {"0": 1}}: a dict given as a value is kept as it is
RecursionError for about 3,000 levels with restore_lists=True No limit on the depth
The annotation of data was dict[str, Any] Mapping[str, Any] (any mapping worked)

12.5 convert_to_jsonable moved to coola.jsonable.to_jsonable

coola.nested.convert_to_jsonable and coola.nested.conversion.to_jsonable are removed, with no alias. The conversion is coola.jsonable.to_jsonable (see Converting Data to JSON), and coola/nested/conversion.py is now coola/nested/records.py.

Before After
convert_to_jsonable(x) and to_jsonable(x) coola.jsonable.to_jsonable(x); it enters containers and dataclasses, so to_jsonable([Path("a")]) converts the path
A value of an unknown type was returned unchanged (json.dumps failed later) TypeError that names the type. Register a ConvertTransformer with register_transformers, or pass on_unsupported="keep" for the old behaviour ("warn" shows which values are kept)
A tuple, a deque, a named tuple or a range kept its type (a range became a list) A list. OrderedDict, defaultdict and Counter become a dict
The keys were not converted: {(1, 2): 1} failed in json.dumps, {1: "a", "1": "b"} wrote "1" twice A key is a str, or written as json.dumps writes it ("true", "1", "null"); a Path, a UUID or an Enum key is converted; a tuple key raises TypeError; two keys that give one string raise ValueError
The output of NumPy and PyTorch was not checked: [(1+2j)], the objects of an object array, a datetime.date for datetime64, timedelta, bytes Complex numbers and objects raise (by default); a datetime64 coarser than the nanosecond gives an ISO string, a timedelta64 gives seconds, np.bytes_ gives Base64. A nanosecond datetime64 still gives an int
A pydantic model was dumped with model_dump(mode="json"): a NumPy field raised, a set field depended on PYTHONHASHSEED model_dump() (Python mode), converted by the registry: the sets are sorted and the arrays are converted. A serializer declared with when_used="json" is no longer applied (register ConvertTransformer(lambda m: m.model_dump(mode="json")) for the class)
pd.NaT gave "NaT"; pd.NA, pd.Series and pd.Index were returned unchanged None, None, and a list. A DataFrame raises TypeError: register the shape you want (df.to_dict(orient="records"))
to_jsonable(Point(1, 2)) converted the dataclass Same, and a set in a dataclass is sorted like every set, in every process
A tuple in a dataclass stayed a tuple A list
A set of mixed types ({1, "a"}), or with NaN, kept the iteration order Sorted by (type name, repr)
RecursionError for 2,000 nested dataclasses No limit; a cycle through a dataclass raises ValueError
coola.nested.try_sorted_list, coola.nested.conversion.SetToListTransformer Private

NaN and the infinities are still kept (as json.dumps writes them by default): pass allow_nan=False to json.dumps or save_json to refuse them.

12.6 Mappings and records

Before After
merge_mappings([{"a": 1}, {"a": 2}, {"a_1": 3}], on_duplicate="suffix") raised {'a': 1, 'a_2': 2, 'a_1': 3}: a generated key skips the keys of the inputs
flatten_mapping(on_duplicate="prefix") moved the renamed first occurrence to the end The first occurrence is renamed in place: the output follows the input order
convert_to_list_of_dicts({"a": {1, 2}}) paired the elements in an arbitrary order TypeError that names the type; the values must be Sequence
A generator as a value: TypeError: object of type 'generator' has no len() TypeError that names the type
convert_to_dict_of_lists(generator): 'generator' object is not subscriptable Any iterable of mappings is accepted
convert_to_dict_of_lists([{"a": 1, "b": 2}, {"a": 3}]) raised KeyError; an extra key of a later mapping was ignored ValueError that names the index and the keys: every mapping must have the keys of the first one
convert_to_list_of_dicts({"a": "xyz"}) split the string per character TypeError for a str, bytes or bytearray value

12.7 flatten_frame

Before After
from coola.nested.polars import flatten_frame from coola.nested import flatten_frame (imports polars on first use)
A LazyFrame was accepted (with a PerformanceWarning) TypeError: call collect() first
The version differences of polars were found with except TypeError compare_version: unnest(separator=) from 1.35.0, explode(empty_as_null=) from 1.36.0; an unrelated TypeError propagates

Part 4: Helpers (sections 13 to 15)

13. coola.utils: Leaf Layer, Formatting

Affects: Code that imports from coola.utils, uses its import helpers, or relies on the behaviour of its functions.

coola.utils imports the standard library and coola.validation only. Some modules moved, with no alias:

Before After
coola.utils.format coola.formatting.structure (str_*, repr_*) and coola.formatting.quantity (make_bar, byte sizes, durations)
coola.utils.string, coola.utils.secret, coola.utils.text_diff coola.formatting.text, coola.formatting.secret, coola.formatting.diff
coola.utils.conversion.to_jsonable coola.jsonable.to_jsonable
coola.utils.singleton.make_default_registry_singleton, reset_default_registries, temporary_default_registries coola.registry.default.<name> (plugin_group is keyword-only)
coola.utils.singleton.load_registry_plugins, get_loaded_plugins, LoadedPlugin, STRICT_PLUGINS_ENV_VAR, DISABLE_PLUGINS_ENV_VAR coola.registry.plugins.<name>
LazySingleton._replace_with_copy, _restore LazySingleton.set(value)
coola.utils.deprecation.deprecate_positional_options Removed: no function of coola is deprecated any more
coola.utils.path.working_directory contextlib.chdir
coola.utils.introspection.get_fully_qualified_name(obj) get_qualified_class_name(type(obj))
numpy_available, torch_available, ... (decorators that returned None), torch_numpy_available, decorator_package_available Removed. Use is_<pkg>_available()
raise_numpy_missing_error(), ... and raise_package_missing_error(name, install_cmd) raise_package_missing_error(name, *, install=None)
check_package(package, command) check_package(package, *, install=None): the message has the pip install and uv pip install commands, and the package is imported
LazyModule, lazy_import, coola.utils.imports.import_object("pkg.mod:name") Removed. Use pkgutil.resolve_name("pkg.mod:name")
patch("coola.utils.imports.numpy.is_numpy_available", ...) patch("coola.utils.imports.optional.is_numpy_available", ...)
str_indent(x, 4), str_mapping(m, True), smallest_items(data, 3) str_indent(x, num_spaces=4), str_mapping(m, sorted_keys=True), smallest_items(data, limit=3)
str_indent(x, num_spaces=-1): Incorrect num_spaces. Expected a non-negative integer...; quantile(v, [2.0]): quantiles must be in [0, 1] num_spaces must be greater than or equal to 0, got -1; quantiles must be in the range [0.0, 1.0], got 2.0
int_repr(-(1 << 20000)) returned 0x-1000... -0x1000...: the hash of a negative integer with more than 4300 digits changes
smallest_items(iter([3, "a", 1])) returned []; LazySingleton(lambda: s.reset()) deadlocked smallest_items takes a collection; reset and set raise RuntimeError from the factory
Rebuild error of recursive: Cannot rebuild an object of type builtins.frozenset ... ... of type frozenset ...

Smaller behaviour changes

The functions below keep their name and most of their behaviour.

Before After
check_env_vars(["X"]) reports a variable set to "" or " " as defined It is reported as not defined, like get_required_env_var; new missing_level= (default logging.WARNING) and the log messages have no emoji
to_date(datetime(2020, 1, 2, 3)) returns the datetime Returns date(2020, 1, 2)
sort_by_keys({"a": 1, 1: 2}) raises TypeError Sorts by type name first, then by key (by repr if the keys of one type cannot be compared)
sanitize_path("file://server/share/x") returns /share/x ValueError: a file:// URI with a host other than localhost is not supported
sanitize_path(path) always resolves the path sanitize_path(path, resolve=False) keeps a relative path and the symbolic links
find_root_package_parent("/dir/missing") starts from the path itself A path that is not a directory is treated as a file: the search starts from its parent
quantile(values, [2.0]) returns the largest value, quantile([], [0.5]) raises IndexError ValueError for a quantile outside [0, 1] and for empty values
batchify(items, size=0): size must be >= 1 size must be greater than or equal to 1, got 0 (ValueError, from coola.validation)
get_package_version("pkg") raises InvalidVersion for a version that is not PEP 440 Returns None, like a package that is not installed; the cache is reset by clear_availability_caches()
BloomFilter hashes with SHA-512; no add, no in BLAKE2b (the positions of the bits change: do not reuse a stored filter), add(item) and item in bloom, at most 32 hash functions
to_array(torch.ones(2, dtype=torch.bfloat16)) raises TypeError Converted to float32
to_tensor(array) with a negative stride raises ValueError The array is copied; MissingPackageError if PyTorch is not installed
hmac.compare_digest(password, confirmation) on str: TypeError for a non-ASCII password Compares the encoded bytes
make_fake_class(raise_error), make_fake_function(raise_error) (coola.utils.fallback.factory) make_fake_class(package), make_fake_function(package): the fake raises MissingPackageError
coola.__version__ is read when coola is imported Read the first time it is used (PEP 562): the value is the same

14. coola.display: Optional rich, Pydantic Formatting

Affects: Users of coola.display, the logging helpers and the pydantic formatting.

import coola.display.rich works without rich and imports no rich module: the first call of a function raises MissingPackageError.

Before After
import coola.display.rich without rich: ModuleNotFoundError The import works; the first call raises MissingPackageError (pip install rich)
import coola.display.rich imports rich It imports no rich module until a name is used
configure_colorlog_logging(logging.INFO, True) TypeError: force is keyword-only
configure_colorlog_logging always attached the coloured handler (raw ANSI escape codes in a file or in CI) The coloured handler is used only if colorlog is installed and the stream is a terminal; otherwise logging.basicConfig
configure_colorlog_logging(level, force=...) had no other option **kwargs go to logging.basicConfig (stream, datefmt, ...); handlers, filename, filemode, format or style bypass the coloured handler
str_pydantic_model(P(c=C(...))) shows c={'name': 'a'} Shows c=C(name=a): a nested model has its name, with the options of the root
A field with Field(repr=False) is shown; a field_serializer runs The field is hidden like in the repr of pydantic; no user code runs
A computed_field is shown (it is in model_dump()) It is not shown (it is not a field)
print_markdown(msg, max_length=-1) cut from the end silently; an open code fence in the cut text swallows the notice ValueError for max_length < 1; the cut prefers paragraph and line breaks and closes a fence
configure_rich_logging, make_progressbar, make_spinner always write to get_console() console= chooses the console; the default is unchanged

15. coola.testing: marks and fixtures

Affects: Test suites that import marks or fixtures from coola.testing.

The two modules of coola.testing are renamed after what they contain, and the fixture is loaded as a pytest plugin.

Before After
from coola.testing.fixtures import numpy_available, requires from coola.testing.marks import numpy_available, requires
from coola.testing.registries import isolated_default_registries pytest_plugins = ["coola.testing.fixtures"] in the top-level conftest.py
requires(True, "PyTorch and CUDA", plural=True) requires(True, "PyTorch with CUDA"): plural= is removed
Reasons "Requires PyTorch and CUDA" / "Skip if PyTorch and CUDA are available" "Requires PyTorch with CUDA" / "Skip if PyTorch with CUDA is available" (also for MPS and NumPy)

The names of the marks (numpy_available, numpy_not_available, ...) are unchanged. The re-export of the fixture in a conftest.py (with an __all__) is no longer needed.

Part 5: Packages that are new since 1.1.10 (sections 16 to 19)

16. coola.io: Template Methods, No Lock

Affects: Only code written against the development branch: the package is new since 1.1.10.

Before After
BaseLoader.load(path) is abstract and each loader calls sanitize_path BaseLoader._load(path: Path) is abstract; load is concrete and converts the path
Every loader and saver implements equal(other, equal_nan) equal is concrete in BaseLoader and BaseSaver: same type and same _display_fields()
InlineDisplayMixin is a base of each format It is a base of BaseLoader and BaseSaver
JsonLoader[T], PickleSaver[T], TorchLoader[T] BaseLoader[Any] / BaseFileSaver[Any]: the classes are not subscripted
save_text(123, path) writes 123; save_text(None, path) writes None TypeError for anything but a str: call str() first
TorchLoader(**kwargs), weights_only read from kwargs TorchLoader(*, weights_only=True, **kwargs); shown in the repr, compared by equal()
from coola.io import add_uuid_suffix; coola.io.utils from coola.utils.path import add_uuid_suffix
save_*(..., lock=True), BaseFileSaver.save(..., lock=True) lock is removed (TypeError); no .lock file, no heartbeat thread
Saves of one path are serialized Saves are atomic and never mix; which one wins is unspecified
exist_ok=False without hard links is protected by the lock Two processes creating the same file at the same instant can both succeed (last wins)
_fsync_file opens the file read-only It opens it with O_RDWR

Before:

class UpperTextLoader(BaseLoader[str]):
    def equal(self, other, equal_nan=False):
        return type(other) is type(self)

    def load(self, path):
        return sanitize_path(path).read_text().upper()

After:

class UpperTextLoader(BaseLoader[str]):
    def _load(self, path):
        return path.read_text().upper()

A caller that needs mutual exclusion around a read-modify-write of one file takes a lock of its own around the read and the write: save never provided it. A temporary file .<name>-<uuid><ext> left by a process killed during a save is hidden and safe to delete.

17. coola.identifier: Four Generators Removed, Strict Inputs, One Clock Rule

Affects: Only code written against the development branch: the package is new since 1.1.10.

generate_uuid4, generate_nano_id, generate_prefixed_id, generate_obfuscated_id and decode_obfuscated_id are removed: each is a line of the standard library or, for the obfuscated identifiers, an affine map that two known pairs break. The stable identifiers keep their values.

Before After
generate_uuid4() str(uuid.uuid4())
generate_nano_id(n, a) secrets.token_urlsafe(16), or "".join(secrets.choice(a) for _ in range(n))
generate_prefixed_id f"cus_{generate_ulid()}"
generate_obfuscated_id a library made for it (sqids), or a keyed permutation; never an affine map

The inputs are strict, and the extractors are the strict inverse of the generators:

Before After
extract_object_id_timestamp(object_id) extract_object_id_timestamp_s(object_id) (the unit is in the name)
generate_ulid(True), generate_snowflake_id(worker_id=True) TypeError: a bool is not an int
generate_ulid(1.5) raised AttributeError or TypeError from the code TypeError: timestamp_ms must be an int
extract_object_id_timestamp accepted whitespace, upper case and bytes Only 24 lowercase hexadecimal characters; TypeError for anything but a str
extract_uuid7_timestamp_ms accepted {...}, urn:uuid:..., no hyphens Only the canonical 36-character form (the case is ignored); pass str(uuid)
extract_uuid7_timestamp_ms(uuid.UUID(...)) raised AttributeError TypeError
extract_ulid_timestamp_ms rejected O, I and L They are read as 0, 1, 1 (Crockford), like verify_checksummed_id
extract_snowflake_timestamp_ms documented 64 bits and checked 63 ValueError outside [0, 2**63); TypeError for a bool or a non-int
ValueError: ... must fit in 48 bits ValueError: ... must be in the range [0, ...] (from coola.validation)
Before After
generate_ulid(ts), generate_uuid7(ts), generate_object_id(ts) Keyword-only: generate_ulid(timestamp_ms=ts), generate_object_id(timestamp_s=ts)
ObjectIdGenerator, objectid.get_default_generator() Removed: use generate_object_id(); pass timestamp_s for a deterministic test
ObjectId: 5-byte per-process value and 3-byte counter, no order in a second 4 bytes of seconds and a 64-bit incremented random counter: strictly increasing
identifier.monotonic.MonotonicRandom identifier.sequence.TickSequence(bits=..., tick_ns=..., random_start=...) (internal)

An ObjectId has the same 24 hexadecimal characters, the same timestamp field and the same sort order; only a tool that decodes the former process field and counter notices.

Before After
generate_snowflake_id(worker_id, timestamp_ms), get_default_generator() Removed: SnowflakeIdGenerator(worker_id=...) created once per process
SnowflakeIdGenerator().generate(worker_id, timestamp_ms) SnowflakeIdGenerator(worker_id=...).generate(timestamp_ms=...): no default worker_id
SnowflakeIdGenerator(last_timestamp_ms, sequence) Removed
RuntimeError("clock moved backward"); time.sleep when 4,096 IDs in a ms Never raises or sleeps: the timestamp is kept (or advanced) and the sequence goes on
A forked child continued the sequence of its parent (same IDs with worker_id=0) RuntimeError in the child: create a generator with its own worker_id
Before After
generate_checksummed_id(12, 4, "-"), verify_checksummed_id(id, "-") Keyword-only: generate_checksummed_id(length=12, group_size=4, sep="-"), verify_checksummed_id(id, sep="-")
5 of 37 identifiers ended with *, ~, $, = or U Never: the payload is drawn again; verify_checksummed_id still accepts the 5 symbols
sep could be a lowercase letter or O, I, L ValueError: sep must not be a character of the alphabet (case-insensitive)
verify_checksummed_id(id, sep=1) raised AttributeError TypeError (and ValueError for an invalid sep)

18. coola.rounding: One Rule, One Registry

Affects: Only code written against the development branch: the package is new since 1.1.10.

round_values follows one rule: a float is read as the decimal that repr shows, and the three modes act on that decimal (see the user guide).

Before After
round_values(2.675, decimals=2) is 2.67 for a float, 2.68 for a NumPy array 2.68 everywhere: the decimal is a halfway case, which goes to the even digit
round_values(8.995, decimals=2) is 8.99 9.0
ceil(5.5200000000000005, 2) is 5.52 (below the input) 5.53: floor(x) <= x <= ceil(x) always
polars: round_values(pl.Series([-4.822]), decimals=1) is -4.800000000000001 -4.8: polars goes through the NumPy kernel, the results are the same as NumPy
Rounding a polars or pyarrow object does not need NumPy It needs NumPy: without it UnsupportedTypeError (cause MissingPackageError), or on_missing_package="warn" / "ignore"
round_values(-0.4, mode="ceil") is 0.0 for a float -0.0, like numpy.ceil: a zero result has the sign of the input
np.matrix and ndarray subclasses become ndarray They keep their type (and a masked array its mask and fill value)
decimals=np.int64(1): TypeError Accepted (any integer; bool and float are still rejected)
pa.RecordBatch: TypeError Supported
TypeError: Unsupported type: ... for an unsupported object UnsupportedTypeError (a TypeError): Cannot round an object of type X; register a rounder with register_rounders
A sparse tensor: NotImplementedError from torch UnsupportedTypeError
recursive=True keeps a leaf that raises any TypeError It keeps only a leaf that raises UnsupportedTypeError; any other exception propagates
The dispatch is a hard-coded table register_rounders({Type: rounder}), get_default_registry(), round_values(..., registry=...), plugin group coola.rounding.rounders
coola.rounding.native.round_native(data, mode, decimals) Removed: round_float and round_int, the rounders registered for float and int

19. coola.validation: Eleven Functions, One Message Format

Affects: Only code written against the development branch: the package is new since 1.1.10.

coola.validation keeps the validators that state a constraint that isinstance or one comparison cannot state, and removes the synonyms, with no alias:

Before After
validate_bool(x), validate_float(x), validate_str(x), validate_list(x), validate_tuple(x), validate_dict(x), validate_set(x), validate_frozenset(x) validate_isinstance(x, bool), validate_isinstance(x, float), ...
validate_mapping(x) validate_isinstance(x, Mapping)
validate_positive(x), validate_non_negative(x) validate_gt(x, 0), validate_ge(x, 0)
validate_negative(x), validate_non_positive(x) validate_lt(x, 0), validate_le(x, 0)
validate_closed_range(x, low, high) validate_range(x, low, high)
validate_open_range(x, low, high) validate_range(x, low, high, low_inclusive=False, high_inclusive=False)
validate_half_open_range(x, low, high) validate_range(x, low, high, high_inclusive=False)
validate_range with low > high (validate_closed_range(0.5, 1, 0) blamed the value) ValueError: ratio: the range [1, 0] is empty
value must be an instance of <class 'int'>, got <class 'str'> value must be an instance of int, got str
value must be a number that can be compared with 0, got None value cannot be compared with 0, got None
value must be positive, got 0, ... in the closed range [0, 1], got 2 value must be greater than 0, got 0, ... in the range [0, 1], got 2
validate_finite("a"), validate_not_empty(5): a raw TypeError with no name TypeError: value must be a real number, got str, value must have a length, got int
validate_finite(True), validate_finite(Decimal("1")) were accepted TypeError: validate_finite checks validate_number first
validate_in([], {1}): TypeError: cannot use 'list' as a set element ValueError: value must be one of {1}, got []
validate_in(-1, list(range(100000))): a message of 688,919 characters The listing is limited to 20 items
hash_object(x, length="64"), length=True: ValueError TypeError: length must be an int, got str
round_values(x, mode="trunc"): Incorrect mode 'trunc'. The valid modes are: ... ValueError: mode must be one of ('round', 'floor', 'ceil'), got 'trunc'
merge_mappings(..., on_duplicate="x"): Incorrect on_duplicate value: 'x'. Valid values are {...} ValueError: on_duplicate must be one of ('raise', 'first', 'last', 'suffix'), got 'x'

Additions

20. New in 2.0

Affects: Nobody: these additions need no migration.

These additions need no migration. Each package has a page in the user guide.

Addition Where
Registry and TypeRegistry (fallback predicates, lazy groups, pickling, plugin loading), LazySingleton coola.registry, coola.utils.singleton
compare, assert_objects_equal, assert_objects_allclose, ComparisonResult, Tolerance; testers for dataclasses, sets, ordered mappings, objects with attributes, pydantic models, SciPy sparse matrices, NumPy time scalars, tables and DifferenceCloseEqualityTester coola.equality
Hashers for dataclasses, enums, sets, JAX, NumPy, pandas, polars, PyArrow, PyTorch and xarray objects, SecretHasher, get_stable_registry, tagged, digest coola.hashing
Summarizers for dataclasses, pydantic models, pandas, polars, PyArrow, xarray and JAX objects; SummaryConfig, Node coola.summary
dfs_items coola.iterator
flatten_frame, add_prefix_suffix_to_keys, DuplicateKeyError, escape= of to_flat_dict and from_flat_dict, restore_lists= coola.nested
PythonRandomManager, sample_torch_seed, make_torch_generator, validate_seed coola.random
coola.io (load_*, save_*, atomic saves), coola.identifier (ULID, UUID7, ObjectId, Snowflake, checksummed and stable identifiers), coola.rounding, coola.validation, coola.formatting, coola.display.rich New packages (sections 13, 14, 16, 17, 18 and 19)
get_file_size, ForkAware, track_fork, get_current_branch, get_current_commit_hash, is_git_repo, run_git_command, int_repr, CaptureHandler, LRUCache, safe_repr, safe_str, smallest_items, raise_recursion_limit_error coola.utils
MissingPackageError, is_scipy_available, check_scipy, is_module_imported, lazy_exports, availability_cache, clear_availability_caches coola.utils.imports
requires, scipy_available, scipy_not_available, torch_cuda_not_available, torch_mps_not_available, torch_numpy_not_available, isolated_default_registries coola.testing.marks, coola.testing.fixtures
BaseDisplayMixin (__repr__ and __str__ from _display_fields) coola.display
configure_colorlog_logging(..., **kwargs), sanitize_path(resolve=), check_env_vars(missing_level=) coola.display, coola.utils