Hashing

coola.hashing ¶

Provide deterministic hashing functions.

These are content hashes for identity and caching, not for security: do not use them for passwords, signatures or integrity against an attacker.

Hashes are deterministic across calls, processes and machines, and follow these rules:

Every hash is domain-separated by a type tag, so values of different types never share a hash even if their text form is the same (1, 1.0, True and "1" are all distinct).
Ordered containers (list, tuple, OrderedDict) are order-sensitive and unordered containers (dict, set) are not, like in coola equality. Each container family has its own tag, so a list and a tuple with the same items hash differently.
Container hashes are built only from the hashes of their children, resolved through a HasherRegistry.
Unsupported types raise TypeNotRegisteredError (a KeyError) unless ignore_unhashable is set.
A container that contains itself raises ValueError.
The output is a lowercase hex string of length characters (an even number from 2 to 128, default 64).

coola.hashing.BaseHasher ¶

Bases: ABC, Generic[T]

Base class for type-specific hashers.

A hasher describes the content of a type, it does not hash. The rules of the contract are:

encode is pure and cheap: it does not call the registry, does not recurse, does not modify data and does not depend on id(), on hash() or on the iteration order of a set or of a mapping (an unordered Node says so instead).
It returns bytes (a leaf, see tagged) or a Node.
The children of a Node are encoded by their own hasher.
The tag is a constant of the hasher, except for the user-defined classes whose qualified name is the identity. A subclass therefore hashes like its base.
An exception propagates unchanged.

Example

>>> from coola.hashing import BaseHasher, Node, hash_object, register_hashers
>>> class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...
>>> class PointHasher(BaseHasher[Point]):
...     def encode(self, data):
...         return Node("Point", [data.x, data.y])
...
>>> register_hashers({Point: PointHasher()})
>>> hash_object(Point(1, 2)) == hash_object(Point(1, 2))
True
>>> hash_object(Point(1, 2)) == hash_object(Point(2, 1))
False

coola.hashing.BaseHasher.encode `abstractmethod` ¶

encode(data: T) -> Encoding

Describe the content of data.

Parameters:

Name	Type	Description	Default
`data`	`T`	The value to describe. Must be of the type `T` that this hasher handles.	required

Returns:

Type	Description
`Encoding`	The bytes of a leaf (`tagged(tag, payload)`), or a
`Encoding`	`Node` that gives the tag and the children of a
`Encoding`	container.

Example

>>> from coola.hashing import StringHasher
>>> StringHasher().encode("abc")
b'str\x00abc'

coola.hashing.BytesHasher ¶

Bases: InlineDisplayMixin, BaseHasher[bytes | bytearray | memoryview]

Hasher for bytes, bytearray and memoryview objects.

This hasher computes the hash of the bytes namespaced by the type tag (bytes, bytearray or memoryview), so a bytes, a bytearray and a memoryview with the same content hash differently.

Example

>>> from coola.hashing import BytesHasher
>>> hasher = BytesHasher()
>>> hasher
BytesHasher()
>>> len(hasher.encode(b"hello"))
70

coola.hashing.BytesHasher.encode ¶

encode(data: bytes | bytearray | memoryview) -> Encoding

Describe a bytes, a bytearray or a memoryview.

The content is reduced with digest so that the encoding is small, whatever the size of the buffer.

Parameters:

Name	Type	Description	Default
`data`	`bytes \| bytearray \| memoryview`	The bytes to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged(tag, digest(content))`.

coola.hashing.ContentHasher ¶

Bases: InlineDisplayMixin, BaseHasher[SupportsCoolaHash]

Hasher for objects that implement __coola_hash__.

The object is described by the tag custom:module.Class and the value of __coola_hash__, which the engine hashes through the registry. The qualified class name is part of the tag, so two classes with the same content hash differently. The default registry uses it as a fallback for every class that defines the method and has no hasher of its own.

Example

>>> from coola.hashing import ContentHasher, hash_object
>>> class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...     def __coola_hash__(self):
...         return (self.x, self.y)
...
>>> hasher = ContentHasher()
>>> hasher
ContentHasher()
>>> hasher.encode(Point(1, 2)).children
[(1, 2)]
>>> hash_object(Point(1, 2)) == hash_object(Point(1, 2))
True

coola.hashing.ContentHasher.encode ¶

encode(data: SupportsCoolaHash) -> Encoding

Describe an object by its content.

Parameters:

Name	Type	Description	Default
`data`	`SupportsCoolaHash`	The object to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` with the tag `custom:module.Class` and the
`Encoding`	value of `data.__coola_hash__()` as only child.

coola.hashing.DataclassHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Any]

Hasher for dataclass instances.

The hash combines the qualified class name with a mapping from field name to field value, with every value hashed through the registry. Only the fields with compare=True are hashed, like in objects_are_equal, so two equal dataclasses have the same hash. Because dataclasses have no common base class, it cannot be registered for a type: the default registry uses it as a fallback (see HasherRegistry.register_fallback) for every dataclass that has no hasher of its own.

Example

>>> from dataclasses import dataclass
>>> from coola.hashing import DataclassHasher
>>> @dataclass
... class Point:
...     x: int
...     y: int
...
>>> hasher = DataclassHasher()
>>> hasher
DataclassHasher()
>>> hasher.encode(Point(1, 2)).ordered
False

coola.hashing.DataclassHasher.encode ¶

encode(data: Any) -> Encoding

Describe a dataclass instance.

Parameters:

Name	Type	Description	Default
`data`	`Any`	The dataclass instance to describe.	required

Returns:

Type	Description
`Encoding`	The unordered `Node` of the names and the values of the
`Encoding`	fields with `compare=True`, with the qualified class name
`Encoding`	in the tag.

Raises:

Type	Description
`TypeError`	If `data` is not a dataclass instance.

coola.hashing.DatetimeHasher ¶

Bases: InlineDisplayMixin, BaseHasher[date | time]

Hasher for datetime.date, datetime.datetime and datetime.time objects.

This hasher converts the object to its ISO 8601 string representation via isoformat() and hashes that string under a tag made of the concrete class name.

Since datetime.datetime is a subclass of datetime.date, both types are handled by this hasher. The class name is part of the tag, so a date and a datetime never produce the same hash, even for the same calendar day.

Aware datetimes are converted to UTC first, so equal instants in different time zones hash the same, like they compare equal. Naive and aware datetimes differ. An aware time is normalised the same way, like time.__eq__ does: the offset is subtracted from the time of day (without wrapping around midnight) and the payload is the resulting number of microseconds. A naive time uses its ISO string.

Example

>>> from datetime import date
>>> from coola.hashing import DatetimeHasher
>>> hasher = DatetimeHasher()
>>> hasher
DatetimeHasher()
>>> hasher.encode(date(2021, 1, 1))
b'date\x002021-01-01'

coola.hashing.DatetimeHasher.encode ¶

encode(data: date | time) -> Encoding

Describe a date, a datetime or a time.

Parameters:

Name	Type	Description	Default
`data`	`date \| time`	The value to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged(tag, payload)`, with the tag `date`,
`Encoding`	`datetime` or `time` and the ISO form as payload (the
`Encoding`	microseconds in UTC for an aware time).

coola.hashing.EnumHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Enum]

Hasher for enum.Enum members.

The hash depends on the qualified class name and the member name, not on the member value. IntEnum and StrEnum members are therefore not collapsed into int and str.

Example

>>> from enum import Enum
>>> from coola.hashing import EnumHasher
>>> class Color(Enum):
...     RED = 1
...
>>> hasher = EnumHasher()
>>> hasher
EnumHasher()
>>> hasher.encode(Color.RED).startswith(b"Enum")
True

coola.hashing.EnumHasher.encode ¶

encode(data: Enum) -> Encoding

Describe an enum member.

The encoding depends on the qualified name of the enum class and on the member name, not on the member value, so it does not collapse IntEnum/StrEnum members into int/str. A combined flag has no name: its value is used instead.

Parameters:

Name	Type	Description	Default
`data`	`Enum`	The enum member to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged("Enum", "module.Class.NAME")`.

coola.hashing.Hasher ¶

Bases: Protocol[T_contra]

Structural interface of a type-specific hasher.

Any object with a compatible encode method can be registered without subclassing BaseHasher, which remains available as a convenience base class.

coola.hashing.Hasher.encode ¶

encode(data: T_contra) -> Encoding

Describe an object for hashing.

Parameters:

Name	Type	Description	Default
`data`	`T_contra`	The object to describe.	required

Returns:

Type	Description
`Encoding`	The `bytes` of a leaf, or a `Node` for a container.

coola.hashing.HasherRegistry ¶

Bases: TypeRegistry[Hasher[Any]]

Registry that manages and dispatches hashers based on data type.

This registry maintains a mapping from Python types to hasher instances and uses the Method Resolution Order (MRO) for type lookup. When hashing data, it automatically selects the most specific registered hasher for the data's type, falling back to parent types or a default hasher if needed.

Parameters:

Name	Type	Description	Default
`initial_state`	`dict[type, T] \| None`	Optional initial mapping of types to hashers. If provided, the state is copied to prevent external mutations.	`None`

Example

Basic usage with a sequence hasher:

>>> from coola.hashing import HasherRegistry, SequenceHasher, StrHasher
>>> registry = HasherRegistry({object: StrHasher(), list: SequenceHasher()})
>>> registry
HasherRegistry(
  (<class 'object'>): StrHasher()
  (<class 'list'>): SequenceHasher(tag='sequence')
)
>>> len(registry.hash([1, 2, 3]))
64

Registering custom hashers:

>>> from coola.hashing import HasherRegistry, SequenceHasher, StrHasher
>>> registry = HasherRegistry({object: StrHasher()})
>>> registry.register(list, SequenceHasher())
>>> len(registry.hash([1, 2, 3]))
64

Working with nested structures:

>>> from coola.hashing import get_default_registry
>>> registry = get_default_registry()
>>> data = {"a": [1, 2], "b": [3, 4]}
>>> len(registry.hash(data))
64

coola.hashing.HasherRegistry.find_hasher ¶

find_hasher(data_type: type) -> Hasher[Any]

Type-specific alias for find: find the appropriate hasher for a given type.

See TypeRegistry.resolve for the full behavior description (MRO resolution, caching, and the KeyError on no match). If no hasher is registered for the type or its parent types, the fallbacks (see register_fallback) are tried.

A subclass of Enum resolves to the hasher of Enum unless a hasher is registered for the enum class itself (or for a parent that is not a builtin base of the mixin). A member of class C(bytes, Enum) would otherwise hash as the plain value, like a member of a different enum with the same value.

Parameters:

Name	Type	Description	Default
`data_type`	`type`	The Python type to find a hasher for.	required

Returns:

Type	Description
`Hasher[Any]`	The most specific registered hasher for this type, resolved
`Hasher[Any]`	via MRO, or the hasher of the first matching fallback.

Raises:

Type	Description
`TypeNotRegisteredError`	If no hasher is found.

Example

>>> from collections.abc import Sequence
>>> from coola.hashing import HasherRegistry, SequenceHasher, StrHasher
>>> registry = HasherRegistry({object: StrHasher()})
>>> registry.register(Sequence, SequenceHasher())
>>> hasher = registry.find_hasher(list)
>>> hasher
StrHasher()

coola.hashing.HasherRegistry.has_hasher ¶

has_hasher(data_type: type) -> bool

Type-specific alias for has: check if a hasher is explicitly registered for the given type.

See Registry.has for the full behavior description.

Parameters:

Name	Type	Description	Default
`data_type`	`type`	The type to check.	required

Returns:

Type	Description
`bool`	`True` if a hasher is explicitly registered for this type, `False` otherwise.

Example

>>> from coola.hashing import HasherRegistry, SequenceHasher
>>> registry = HasherRegistry()
>>> registry.register(list, SequenceHasher())
>>> registry.has_hasher(list)
True
>>> registry.has_hasher(tuple)
False

coola.hashing.HasherRegistry.hash ¶

hash(
    data: object,
    *,
    length: int = 64,
    ignore_unhashable: bool = False,
) -> str

Hash the given data.

This is the registry-level entry point for hashing. It finds the hasher of each object with find_hasher and walks the nested structure with the engine (see coola.hashing.engine). The hash is the first length characters of the 128-character hex digest, so a hash is the start of the longer ones.

Parameters:

Name	Type	Description	Default
`data`	`object`	The data to hash. Can be a nested structure such as a `list`, `dict`, or `tuple`.	required
`length`	`int`	The length of the returned hex string: an even `int` between 2 and 128 inclusive.	`64`
`ignore_unhashable`	`bool`	If `True`, objects for which no hasher is registered (including their parent types via MRO) are replaced by a deterministic placeholder hash instead of raising a `TypeNotRegisteredError`. This also applies to nested objects. The placeholder only depends on the type, so any two unsupported objects of the same type hash the same; a `UserWarning` is emitted once per type. Defaults to `False`, which raises an error.	`False`

Returns:

Type	Description
`str`	A string representing the hash of the input data.

Raises:

Type	Description
`TypeNotRegisteredError`	If no hasher is registered for the type of `data` or of a nested object (or any of their parent types) and `ignore_unhashable` is `False`. It is a subclass of `KeyError`.
`ValueError`	If `length` is not valid, or if `data` contains itself (a self-referencing container).

Example

>>> from coola.hashing import get_default_registry
>>> registry = get_default_registry()
>>> len(registry.hash({"scores": [95, 87, 92], "name": "test"}))
64

coola.hashing.JaxArrayHasher ¶

Bases: InlineDisplayMixin, BaseHasher['jax.Array']

Hasher for jax.Array objects.

The hash depends on the dtype, the shape and the values, like NumpyArrayHasher, but a JAX array and a NumPy array with the same content hash differently. -0.0 hashes like 0.0.

Example

>>> import jax.numpy as jnp
>>> from coola.hashing import HasherRegistry, JaxArrayHasher
>>> hasher = JaxArrayHasher()
>>> hasher
JaxArrayHasher()
>>> hasher.encode(jnp.arange(3)).startswith(b"jax.Array")
True

coola.hashing.JaxArrayHasher.encode ¶

encode(data: Array) -> Encoding

Describe a JAX array.

Parameters:

Name	Type	Description	Default
`data`	`Array`	The array to describe.	required

Returns:

Type	Description
`Encoding`	The leaf that describes `data`: the dtype, the shape and
`Encoding`	the digest of the canonical values.

coola.hashing.MappingHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Mapping[Any, Any]]

Hasher for mapping types.

This hasher describes a mapping as a Node whose children are the keys and the values, which the engine hashes through the registry. The entries (a key and its value) are sorted by digest, not by key, so mappings with the same key-value pairs but different insertion orders produce the same hash, without requiring keys to be orderable. This matches coola equality, which ignores the order of the keys of a dict. Register the hasher with ordered=True for a mapping type whose order is meaningful, like OrderedDict: its equality is order-sensitive too.

The tag identifies the mapping family, so a dict and an OrderedDict with the same items hash differently.

Parameters:

Name	Type	Description	Default
`tag`	`str`	The type tag mixed into the hash. Defaults to `"mapping"`.	`'mapping'`
`ordered`	`bool`	If `True`, items are hashed in iteration order instead of being sorted. Defaults to `False`.	`False`

Example

>>> from coola.hashing import MappingHasher
>>> hasher = MappingHasher()
>>> hasher
MappingHasher(tag='mapping', ordered=False)
>>> hasher.encode({"a": 1}).tag
'mapping'

coola.hashing.MappingHasher.encode ¶

encode(data: Mapping[Any, Any]) -> Encoding

Describe a mapping.

Parameters:

Name	Type	Description	Default
`data`	`Mapping[Any, Any]`	The mapping to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the keys and values (key, value, key, value,
`Encoding`	...) with `group=2`. It is unordered unless the hasher was
`Encoding`	built with `ordered=True`.

coola.hashing.Node `dataclass` ¶

Describe a container: a tag and the children to hash.

Each child is encoded by its own hasher, found by the registry, so the children can be any value, including values built on the spot (a list of field names, a NumPy array, a str dtype).

Parameters:

Name	Type	Description	Default
`tag`	`str`	The constant tag of the hasher.	required
`children`	`Iterable[Any]`	The values to hash. They are consumed once, lazily.	required
`ordered`	`bool`	If `True`, the position of a child matters. If `False`, the groups of children are sorted by digest, so the hash does not depend on the iteration order.	`True`
`group`	`int`	The number of consecutive children that form one entry of an unordered node (a mapping uses `2`: key, value).	`1`

Example

>>> from coola.hashing import Node
>>> node = Node("Point", [1, 2])
>>> node.tag
'Point'

coola.hashing.NumpyArrayHasher ¶

Bases: InlineDisplayMixin, BaseHasher['np.ndarray']

Hasher for numpy.ndarray objects.

The hash depends on the dtype (with its byte order), the shape and the values, not on the memory layout: a non-contiguous view hashes like its contiguous copy. -0.0 hashes like 0.0, like in the equality. The items of an object array are hashed through the registry. For a masked array, the mask is part of the hash and the masked values are not.

Example

>>> import numpy as np
>>> from coola.hashing import HasherRegistry, NumpyArrayHasher
>>> hasher = NumpyArrayHasher()
>>> hasher
NumpyArrayHasher()
>>> hasher.encode(np.arange(3)).startswith(b"numpy.ndarray")
True

coola.hashing.NumpyArrayHasher.encode ¶

encode(data: ndarray) -> Encoding

Describe a NumPy array.

A numeric array is a leaf: the dtype, the shape and the digest of the canonical values (and of the mask for a masked array). An object array is a Node whose children are the dtype, the shape and the items, which the engine hashes through the registry.

Parameters:

Name	Type	Description	Default
`data`	`ndarray`	The array to describe.	required

Returns:

Type	Description
`Encoding`	The leaf or the `Node` that describes `data`.

coola.hashing.PandasDataFrameHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pd.DataFrame']

Hasher for pandas.DataFrame objects.

The hash depends on the column labels (and the name of the columns index), the index (its names, its dtype and its values) and, for each column, its dtype and its values (see PandasSeriesHasher).

Example

>>> import pandas as pd
>>> from coola.hashing import PandasDataFrameHasher
>>> hasher = PandasDataFrameHasher()
>>> hasher
PandasDataFrameHasher()
>>> hasher.encode(pd.DataFrame({"a": [1, 2]})).tag
'pandas.DataFrame'

coola.hashing.PandasDataFrameHasher.encode ¶

encode(data: DataFrame) -> Encoding

Describe a pandas data frame.

Parameters:

Name	Type	Description	Default
`data`	`DataFrame`	The data frame to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the columns index, the index and, for each
`Encoding`	column, its dtype and its values.

coola.hashing.PandasSeriesHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pd.Series']

Hasher for pandas.Series objects.

The hash depends on the name, the dtype, the index (its names, its dtype and its values) and the values. For a categorical dtype, the categories and the ordered flag are part of the dtype. The values are hashed as a NumPy array, so the elements of an object column are hashed through the registry. The values of the other extension dtypes are converted to an object array in which a missing value is None.

Example

>>> import pandas as pd
>>> from coola.hashing import PandasSeriesHasher
>>> hasher = PandasSeriesHasher()
>>> hasher
PandasSeriesHasher()
>>> hasher.encode(pd.Series([1, 2, 3])).tag
'pandas.Series'

coola.hashing.PandasSeriesHasher.encode ¶

encode(data: Series) -> Encoding

Describe a pandas series.

Parameters:

Name	Type	Description	Default
`data`	`Series`	The series to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the name, the dtype, the index and the
`Encoding`	values.

coola.hashing.PathHasher ¶

Bases: InlineDisplayMixin, BaseHasher[PurePath]

Hasher for pathlib.PurePath and pathlib.Path objects.

This hasher describes the POSIX string representation of the path as written. It never touches the filesystem and does not depend on the current working directory, so hashes are portable across machines. A relative path and the absolute path of the same location therefore hash differently; use ResolvedPathHasher to hash the real location instead.

Example

>>> from pathlib import Path
>>> from coola.hashing import PathHasher
>>> hasher = PathHasher()
>>> hasher
PathHasher()
>>> hasher.encode(Path("/tmp/file.txt"))
b'Path\x00/tmp/file.txt'

coola.hashing.PathHasher.encode ¶

encode(data: PurePath) -> Encoding

Describe a path.

Parameters:

Name	Type	Description	Default
`data`	`PurePath`	The path to describe.	required

Returns:

Type	Description
`Encoding`	The leaf ``tagged("Path", posix_form).

coola.hashing.PolarsDataFrameHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pl.DataFrame']

Hasher for polars.DataFrame objects.

The hash depends on the column names, the dtypes and the values.

Example

>>> import polars as pl
>>> from coola.hashing import PolarsDataFrameHasher
>>> hasher = PolarsDataFrameHasher()
>>> hasher
PolarsDataFrameHasher()
>>> hasher.encode(pl.DataFrame({"a": [1, 2]})).tag
'polars.DataFrame'

coola.hashing.PolarsDataFrameHasher.encode ¶

encode(data: DataFrame) -> Encoding

Describe a polars data frame.

Parameters:

Name	Type	Description	Default
`data`	`DataFrame`	The data frame to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the columns.

coola.hashing.PolarsSeriesHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pl.Series']

Hasher for polars.Series objects.

The hash depends on the name, the dtype and the values, null included.

Example

>>> import polars as pl
>>> from coola.hashing import PolarsSeriesHasher
>>> hasher = PolarsSeriesHasher()
>>> hasher
PolarsSeriesHasher()
>>> hasher.encode(pl.Series("a", [1, 2, 3])).tag
'polars.Series'

coola.hashing.PolarsSeriesHasher.encode ¶

encode(data: Series) -> Encoding

Describe a polars series.

Parameters:

Name	Type	Description	Default
`data`	`Series`	The series to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the name, the dtype and the values.

Raises:

Type	Description
`MissingPackageError`	If `numpy` is not installed and the series is numeric without null: the values are converted to a NumPy array.

coola.hashing.PyarrowArrayHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pa.Array | pa.ChunkedArray']

Hasher for pyarrow.Array and pyarrow.ChunkedArray objects.

The hash depends on the type and the values, null included. The chunk layout of a ChunkedArray is not part of the hash. The integer, floating-point and boolean arrays are hashed from their NumPy form (-0.0 and NaN are canonicalized like in NumPy) when NumPy is installed, so they are as fast as NumPy arrays; the other types are hashed through Python objects.

Example

>>> import pyarrow as pa
>>> from coola.hashing import PyarrowArrayHasher
>>> hasher = PyarrowArrayHasher()
>>> hasher
PyarrowArrayHasher()
>>> hasher.encode(pa.array([1, 2, 3])).startswith(b'pyarrow.Array')
True

coola.hashing.PyarrowArrayHasher.encode ¶

encode(data: Array | ChunkedArray) -> Encoding

Describe an array.

Parameters:

Name	Type	Description	Default
`data`	`Array \| ChunkedArray`	The array to describe.	required

Returns:

Type	Description
`Encoding`	A leaf (the type, the digest of the values and of the
`Encoding`	validity) for the fixed-width types, and a `Node` of the
`Encoding`	type and the Python values for the other types.

coola.hashing.PyarrowTableHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pa.Table | pa.RecordBatch']

Hasher for pyarrow.Table and pyarrow.RecordBatch objects.

The hash depends on the column names, the types and the values.

Example

>>> import pyarrow as pa
>>> from coola.hashing import PyarrowTableHasher
>>> hasher = PyarrowTableHasher()
>>> hasher
PyarrowTableHasher()
>>> hasher.encode(pa.table({"a": [1, 2]})).tag
'pyarrow.Table'

coola.hashing.PyarrowTableHasher.encode ¶

encode(data: Table | RecordBatch) -> Encoding

Describe a table or a record batch.

Parameters:

Name	Type	Description	Default
`data`	`Table \| RecordBatch`	The table to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the column names and the columns.

coola.hashing.PydanticModelHasher ¶

Bases: InlineDisplayMixin, BaseHasher[BaseModel]

Hasher for pydantic BaseModel objects.

The model is described by its qualified class name and its fields (extra fields included), with every field value hashed through the registry. The fields are unordered, so the hash does not depend on the order of the declaration. The secrets (SecretStr and SecretBytes) are values like the others: their hasher decides what happens (see SecretHasher), wherever the secret is.

Example

>>> from pydantic import BaseModel
>>> from coola.hashing import PydanticModelHasher
>>> class Point(BaseModel):
...     x: int
...     y: int
...
>>> hasher = PydanticModelHasher()
>>> hasher
PydanticModelHasher()
>>> hasher.encode(Point(x=1, y=2)).ordered
False

coola.hashing.PydanticModelHasher.encode ¶

encode(data: BaseModel) -> Encoding

Describe a pydantic model.

Parameters:

Name	Type	Description	Default
`data`	`BaseModel`	The pydantic model instance to describe.	required

Returns:

Type	Description
`Encoding`	The unordered `Node` of the field names and values (extra
`Encoding`	fields included), with the qualified class name in the tag.

coola.hashing.ReprHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Any]

Hasher for objects whose repr() is a reliable canonical representation.

This hasher converts the object to its repr() string and hashes it under a type tag. Instances of bool, int, float and complex (including subclasses) and the NumPy scalars use the builtin repr and the builtin type name as tag, so they hash like the equivalent builtin value (np.float32(1.5) hashes like 1.5, np.int64(3) like 3). Negative zero is normalised to zero, since -0.0 == 0.0. It is preferable to StrHasher for numeric types (int, float, complex, bool) because repr() guarantees round-trip accuracy for floating point values, whereas str() may lose precision on some platforms.

For the other types, the tag is the module and the qualified name of the class by default, so a user class never shares the tag of another class with the same name. Pass tag to register a constant tag for a type (ReprHasher("timedelta")): a subclass then hashes like its base.

Parameters:

Name	Type	Description	Default
`tag`	`str \| None`	The type tag used for the values that are not a `bool`, an `int`, a `float` or a `complex`. If `None`, the tag is derived from the class of the hashed object. Defaults to `None`.	`None`

Example

>>> from coola.hashing import ReprHasher
>>> hasher = ReprHasher()
>>> hasher
ReprHasher()
>>> hasher.encode(1234)
b'int\x001234'

coola.hashing.ReprHasher.encode ¶

encode(data: Any) -> Encoding

Describe an object by its repr().

Parameters:

Name	Type	Description	Default
`data`	`Any`	The object to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged(tag, repr(data))`. A `bool`, an
`Encoding`	`int`, a `float` or a `complex` (and the NumPy
`Encoding`	scalars) use the builtin type name as tag and the builtin
`Encoding`	`repr`.

coola.hashing.ResolvedPathHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Path]

Hasher for pathlib.Path objects that hashes the resolved location.

The path is resolved (following symlinks, collapsing ./.. segments, making it absolute) before hashing, so that two differently written but logically equivalent paths hash the same. The result depends on the filesystem and on the current working directory, so it is not portable across machines. If resolution fails, the unresolved path is hashed instead.

Example

>>> from pathlib import Path
>>> from coola.hashing import ResolvedPathHasher
>>> hasher = ResolvedPathHasher()
>>> hasher
ResolvedPathHasher()
>>> hasher.encode(Path("file.txt")).startswith(b"Path")
True

coola.hashing.ResolvedPathHasher.encode ¶

encode(data: Path) -> Encoding

Describe a path.

Parameters:

Name	Type	Description	Default
`data`	`Path`	The path to describe.	required

Returns:

Type	Description
`Encoding`	The leaf ``tagged("Path", posix_form) of the resolved path.

coola.hashing.SecretHasher ¶

Bases: InlineDisplayMixin, BaseHasher['pydantic.SecretStr | pydantic.SecretBytes']

Hasher for SecretStr and SecretBytes values.

A secret is a value with a hasher, so the policy is the one of the registry and applies wherever the secret is (in a model, a dict, a list, a dataclass...).

Parameters:

Name	Type	Description	Default
`on_secret`	`Literal['reveal', 'exclude', 'error']`	What to do with a secret: `"error"`: raise a `ValueError`. The message never contains the secret. It forces an explicit decision instead of hashing, or silently ignoring, secret data by accident. `"reveal"`: hash the real value (`get_secret_value()`). Use it only when the hash itself is treated as sensitive. `"exclude"`: hash a constant, so the value is never hashed and two objects that differ only by a secret have the same hash. The position of the secret is kept: it is part of the schema of the class. Defaults to `"error"`.	`_ERROR`

Raises:

Type	Description
`ValueError`	If `on_secret` is not one of the three values.

Example

>>> from pydantic import SecretStr
>>> from coola.hashing import SecretHasher
>>> hasher = SecretHasher("exclude")
>>> hasher
SecretHasher(on_secret='exclude')
>>> hasher.encode(SecretStr("a")) == hasher.encode(SecretStr("b"))
True

coola.hashing.SecretHasher.encode ¶

encode(data: SecretStr | SecretBytes) -> Encoding

Describe a secret.

Parameters:

Name	Type	Description	Default
`data`	`SecretStr \| SecretBytes`	The secret to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged("pydantic.SecretStr", value)` for
`Encoding`	`"reveal"` and `tagged("pydantic.SecretStr")` for
`Encoding`	`"exclude"` (`pydantic.SecretBytes` for a
`Encoding`	`SecretBytes`).

Raises:

Type	Description
`ValueError`	If `on_secret="error"`. The message never contains the secret.

coola.hashing.SequenceHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Sequence[Any]]

Hasher for sequence types.

This hasher describes a sequence as an ordered Node whose children are the items, which the engine hashes through the registry. Item order matters.

The tag identifies the sequence family, so sequences of different families with the same items (e.g. a list and a tuple) hash differently. Register one instance per family, e.g. SequenceHasher("list") for list. A named tuple is tagged with its class name and field names, so it differs from a plain tuple and from other named tuples.

Parameters:

Name	Type	Description	Default
`tag`	`str`	The type tag mixed into the hash. Defaults to `"sequence"`.	`'sequence'`

Example

>>> from coola.hashing import SequenceHasher
>>> hasher = SequenceHasher()
>>> hasher
SequenceHasher(tag='sequence')
>>> hasher.encode([1, 2, 3])
Node(tag='sequence', children=[1, 2, 3], ordered=True, group=1)

coola.hashing.SequenceHasher.encode ¶

encode(data: Sequence[Any]) -> Encoding

Describe a sequence.

Parameters:

Name	Type	Description	Default
`data`	`Sequence[Any]`	The sequence to describe.	required

Returns:

Type	Description
`Encoding`	The ordered `Node` of the items. A named tuple has the tag
`Encoding`	`"<tag>:Class(field1,field2)"`.

coola.hashing.SetHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Set[Any]]

Hasher for set types.

This hasher describes a set as an unordered Node whose children are the items, which the engine hashes through the registry. The digests are sorted, not the items, so the result is independent of iteration order without requiring items to be orderable.

The tag identifies the set family, so a set and a frozenset with the same items hash differently, and neither collides with a list of the same items.

Parameters:

Name	Type	Description	Default
`tag`	`str`	The type tag mixed into the hash. Defaults to `"set"`.	`'set'`

Example

>>> from coola.hashing import SetHasher
>>> hasher = SetHasher()
>>> hasher
SetHasher(tag='set')
>>> hasher.encode({1})
Node(tag='set', children={1}, ordered=False, group=1)

coola.hashing.SetHasher.encode ¶

encode(data: Set[Any]) -> Encoding

Describe a set.

Parameters:

Name	Type	Description	Default
`data`	`Set[Any]`	The set to describe.	required

Returns:

Type	Description
`Encoding`	The unordered `Node` of the items.

coola.hashing.StrHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Any]

Hasher for objects whose str() is a reliable canonical representation.

This hasher converts the object to its str() string and then computes the hash of that string. Unlike ReprHasher, which uses repr(), this hasher uses str(), which is more human-readable but does not guarantee round-trip accuracy for floating point values.

The tag identifies the type family. By default it is str: followed by the module and the qualified name of the class, so a user class never shares the tag of another class with the same name. Pass tag to register a constant tag for a type (StrHasher("UUID")): a subclass then hashes like its base.

Parameters:

Name	Type	Description	Default
`tag`	`str \| None`	The type tag mixed into the hash. If `None`, the tag is derived from the class of the hashed object. Defaults to `None`.	`None`

Example

>>> from coola.hashing import StrHasher
>>> hasher = StrHasher()
>>> hasher
StrHasher()
>>> hasher.encode("hello")
b'str:str\x00hello'

coola.hashing.StrHasher.encode ¶

encode(data: Any) -> Encoding

Describe an object by its str().

Parameters:

Name	Type	Description	Default
`data`	`Any`	The object to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged(tag, str(data))`.

coola.hashing.StringHasher ¶

Bases: InlineDisplayMixin, BaseHasher[str | UserString]

Hasher for string objects.

This hasher describes the string under the str type tag, without any intermediate conversion. str subclasses hash like plain strings (the enum members are dispatched to EnumHasher by the registry). A UserString is described by its text under the UserString tag, so it differs from a str.

Example

>>> from coola.hashing import StringHasher
>>> hasher = StringHasher()
>>> hasher
StringHasher()
>>> hasher.encode("Meowwwwww")
b'str\x00Meowwwwww'

coola.hashing.StringHasher.encode ¶

encode(data: str | UserString) -> Encoding

Describe a string.

A lone surrogate (for example in the result of os.fsdecode for a file name that is not valid UTF-8) is encoded with the surrogatepass error handler.

Parameters:

Name	Type	Description	Default
`data`	`str \| UserString`	The string to describe.	required

Returns:

Type	Description
`Encoding`	The leaf `tagged("str", data)`, or
`Encoding`	`tagged("UserString", str(data))` for a `UserString`.

coola.hashing.SupportsCoolaHash ¶

Bases: Protocol

Implement a protocol to represent objects that say what their content is.

The method returns the content, not a hash: the engine hashes it through the registry, so the object cannot bypass the protections of the engine and does not have to forward any option.

Example

>>> from coola.hashing import SupportsCoolaHash
>>> class Point:
...     def __init__(self, x, y):
...         self.x, self.y = x, y
...     def __coola_hash__(self):
...         return (self.x, self.y)
...
>>> isinstance(Point(1, 2), SupportsCoolaHash)
True

coola.hashing.SupportsCoolaHash.__coola_hash__ ¶

__coola_hash__() -> object

Return the content of this object.

Returns:

Type	Description
`object`	Any value that can be hashed, for example a tuple of the
`object`	attributes that identify the object.

coola.hashing.TorchTensorHasher ¶

Bases: InlineDisplayMixin, BaseHasher['torch.Tensor']

Hasher for torch.Tensor objects.

The hash depends on the dtype, the shape and the values. The device and requires_grad are not part of the hash, so a tensor and its copy on another device hash the same. -0.0 hashes like 0.0. A sparse tensor is hashed from its layout, its positions and its stored values, without a dense copy, so it does not hash like the dense tensor with the same values. A meta tensor has no values and raises a TypeError.

Example

>>> import torch
>>> from coola.hashing import HasherRegistry, TorchTensorHasher
>>> hasher = TorchTensorHasher()
>>> hasher
TorchTensorHasher()
>>> hasher.encode(torch.arange(3)).startswith(b"torch.Tensor")
True

coola.hashing.TorchTensorHasher.encode ¶

encode(data: Tensor) -> Encoding

Describe a PyTorch tensor.

The dense values on the CPU are described: the device, the layout and requires_grad are not part of the encoding.

Parameters:

Name	Type	Description	Default
`data`	`Tensor`	The tensor to describe.	required

Returns:

Type	Description
`Encoding`	The leaf that describes `data`.

Raises:

Type	Description
`TypeError`	If `data` is on the `meta` device.

coola.hashing.XarrayHasher ¶

Bases: InlineDisplayMixin, BaseHasher[Any]

Hasher for xarray.DataArray, xarray.Dataset and xarray.Variable objects.

The hash depends on the dimensions, the values, the coordinates and the attributes (and on the name of a DataArray). The values are hashed as NumPy arrays.

Parameters:

Name	Type	Description	Default
`tag`	`str`	The type tag mixed into the hash, e.g. `"DataArray"`.	required

Example

>>> import xarray as xr
>>> from coola.hashing import XarrayHasher
>>> hasher = XarrayHasher("DataArray")
>>> hasher
XarrayHasher(tag='DataArray')
>>> hasher.encode(xr.DataArray([1, 2], dims=["x"])).tag
'xarray.DataArray'

coola.hashing.XarrayHasher.encode ¶

encode(data: Any) -> Encoding

Describe an xarray object.

Parameters:

Name	Type	Description	Default
`data`	`Any`	The `Variable`, `DataArray` or `Dataset` to describe.	required

Returns:

Type	Description
`Encoding`	The `Node` of the parts of the object: the dimensions, the
`Encoding`	values and the attributes for a variable; the name, the
`Encoding`	variable and the coordinates for a data array; the
`Encoding`	variables, the coordinates and the attributes for a dataset.

coola.hashing.digest ¶

digest(data: bytes) -> bytes

Return the 64-byte BLAKE2b digest of the package.

A hasher uses it to reduce a big buffer (the values of an array, the content of a bytes) to a fixed-width value before it enters the encoding, so the engine never copies a big preimage.

Parameters:

Name	Type	Description	Default
`data`	`bytes`	The bytes to digest.	required

Returns:

Type	Description
`bytes`	The 64-byte digest.

Example

>>> from coola.hashing import digest
>>> len(digest(b"abc"))
64

coola.hashing.get_default_registry ¶

get_default_registry() -> HasherRegistry

Return the default global hasher registry.

The registry is created on the first call and reused on all subsequent calls (singleton pattern). It is pre-configured with hashers for common Python built-in types.

Returns:

Type	Description
`HasherRegistry`	A singleton `HasherRegistry` configured for common Python
`HasherRegistry`	built-in types.

Notes

Because the registry is a singleton, modifications to it (e.g. via register_hashers) affect all future calls to this function. If you need an isolated registry, create a new HasherRegistry instance directly.

Example

>>> from coola.hashing import get_default_registry
>>> registry = get_default_registry()
>>> len(registry.hash("meowwwwww"))
64

coola.hashing.get_secret_hashers ¶

get_secret_hashers(
    on_secret: Literal[
        "reveal", "exclude", "error"
    ] = _ERROR,
) -> dict[type, Hasher[Any]]

Get the hashers of SecretStr and SecretBytes for a policy.

Parameters:

Name	Type	Description	Default
`on_secret`	`Literal['reveal', 'exclude', 'error']`	The policy of the hashers, see `SecretHasher`. Defaults to `"error"`.	`_ERROR`

Returns:

Type	Description
`dict[type, Hasher[Any]]`	The mapping from `SecretStr` and `SecretBytes` to a
`dict[type, Hasher[Any]]`	`SecretHasher`, to give to `HasherRegistry.register_many`.

Example

>>> from pydantic import BaseModel, SecretStr
>>> from coola.hashing import get_default_registry, get_secret_hashers
>>> class Creds(BaseModel):
...     user: str
...     password: SecretStr
...
>>> registry = get_default_registry().copy()
>>> registry.register_many(get_secret_hashers("exclude"), exist_ok=True)
>>> registry.hash(Creds(user="a", password="x")) == registry.hash(
...     Creds(user="a", password="y")
... )
True

coola.hashing.get_stable_registry ¶

get_stable_registry() -> HasherRegistry

Return the registry of the stable identifiers.

This registry only has the hashers that are built in coola. Unlike the default registry, it does not load the coola.hashing.hashers entry points of the installed packages and it is not changed by register_hashers, so the identifiers computed with it depend on the version of coola and not on the installed packages or on the code that runs in the process. It is the registry used by generate_stable_content_id and generate_stable_uuid5 when no registry is given.

Returns:

Type	Description
`HasherRegistry`	A singleton `HasherRegistry` with the built-in hashers.

Notes

The registry is shared by the process: do not register hashers in it. Pass your own registry to the functions that accept one to opt out of the stability.

Example

>>> from coola.hashing import get_stable_registry
>>> registry = get_stable_registry()
>>> len(registry.hash("meowwwwww"))
64

coola.hashing.hash_object ¶

hash_object(
    data: object,
    *,
    registry: HasherRegistry | None = None,
    length: int = 64,
    ignore_unhashable: bool = False,
) -> str

Compute a hash of a nested data structure.

The hash is deterministic and depends on the type of every value, on the order of ordered containers, and not on the order of unordered ones. For example [1, 2], (1, 2) and {1, 2} all hash differently, while {"a": 1, "b": 2} and {"b": 2, "a": 1} hash the same. An OrderedDict is the exception among mappings: the order of its keys matters, so two OrderedDict with the same items in a different order hash differently.

Note

The hash of a dataclass, a pydantic model, an enum member or an object with a __coola_hash__ method includes the module and the qualified name of its class. The hash changes when the class is renamed or moved to another module, and a class defined in a script run as __main__ hashes differently from the same class imported. Do not persist these hashes (or the IDs derived from them, such as generate_stable_uuid5) across such a change.

The options are keyword-only: it was easy to write hash_object(data, 32) and pass 32 as the registry.

Parameters:

Name	Type	Description	Default
`data`	`object`	The data to hash. Can be a nested structure such as a `list`, `dict`, or `tuple`.	required
`registry`	`HasherRegistry \| None`	The registry used to resolve hashers for each data type. If `None`, the default registry is used.	`None`
`length`	`int`	The desired length of the returned hex string. Must be an even number between 2 and 128 inclusive. Defaults to `64`.	`64`
`ignore_unhashable`	`bool`	If `True`, objects for which no hasher is registered (including nested objects found while hashing sequences or mappings) are replaced by a deterministic placeholder hash instead of raising an error. The placeholder only depends on the type, so it is lossy, and a `UserWarning` is emitted each time one is used. Defaults to `False`, which raises a `TypeNotRegisteredError` (a `KeyError`) when an unhashable object is encountered.	`False`

Returns:

Type	Description
`str`	A string representing the hash of the input data.

Raises:

Type	Description
`TypeNotRegisteredError`	If `data` (or a nested object within it) has a type for which no hasher is registered and `ignore_unhashable` is `False`. It is a subclass of `KeyError`.
`TypeError`	If `length` is not an `int` (`bool` is rejected too).
`ValueError`	If `length` is not an even number between 2 and 128, or if `data` contains itself (a self-referencing container).

Example

>>> from coola.hashing import hash_object
>>> len(hash_object({"a": 1, "b": "abc"}))
64
>>> len(hash_object({"a": 1, "b": "abc"}, length=16))
16

coola.hashing.register_hashers ¶

register_hashers(
    mapping: Mapping[type, Hasher[Any]],
    *,
    exist_ok: bool = False,
) -> None

Register custom hashers into the default global registry.

This function extends the singleton registry returned by get_default_registry with additional type-to-hasher mappings.

Parameters:

Name	Type	Description	Default
`mapping`	`Mapping[type, Hasher[Any]]`	Mapping of Python types to hasher instances.	required
`exist_ok`	`bool`	If `False` (default), raises an error if any type is already registered. If `True`, overwrites existing registrations silently.	`False`

Raises:

Type	Description
`AlreadyRegisteredError`	If any type is already registered and `exist_ok` is `False`.

Example

>>> from coola.hashing import register_hashers, StrHasher
>>> class MyType:
...     def __init__(self, value):
...         self.value = value
...
>>> register_hashers({MyType: StrHasher()})

coola.hashing.tagged ¶

tagged(tag: str, payload: str | bytes = b'') -> bytes

Return the encoding of a leaf: the tag, a NUL byte and the payload.

The text is encoded in UTF-8 with surrogatepass, so a lone surrogate never collides with a valid string.

Parameters:

Name	Type	Description	Default
`tag`	`str`	The constant tag of the type.	required
`payload`	`str \| bytes`	The content of the value.	`b''`

Returns:

Type	Description
`bytes`	The bytes of the leaf.

Example

>>> from coola.hashing import tagged
>>> tagged("int", "1")
b'int\x001'

Hashing

coola.hashing ¶

coola.hashing.BaseHasher ¶

coola.hashing.BaseHasher.encode abstractmethod ¶

coola.hashing.BytesHasher ¶

coola.hashing.BytesHasher.encode ¶

coola.hashing.ContentHasher ¶

coola.hashing.ContentHasher.encode ¶

coola.hashing.DataclassHasher ¶

coola.hashing.DataclassHasher.encode ¶

coola.hashing.DatetimeHasher ¶

coola.hashing.DatetimeHasher.encode ¶

coola.hashing.EnumHasher ¶

coola.hashing.EnumHasher.encode ¶

coola.hashing.Hasher ¶

coola.hashing.Hasher.encode ¶

coola.hashing.HasherRegistry ¶

coola.hashing.HasherRegistry.find_hasher ¶

coola.hashing.HasherRegistry.has_hasher ¶

coola.hashing.HasherRegistry.hash ¶

coola.hashing.JaxArrayHasher ¶

coola.hashing.JaxArrayHasher.encode ¶

coola.hashing.MappingHasher ¶

coola.hashing.MappingHasher.encode ¶

coola.hashing.Node dataclass ¶

coola.hashing.NumpyArrayHasher ¶

coola.hashing.NumpyArrayHasher.encode ¶

coola.hashing.PandasDataFrameHasher ¶

coola.hashing.PandasDataFrameHasher.encode ¶

coola.hashing.PandasSeriesHasher ¶

coola.hashing.PandasSeriesHasher.encode ¶

coola.hashing.PathHasher ¶

coola.hashing.PathHasher.encode ¶

coola.hashing.PolarsDataFrameHasher ¶

coola.hashing.PolarsDataFrameHasher.encode ¶

coola.hashing.PolarsSeriesHasher ¶

coola.hashing.PolarsSeriesHasher.encode ¶

coola.hashing.PyarrowArrayHasher ¶

coola.hashing.PyarrowArrayHasher.encode ¶

coola.hashing.PyarrowTableHasher ¶

coola.hashing.PyarrowTableHasher.encode ¶

coola.hashing.PydanticModelHasher ¶

coola.hashing.PydanticModelHasher.encode ¶

coola.hashing.ReprHasher ¶

coola.hashing.ReprHasher.encode ¶

coola.hashing.ResolvedPathHasher ¶

coola.hashing.ResolvedPathHasher.encode ¶

coola.hashing.SecretHasher ¶

coola.hashing.SecretHasher.encode ¶

coola.hashing.SequenceHasher ¶

coola.hashing.SequenceHasher.encode ¶

coola.hashing.SetHasher ¶

coola.hashing.SetHasher.encode ¶

coola.hashing.StrHasher ¶

coola.hashing.StrHasher.encode ¶

coola.hashing.StringHasher ¶

coola.hashing.StringHasher.encode ¶

coola.hashing.SupportsCoolaHash ¶

coola.hashing.SupportsCoolaHash.__coola_hash__ ¶

coola.hashing.TorchTensorHasher ¶

coola.hashing.TorchTensorHasher.encode ¶

coola.hashing.XarrayHasher ¶

coola.hashing.XarrayHasher.encode ¶

coola.hashing.digest ¶

coola.hashing.get_default_registry ¶

coola.hashing.get_secret_hashers ¶

coola.hashing.get_stable_registry ¶

coola.hashing.hash_object ¶

coola.hashing.register_hashers ¶

coola.hashing.tagged ¶

coola.hashing.BaseHasher.encode `abstractmethod` ¶

coola.hashing.Node `dataclass` ¶