Evaluator

analora.evaluator

Contain evaluators.

analora.evaluator.AccuracyEvaluator

Bases: BaseStateEvaluator[AccuracyState]

Implement the accuracy evaluator.

Parameters:

Name	Type	Description	Default
state	AccuracyState	The state containing the ground truth and predicted labels.	required

Example usage:

>>> import numpy as np
>>> from analora.evaluator import AccuracyEvaluator
>>> from analora.state import AccuracyState
>>> evaluator = AccuracyEvaluator(
...     AccuracyState(
...         y_true=np.array([1, 0, 0, 1, 1]),
...         y_pred=np.array([1, 0, 0, 1, 1]),
...         y_true_name="target",
...         y_pred_name="pred",
...     )
... )
>>> evaluator
AccuracyEvaluator(
  (state): AccuracyState(y_true=(5,), y_pred=(5,), y_true_name='target', y_pred_name='pred', nan_policy='propagate')
)
>>> evaluator.evaluate()
{'accuracy': 1.0, 'count_correct': 5, 'count_incorrect': 0, 'count': 5, 'error': 0.0}

analora.evaluator.BalancedAccuracyEvaluator

Bases: BaseStateEvaluator[AccuracyState]

Implement the accuracy evaluator.

Parameters:

Name	Type	Description	Default
state	AccuracyState	The state containing the ground truth and predicted labels.	required

Example usage:

>>> import numpy as np
>>> from analora.evaluator import BalancedAccuracyEvaluator
>>> from analora.state import AccuracyState
>>> evaluator = BalancedAccuracyEvaluator(
...     AccuracyState(
...         y_true=np.array([1, 0, 0, 1, 1]),
...         y_pred=np.array([1, 0, 0, 1, 1]),
...         y_true_name="target",
...         y_pred_name="pred",
...     )
... )
>>> evaluator
BalancedAccuracyEvaluator(
  (state): AccuracyState(y_true=(5,), y_pred=(5,), y_true_name='target', y_pred_name='pred', nan_policy='propagate')
)
>>> evaluator.evaluate()
{'balanced_accuracy': 1.0, 'count': 5}

analora.evaluator.BaseEvaluator

Bases: ABC

Define the base class to implement an evaluator.

Example usage:

>>> from analora.evaluator import Evaluator
>>> evaluator = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator
Evaluator(count=2)
>>> evaluator.evaluate()
{'accuracy': 1.0, 'total': 42}

analora.evaluator.BaseEvaluator.compute abstractmethod

compute() -> BaseEvaluator

Compute the metrics and return a new evaluator.

Returns:

Type	Description
BaseEvaluator	A new evaluator with the computed metrics.

Example usage:

>>> from analora.evaluator import Evaluator
>>> evaluator = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator2 = evaluator.compute()
>>> evaluator2
Evaluator(count=2)

analora.evaluator.BaseEvaluator.equal abstractmethod

equal(other: Any, equal_nan: bool = False) -> bool

Indicate if two evaluators are equal or not.

Parameters:

Name	Type	Description	Default
other	Any	The other evaluator to compare.	required
equal_nan	bool	Whether to compare NaN's as equal. If True, NaN's in both objects will be considered equal.	False

Returns:

Type	Description
bool	True if the two evaluators are equal, otherwise False.

Example usage:

>>> from analora.evaluator import Evaluator
>>> evaluator1 = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator2 = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator3 = Evaluator({"accuracy": 0.5, "total": 42})
>>> evaluator1.equal(evaluator2)
True
>>> evaluator1.equal(evaluator3)
False

analora.evaluator.BaseEvaluator.evaluate abstractmethod

evaluate(prefix: str = '', suffix: str = '') -> dict

Evaluate the metrics.

Parameters:

Name	Type	Description	Default
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''

Returns:

Type	Description
dict	The metrics.

Example usage:

>>> from analora.evaluator import Evaluator
>>> evaluator = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator.evaluate()
{'accuracy': 1.0, 'total': 42}

analora.evaluator.BaseStateEvaluator

Bases: BaseEvaluator, Generic[T]

Define the base class to implement evaluators that compute the metrics from a state object.

Parameters:

Name	Type	Description	Default
state	T	The state with the data.	required

analora.evaluator.Evaluator

Bases: BaseEvaluator

Implement a simple evaluator.

Parameters:

Name	Type	Description	Default
metrics	dict | None	The dictionary of metrics.	None

Example usage:

>>> import numpy as np
>>> from analora.evaluator import Evaluator
>>> evaluator = Evaluator({"accuracy": 1.0, "total": 42})
>>> evaluator
Evaluator(count=2)
>>> evaluator.evaluate()
{'accuracy': 1.0, 'total': 42}