arkas.evaluator

arkas.evaluator

Contain evaluators.

arkas.evaluator.AccuracyEvaluator

Bases: BaseLazyEvaluator[AccuracyResult]

Implement the accuracy evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The column name of the ground truth target labels.	required
y_pred	str	The column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> evaluator = AccuracyEvaluator(y_true="target", y_pred="pred")
>>> evaluator
AccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
>>> result = evaluator.evaluate(data)
>>> result
AccuracyResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.AveragePrecisionEvaluator

Bases: BaseLazyEvaluator[AveragePrecisionResult]

Implement the average precision evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The key or column name of the predicted labels.	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', 'multilabel', and 'auto'. If 'auto', it tries to automatically find the label type from the arrays' shape.	'auto'
drop_nulls	bool	If True, the rows with null values in y_true or y_score columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import AveragePrecisionEvaluator
>>> evaluator = AveragePrecisionEvaluator(y_true="target", y_score="pred")
>>> evaluator
AveragePrecisionEvaluator(y_true='target', y_score='pred', label_type='auto', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [2, -1, 0, 3, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
AveragePrecisionResult(y_true=(5,), y_score=(5,), label_type='binary', nan_policy='propagate')

arkas.evaluator.BalancedAccuracyEvaluator

Bases: BaseLazyEvaluator[BalancedAccuracyResult]

Implement the accuracy evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BalancedAccuracyEvaluator
>>> evaluator = BalancedAccuracyEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BalancedAccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BalancedAccuracyResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.BaseEvaluator

Bases: ABC

Define the base class to evaluate a DataFrame.

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> evaluator = AccuracyEvaluator(y_true="target", y_pred="pred")
>>> evaluator
AccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
>>> result = evaluator.evaluate(data)
>>> result
AccuracyResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.BaseEvaluator.evaluate

evaluate(data: DataFrame, lazy: bool = True) -> BaseResult

Evaluate the result.

Parameters:

Name	Type	Description	Default
data	DataFrame	The data to evaluate.	required
lazy	bool	If True, it forces the computation of the result, otherwise it returns a result object that delays the evaluation of the result.	True

Returns:

Type	Description
BaseResult	The generated result.

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> evaluator = AccuracyEvaluator(y_true="target", y_pred="pred")
>>> data = pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
>>> result = evaluator.evaluate(data)
>>> result
AccuracyResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.BaseLazyEvaluator

Bases: BaseEvaluator, Generic[T]

Define the base class to evaluate the result lazily.

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> evaluator = AccuracyEvaluator(y_true="target", y_pred="pred")
>>> evaluator
AccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
>>> result = evaluator.evaluate(data)
>>> result
AccuracyResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.BinaryAveragePrecisionEvaluator

Bases: BaseLazyEvaluator[BinaryAveragePrecisionResult]

Implement the average precision evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryAveragePrecisionEvaluator
>>> evaluator = BinaryAveragePrecisionEvaluator(y_true="target", y_score="pred")
>>> evaluator
BinaryAveragePrecisionEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [2, -1, 0, 3, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryAveragePrecisionResult(y_true=(5,), y_score=(5,), nan_policy='propagate')

arkas.evaluator.BinaryClassificationEvaluator

Bases: BaseLazyEvaluator[BinaryClassificationResult]

Implement the average precision evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
y_score	str | None	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	None
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryClassificationEvaluator
>>> evaluator = BinaryClassificationEvaluator(
...     y_true="target", y_pred="pred", y_score="score"
... )
>>> evaluator
BinaryClassificationEvaluator(y_true='target', y_pred='pred', y_score='score', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [1, 0, 0, 1, 1],
...         "score": [2, -1, 0, 3, 1],
...         "target": [1, 0, 0, 1, 1],
...     }
... )
>>> result = evaluator.evaluate(data)
>>> result
BinaryClassificationResult(y_true=(5,), y_pred=(5,), y_score=(5,), betas=(1,), nan_policy='propagate')

arkas.evaluator.BinaryConfusionMatrixEvaluator

Bases: BaseLazyEvaluator[BinaryConfusionMatrixResult]

Implement the confusion matrix evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryConfusionMatrixEvaluator
>>> evaluator = BinaryConfusionMatrixEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BinaryConfusionMatrixEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryConfusionMatrixResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.BinaryFbetaScoreEvaluator

Bases: BaseLazyEvaluator[BinaryFbetaScoreResult]

Implement the F-beta evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryFbetaScoreEvaluator
>>> evaluator = BinaryFbetaScoreEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BinaryFbetaScoreEvaluator(y_true='target', y_pred='pred', betas=(1,), drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(
...     data=pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
... )
>>> result
BinaryFbetaScoreResult(y_true=(5,), y_pred=(5,), betas=(1,), nan_policy='propagate')

arkas.evaluator.BinaryJaccardEvaluator

Bases: BaseLazyEvaluator[BinaryJaccardResult]

Implement the Jaccard evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import numpy as np
>>> import polars as pl
>>> from arkas.evaluator import BinaryJaccardEvaluator
>>> evaluator = BinaryJaccardEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BinaryJaccardEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryJaccardResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.BinaryPrecisionEvaluator

Bases: BaseLazyEvaluator[BinaryPrecisionResult]

Implement the precision evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryPrecisionEvaluator
>>> evaluator = BinaryPrecisionEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BinaryPrecisionEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryPrecisionResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.BinaryRecallEvaluator

Bases: BaseLazyEvaluator[BinaryRecallResult]

Implement the recall evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryRecallEvaluator
>>> evaluator = BinaryRecallEvaluator(y_true="target", y_pred="pred")
>>> evaluator
BinaryRecallEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryRecallResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.BinaryRocAucEvaluator

Bases: BaseLazyEvaluator[BinaryRocAucResult]

Implement the Area Under the Receiver Operating Characteristic Curve (ROC AUC) evaluator for binary labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import BinaryRocAucEvaluator
>>> evaluator = BinaryRocAucEvaluator(y_true="target", y_score="pred")
>>> evaluator
BinaryRocAucEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [2, -1, 0, 3, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
BinaryRocAucResult(y_true=(5,), y_score=(5,), nan_policy='propagate')

arkas.evaluator.EnergyDistanceEvaluator

Bases: BaseLazyEvaluator[EnergyDistanceResult]

Implement the energy distance between two 1D distributions evaluator.

Parameters:

Name	Type	Description	Default
u_values	str	The values observed in the (empirical) distribution.	required
v_values	str	The values observed in the (empirical) distribution.	required
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import EnergyDistanceEvaluator
>>> evaluator = EnergyDistanceEvaluator(u_values="target", v_values="pred")
>>> evaluator
EnergyDistanceEvaluator(u_values='target', v_values='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
EnergyDistanceResult(u_values=(5,), v_values=(5,), nan_policy='propagate')

arkas.evaluator.EvaluatorDict

Bases: BaseEvaluator

Implement an evaluator that sequentially evaluates a mapping of evaluators.

Parameters:

Name	Type	Description	Default
evaluators	Mapping[Hashable, BaseEvaluator]	The mapping of evaluators to evaluate.	required

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import (
...     EvaluatorDict,
...     BinaryPrecisionEvaluator,
...     BinaryRecallEvaluator,
... )
>>> evaluator = EvaluatorDict(
...     {
...         "precision": BinaryPrecisionEvaluator(y_true="target", y_pred="pred"),
...         "recall": BinaryRecallEvaluator(y_true="target", y_pred="pred"),
...     }
... )
>>> evaluator
EvaluatorDict(
  (precision): BinaryPrecisionEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
  (recall): BinaryRecallEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
)
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
MappingResult(count=2)
>>> result = evaluator.evaluate(data, lazy=False)
>>> result
Result(metrics=2, figures=2)

arkas.evaluator.JensenShannonDivergenceEvaluator

Bases: BaseLazyEvaluator[JensenShannonDivergenceResult]

Implement the Jensen-Shannon (JS) divergence between two distributions evaluator.

Parameters:

Name	Type	Description	Default
p	str	The values observed in the (empirical) distribution.	required
q	str	The values observed in the (empirical) distribution.	required
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import JensenShannonDivergenceEvaluator
>>> evaluator = JensenShannonDivergenceEvaluator(p="target", q="pred")
>>> evaluator
JensenShannonDivergenceEvaluator(p='target', q='pred', drop_nulls=True)
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
JensenShannonDivergenceResult(p=(5,), q=(5,))

arkas.evaluator.KLDivEvaluator

Bases: BaseLazyEvaluator[KLDivResult]

Implement the Kullback-Leibler (KL) divergence between two distributions evaluator.

Parameters:

Name	Type	Description	Default
p	str	The values observed in the (empirical) distribution.	required
q	str	The values observed in the (empirical) distribution.	required
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import KLDivEvaluator
>>> evaluator = KLDivEvaluator(p="target", q="pred")
>>> evaluator
KLDivEvaluator(p='target', q='pred', drop_nulls=True)
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
KLDivResult(p=(5,), q=(5,))

arkas.evaluator.MeanAbsoluteErrorEvaluator

Bases: BaseLazyEvaluator[MeanAbsoluteErrorResult]

Implement the mean absolute error (MAE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MeanAbsoluteErrorEvaluator
>>> evaluator = MeanAbsoluteErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MeanAbsoluteErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MeanAbsoluteErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.MeanAbsolutePercentageErrorEvaluator

Bases: BaseLazyEvaluator[MeanAbsolutePercentageErrorResult]

Implement the mean absolute percentage error (MAPE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MeanAbsolutePercentageErrorEvaluator
>>> evaluator = MeanAbsolutePercentageErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MeanAbsolutePercentageErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MeanAbsolutePercentageErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.MeanSquaredErrorEvaluator

Bases: BaseLazyEvaluator[MeanSquaredErrorResult]

Implement the mean squared error (MSE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MeanSquaredErrorEvaluator
>>> evaluator = MeanSquaredErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MeanSquaredErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MeanSquaredErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.MeanSquaredLogErrorEvaluator

Bases: BaseLazyEvaluator[MeanSquaredLogErrorResult]

Implement the mean squared logarithmic error (MSLE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MeanSquaredLogErrorEvaluator
>>> evaluator = MeanSquaredLogErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MeanSquaredLogErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MeanSquaredLogErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.MeanTweedieDevianceEvaluator

Bases: BaseLazyEvaluator[MeanTweedieDevianceResult]

Implement the mean Tweedie deviance regression loss evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MeanTweedieDevianceEvaluator
>>> evaluator = MeanTweedieDevianceEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MeanTweedieDevianceEvaluator(y_true='target', y_pred='pred', powers=(0,), drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MeanTweedieDevianceResult(y_true=(5,), y_pred=(5,), powers=(0,), nan_policy='propagate')

arkas.evaluator.MedianAbsoluteErrorEvaluator

Bases: BaseLazyEvaluator[MedianAbsoluteErrorResult]

Implement the median absolute error (MAE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MedianAbsoluteErrorEvaluator
>>> evaluator = MedianAbsoluteErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MedianAbsoluteErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
MedianAbsoluteErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.MulticlassAveragePrecisionEvaluator

Bases: BaseLazyEvaluator[MulticlassAveragePrecisionResult]

Implement the average precision evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_score columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassAveragePrecisionEvaluator
>>> evaluator = MulticlassAveragePrecisionEvaluator(y_true="target", y_score="pred")
>>> evaluator
MulticlassAveragePrecisionEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [
...             [0.7, 0.2, 0.1],
...             [0.4, 0.3, 0.3],
...             [0.1, 0.8, 0.1],
...             [0.2, 0.5, 0.3],
...             [0.3, 0.3, 0.4],
...             [0.1, 0.2, 0.7],
...         ],
...         "target": [0, 0, 1, 1, 2, 2],
...     },
...     schema={"pred": pl.Array(pl.Float64, 3), "target": pl.Int64},
... )
>>> result = evaluator.evaluate(data)
>>> result
MulticlassAveragePrecisionResult(y_true=(6,), y_score=(6, 3), nan_policy='propagate')

arkas.evaluator.MulticlassConfusionMatrixEvaluator

Bases: BaseLazyEvaluator[MulticlassConfusionMatrixResult]

Implement the confusion matrix evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassConfusionMatrixEvaluator
>>> evaluator = MulticlassConfusionMatrixEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MulticlassConfusionMatrixEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [0, 0, 1, 1, 2, 2], "target": [0, 0, 1, 1, 2, 2]})
>>> result = evaluator.evaluate(data)
>>> result
MulticlassConfusionMatrixResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.MulticlassFbetaScoreEvaluator

Bases: BaseLazyEvaluator[MulticlassFbetaScoreResult]

Implement the F-beta evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassFbetaScoreEvaluator
>>> evaluator = MulticlassFbetaScoreEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MulticlassFbetaScoreEvaluator(y_true='target', y_pred='pred', betas=(1,), drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [0, 0, 1, 1, 2, 2], "target": [0, 0, 1, 1, 2, 2]})
>>> result = evaluator.evaluate(data)
>>> result
MulticlassFbetaScoreResult(y_true=(6,), y_pred=(6,), betas=(1,), nan_policy='propagate')

arkas.evaluator.MulticlassJaccardEvaluator

Bases: BaseLazyEvaluator[MulticlassJaccardResult]

Implement the Jaccard evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassJaccardEvaluator
>>> evaluator = MulticlassJaccardEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MulticlassJaccardEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [0, 0, 1, 1, 2, 2], "target": [0, 0, 1, 1, 2, 2]})
>>> result = evaluator.evaluate(data)
>>> result
MulticlassJaccardResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.MulticlassPrecisionEvaluator

Bases: BaseLazyEvaluator[MulticlassPrecisionResult]

Implement the precision evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassPrecisionEvaluator
>>> evaluator = MulticlassPrecisionEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MulticlassPrecisionEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [0, 0, 1, 1, 2, 2], "target": [0, 0, 1, 1, 2, 2]})
>>> result = evaluator.evaluate(data)
>>> result
MulticlassPrecisionResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.MulticlassRecallEvaluator

Bases: BaseLazyEvaluator[MulticlassRecallResult]

Implement the recall evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassRecallEvaluator
>>> evaluator = MulticlassRecallEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MulticlassRecallEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [0, 0, 1, 1, 2, 2], "target": [0, 0, 1, 1, 2, 2]})
>>> result = evaluator.evaluate(data)
>>> result
MulticlassRecallResult(y_true=(6,), y_pred=(6,), nan_policy='propagate')

arkas.evaluator.MulticlassRocAucEvaluator

Bases: BaseLazyEvaluator[MulticlassRocAucResult]

Implement the Area Under the Receiver Operating Characteristic Curve (ROC AUC) evaluator for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_score columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MulticlassRocAucEvaluator
>>> evaluator = MulticlassRocAucEvaluator(y_true="target", y_score="pred")
>>> evaluator
MulticlassRocAucEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [
...             [0.7, 0.2, 0.1],
...             [0.4, 0.3, 0.3],
...             [0.1, 0.8, 0.1],
...             [0.2, 0.5, 0.3],
...             [0.3, 0.3, 0.4],
...             [0.1, 0.2, 0.7],
...         ],
...         "target": [0, 0, 1, 1, 2, 2],
...     },
...     schema={"pred": pl.Array(pl.Float64, 3), "target": pl.Int64},
... )
>>> result = evaluator.evaluate(data)
>>> result
MulticlassRocAucResult(y_true=(6,), y_score=(6, 3), nan_policy='propagate')

arkas.evaluator.MultilabelAveragePrecisionEvaluator

Bases: BaseLazyEvaluator[MultilabelAveragePrecisionResult]

Implement the average precision evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_score columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelAveragePrecisionEvaluator
>>> evaluator = MultilabelAveragePrecisionEvaluator(y_true="target", y_score="pred")
>>> evaluator
MultilabelAveragePrecisionEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[2, -1, 1], [-1, 1, -2], [0, 2, -3], [3, -2, 4], [1, -3, 5]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelAveragePrecisionResult(y_true=(5, 3), y_score=(5, 3), nan_policy='propagate')

arkas.evaluator.MultilabelConfusionMatrixEvaluator

Bases: BaseLazyEvaluator[MultilabelConfusionMatrixResult]

Implement the confusion matrix evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelConfusionMatrixEvaluator
>>> evaluator = MultilabelConfusionMatrixEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MultilabelConfusionMatrixEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[1, 0, 0], [0, 1, 1], [0, 1, 1], [1, 0, 0], [1, 0, 0]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelConfusionMatrixResult(y_true=(5, 3), y_pred=(5, 3), nan_policy='propagate')

arkas.evaluator.MultilabelFbetaScoreEvaluator

Bases: BaseLazyEvaluator[MultilabelFbetaScoreResult]

Implement the F-beta evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelFbetaScoreEvaluator
>>> evaluator = MultilabelFbetaScoreEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MultilabelFbetaScoreEvaluator(y_true='target', y_pred='pred', betas=(1,), drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[1, 0, 0], [0, 1, 1], [0, 1, 1], [1, 0, 0], [1, 0, 0]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelFbetaScoreResult(y_true=(5, 3), y_pred=(5, 3), betas=(1,), nan_policy='propagate')

arkas.evaluator.MultilabelJaccardEvaluator

Bases: BaseLazyEvaluator[MultilabelJaccardResult]

Implement the Jaccard evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelJaccardEvaluator
>>> evaluator = MultilabelJaccardEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MultilabelJaccardEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[1, 0, 0], [0, 1, 1], [0, 1, 1], [1, 0, 0], [1, 0, 0]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelJaccardResult(y_true=(5, 3), y_pred=(5, 3), nan_policy='propagate')

arkas.evaluator.MultilabelPrecisionEvaluator

Bases: BaseLazyEvaluator[MultilabelPrecisionResult]

Implement the precision evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelPrecisionEvaluator
>>> evaluator = MultilabelPrecisionEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MultilabelPrecisionEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[1, 0, 0], [0, 1, 1], [0, 1, 1], [1, 0, 0], [1, 0, 0]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelPrecisionResult(y_true=(5, 3), y_pred=(5, 3), nan_policy='propagate')

arkas.evaluator.MultilabelRecallEvaluator

Bases: BaseLazyEvaluator[MultilabelRecallResult]

Implement the recall evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_pred	str	The key or column name of the predicted labels.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelRecallEvaluator
>>> evaluator = MultilabelRecallEvaluator(y_true="target", y_pred="pred")
>>> evaluator
MultilabelRecallEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[1, 0, 0], [0, 1, 1], [0, 1, 1], [1, 0, 0], [1, 0, 0]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelRecallResult(y_true=(5, 3), y_pred=(5, 3), nan_policy='propagate')

arkas.evaluator.MultilabelRocAucEvaluator

Bases: BaseLazyEvaluator[MultilabelRocAucResult]

Implement the Area Under the Receiver Operating Characteristic Curve (ROC AUC) evaluator for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target labels.	required
y_score	str	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_score columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import MultilabelRocAucEvaluator
>>> evaluator = MultilabelRocAucEvaluator(y_true="target", y_score="pred")
>>> evaluator
MultilabelRocAucEvaluator(y_true='target', y_score='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame(
...     {
...         "pred": [[2, -1, 1], [-1, 1, -2], [0, 2, -3], [3, -2, 4], [1, -3, 5]],
...         "target": [[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]],
...     },
...     schema={"pred": pl.Array(pl.Int64, 3), "target": pl.Array(pl.Int64, 3)},
... )
>>> result = evaluator.evaluate(data)
>>> result
MultilabelRocAucResult(y_true=(5, 3), y_score=(5, 3), nan_policy='propagate')

arkas.evaluator.PearsonCorrelationEvaluator

Bases: BaseLazyEvaluator[PearsonCorrelationResult]

Implement the Pearson correlation evaluator.

Parameters:

Name	Type	Description	Default
x	str	The key or column name of the ground truth target values.	required
y	str	The key or column name of the predicted values.	required
alternative	str	The alternative hypothesis. Default is 'two-sided'. The following options are available: - 'two-sided': the correlation is nonzero - 'less': the correlation is negative (less than zero) - 'greater': the correlation is positive (greater than zero)	'two-sided'
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import PearsonCorrelationEvaluator
>>> evaluator = PearsonCorrelationEvaluator(x="target", y="pred")
>>> evaluator
PearsonCorrelationEvaluator(x='target', y='pred', alternative='two-sided', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
PearsonCorrelationResult(x=(5,), y=(5,), alternative='two-sided', nan_policy='propagate')

arkas.evaluator.R2ScoreEvaluator

Bases: BaseLazyEvaluator[R2ScoreResult]

Implement the R^2 (coefficient of determination) regression score evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import R2ScoreEvaluator
>>> evaluator = R2ScoreEvaluator(y_true="target", y_pred="pred")
>>> evaluator
R2ScoreEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
R2ScoreResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.RootMeanSquaredErrorEvaluator

Bases: BaseLazyEvaluator[RootMeanSquaredErrorResult]

Implement the root mean squared error (RMSE) evaluator.

Parameters:

Name	Type	Description	Default
y_true	str	The key or column name of the ground truth target values.	required
y_pred	str	The key or column name of the predicted values.	required
drop_nulls	bool	If True, the rows with null values in y_true or y_pred columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import RootMeanSquaredErrorEvaluator
>>> evaluator = RootMeanSquaredErrorEvaluator(y_true="target", y_pred="pred")
>>> evaluator
RootMeanSquaredErrorEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
RootMeanSquaredErrorResult(y_true=(5,), y_pred=(5,), nan_policy='propagate')

arkas.evaluator.SequentialEvaluator

Bases: BaseEvaluator

Implement an evaluator that sequentially evaluates several evaluators.

Parameters:

Name	Type	Description	Default
evaluators	Sequence[BaseEvaluator | dict]	The sequence of evaluators to evaluate.	required

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import (
...     SequentialEvaluator,
...     BinaryPrecisionEvaluator,
...     BinaryRecallEvaluator,
... )
>>> evaluator = SequentialEvaluator(
...     [
...         BinaryPrecisionEvaluator(y_true="target", y_pred="pred"),
...         BinaryRecallEvaluator(y_true="target", y_pred="pred"),
...     ]
... )
>>> evaluator
SequentialEvaluator(
  (0): BinaryPrecisionEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
  (1): BinaryRecallEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
)
>>> data = pl.DataFrame({"pred": [1, 0, 0, 1, 1], "target": [1, 0, 0, 1, 1]})
>>> result = evaluator.evaluate(data)
>>> result
SequentialResult(count=2)
>>> result = evaluator.evaluate(data, lazy=False)
>>> result
Result(metrics=3, figures=1)

arkas.evaluator.SpearmanCorrelationEvaluator

Bases: BaseLazyEvaluator[SpearmanCorrelationResult]

Implement the Spearman correlation evaluator.

Parameters:

Name	Type	Description	Default
x	str	The key or column name of the ground truth target values.	required
y	str	The key or column name of the predicted values.	required
alternative	str	The alternative hypothesis. Default is 'two-sided'. The following options are available: - 'two-sided': the correlation is nonzero - 'less': the correlation is negative (less than zero) - 'greater': the correlation is positive (greater than zero)	'two-sided'
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import SpearmanCorrelationEvaluator
>>> evaluator = SpearmanCorrelationEvaluator(x="target", y="pred")
>>> evaluator
SpearmanCorrelationEvaluator(x='target', y='pred', alternative='two-sided', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
SpearmanCorrelationResult(x=(5,), y=(5,), alternative='two-sided', nan_policy='propagate')

arkas.evaluator.WassersteinDistanceEvaluator

Bases: BaseLazyEvaluator[WassersteinDistanceResult]

Implement the Wasserstein distance between two 1D distributions evaluator.

Parameters:

Name	Type	Description	Default
u_values	str	The values observed in the (empirical) distribution.	required
v_values	str	The values observed in the (empirical) distribution.	required
drop_nulls	bool	If True, the rows with null values in x or y columns are dropped.	True
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Example usage:

>>> import polars as pl
>>> from arkas.evaluator import WassersteinDistanceEvaluator
>>> evaluator = WassersteinDistanceEvaluator(u_values="target", v_values="pred")
>>> evaluator
WassersteinDistanceEvaluator(u_values='target', v_values='pred', drop_nulls=True, nan_policy='propagate')
>>> data = pl.DataFrame({"pred": [1, 2, 3, 4, 5], "target": [1, 2, 3, 4, 5]})
>>> result = evaluator.evaluate(data)
>>> result
WassersteinDistanceResult(u_values=(5,), v_values=(5,), nan_policy='propagate')

arkas.evaluator.is_evaluator_config

is_evaluator_config(config: dict) -> bool

Indicate if the input configuration is a configuration for a BaseEvaluator.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict	The configuration to check.	required

Returns:

Type	Description
bool	True if the input configuration is a configuration for a BaseEvaluator object.

Example usage:

>>> from arkas.evaluator import is_evaluator_config
>>> is_evaluator_config({"_target_": "arkas.evaluator.AccuracyEvaluator"})
True

arkas.evaluator.setup_evaluator

setup_evaluator(
    evaluator: BaseEvaluator | dict,
) -> BaseEvaluator

Set up an evaluator.

The evaluator is instantiated from its configuration by using the BaseEvaluator factory function.

Parameters:

Name	Type	Description	Default
evaluator	BaseEvaluator | dict	An evaluator or its configuration.	required

Returns:

Type	Description
BaseEvaluator	An instantiated evaluator.

Example usage:

>>> from arkas.evaluator import setup_evaluator
>>> evaluator = setup_evaluator(
...     {
...         "_target_": "arkas.evaluator.AccuracyEvaluator",
...         "y_true": "target",
...         "y_pred": "pred",
...     }
... )
>>> evaluator
AccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')