Metric

analora.metric

Contain functions to compute metrics.

analora.metric.accuracy

accuracy(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the accuracy metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import accuracy
>>> accuracy(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'accuracy': 1.0, 'count_correct': 5, 'count_incorrect': 0, 'count': 5, 'error': 0.0}

analora.metric.average_precision

average_precision(
    y_true: ndarray,
    y_score: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the average precision metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import average_precision
>>> # auto
>>> metrics = average_precision(
...     y_true=np.array([1, 0, 0, 1, 1]), y_score=np.array([2, -1, 0, 3, 1])
... )
>>> metrics
{'average_precision': 1.0, 'count': 5}
>>> # binary
>>> metrics = average_precision(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_score=np.array([2, -1, 0, 3, 1]),
...     label_type="binary",
... )
>>> metrics
{'average_precision': 1.0, 'count': 5}
>>> # multiclass
>>> metrics = average_precision(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_score=np.array(
...         [
...             [0.7, 0.2, 0.1],
...             [0.4, 0.3, 0.3],
...             [0.1, 0.8, 0.1],
...             [0.2, 0.3, 0.5],
...             [0.4, 0.4, 0.2],
...             [0.1, 0.2, 0.7],
...         ]
...     ),
...     label_type="multiclass",
... )
>>> metrics
{'average_precision': array([0.833..., 0.75 , 0.75 ]),
 'count': 6,
 'macro_average_precision': 0.777...,
 'micro_average_precision': 0.75,
 'weighted_average_precision': 0.777...}
>>> # multilabel
>>> metrics = average_precision(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_score=np.array([[2, -1, -1], [-1, 1, 2], [0, 2, 3], [3, -2, -4], [1, -3, -5]]),
...     label_type="multilabel",
... )
>>> metrics
{'average_precision': array([1. , 1. , 0.477...]),
 'count': 5,
 'macro_average_precision': 0.825...,
 'micro_average_precision': 0.588...,
 'weighted_average_precision': 0.804...}

analora.metric.balanced_accuracy

balanced_accuracy(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the accuracy metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import balanced_accuracy
>>> balanced_accuracy(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'balanced_accuracy': 1.0, 'count': 5}

analora.metric.binary_average_precision

binary_average_precision(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the average precision metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, *).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, *).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_average_precision
>>> metrics = binary_average_precision(
...     y_true=np.array([1, 0, 0, 1, 1]), y_score=np.array([2, -1, 0, 3, 1])
... )
>>> metrics
{'average_precision': 1.0, 'count': 5}

analora.metric.binary_confusion_matrix

binary_confusion_matrix(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the confusion matrix metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_confusion_matrix
>>> binary_confusion_matrix(
...     y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1])
... )
{'confusion_matrix': array([[2, 0], [0, 3]]),
 'count': 5,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 3}

analora.metric.binary_fbeta_score

binary_fbeta_score(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    betas: Sequence[float] = (1,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the F-beta metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_fbeta_score
>>> binary_fbeta_score(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
... )
{'count': 5, 'f1': 1.0}

analora.metric.binary_jaccard

binary_jaccard(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Jaccard metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_jaccard
>>> binary_jaccard(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'jaccard': 1.0}

analora.metric.binary_precision

binary_precision(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the precision metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_precision
>>> binary_precision(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'precision': 1.0}

analora.metric.binary_recall

binary_recall(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the recall metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import binary_recall
>>> binary_recall(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'recall': 1.0}

analora.metric.binary_roc_auc

binary_roc_auc(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Area Under the Receiver Operating Characteristic Curve (ROC AUC) metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

analora.metric.binary_top_k_accuracy

binary_top_k_accuracy(
    y_true: ndarray,
    y_score: ndarray,
    *,
    k: Sequence[int] = (2,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Area Under the Top-k Accuracy classification metrics for binary labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples,).	required
k	Sequence[int]	The numbers of most likely outcomes considered to find the correct label.	(2,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

analora.metric.confusion_matrix

confusion_matrix(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the confusion matrix metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import confusion_matrix
>>> # binary
>>> confusion_matrix(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
...     label_type="binary",
... )
{'confusion_matrix': array([[2, 0], [0, 3]]),
 'count': 5,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 3}
>>> # multiclass
>>> confusion_matrix(
...     y_true=np.array([0, 1, 1, 2, 2, 2]),
...     y_pred=np.array([0, 1, 1, 2, 2, 2]),
...     label_type="multiclass",
... )
{'confusion_matrix': array([[1, 0, 0], [0, 2, 0], [0, 0, 3]]), 'count': 6}
>>> # multilabel
>>> confusion_matrix(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     label_type="multilabel",
... )
{'confusion_matrix': array([[[2, 0], [0, 3]],
                            [[3, 0], [0, 2]],
                            [[2, 0], [0, 3]]]),
 'count': 5}

analora.metric.energy_distance

energy_distance(
    u_values: ndarray,
    v_values: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the energy distance between two 1D distributions.

Parameters:

Name	Type	Description	Default
u_values	ndarray	The values observed in the (empirical) distribution.	required
v_values	ndarray	The values observed in the (empirical) distribution.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import energy_distance
>>> energy_distance(u_values=np.array([1, 2, 3, 4, 5]), v_values=np.array([1, 2, 3, 4, 5]))
{'count': 5, 'energy_distance': 0.0}

analora.metric.fbeta_score

fbeta_score(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    betas: Sequence[float] = (1,),
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the F-beta metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import fbeta_score
>>> # auto
>>> fbeta_score(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'f1': 1.0}
>>> # binary
>>> fbeta_score(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
...     label_type="binary",
... )
{'count': 5, 'f1': 1.0}
>>> # multiclass
>>> fbeta_score(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_pred=np.array([0, 0, 1, 1, 2, 2]),
...     label_type="multiclass",
... )
{'count': 6,
 'f1': array([1., 1., 1.]),
 'macro_f1': 1.0,
 'micro_f1': 1.0,
 'weighted_f1': 1.0}
>>> # multilabel
>>> fbeta_score(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     label_type="multilabel",
... )
{'count': 5,
 'f1': array([1., 1., 1.]),
 'macro_f1': 1.0,
 'micro_f1': 1.0,
 'weighted_f1': 1.0}

analora.metric.jaccard

jaccard(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Jaccard metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import jaccard
>>> # auto
>>> jaccard(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'jaccard': 1.0}
>>> # binary
>>> jaccard(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
...     label_type="binary",
... )
{'count': 5, 'jaccard': 1.0}
>>> # multiclass
>>> jaccard(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_pred=np.array([0, 0, 1, 1, 2, 2]),
...     label_type="multiclass",
... )
{'count': 6,
 'jaccard': array([1., 1., 1.]),
 'macro_jaccard': 1.0,
 'micro_jaccard': 1.0,
 'weighted_jaccard': 1.0}
>>> # multilabel
>>> jaccard(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     label_type="multilabel",
... )
{'count': 5,
 'jaccard': array([1., 1., 1.]),
 'macro_jaccard': 1.0,
 'micro_jaccard': 1.0,
 'weighted_jaccard': 1.0}

analora.metric.jensen_shannon_divergence

jensen_shannon_divergence(
    p: ndarray,
    q: ndarray,
    *,
    prefix: str = "",
    suffix: str = ""
) -> dict[str, float]

Return the Jensen-Shannon (JS) divergence between two distributions.

Parameters:

Name	Type	Description	Default
p	ndarray	The true probability distribution.	required
q	ndarray	The model probability distribution.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import jensen_shannon_divergence
>>> jensen_shannon_divergence(
...     p=np.array([0.1, 0.6, 0.1, 0.2]), q=np.array([0.2, 0.5, 0.2, 0.1])
... )
{'size': 4, 'jensen_shannon_divergence': 0.027...}

analora.metric.kl_div

kl_div(
    p: ndarray,
    q: ndarray,
    *,
    prefix: str = "",
    suffix: str = ""
) -> dict[str, float]

Return the Kullback-Leibler (KL) divergence between two distributions.

Parameters:

Name	Type	Description	Default
p	ndarray	The true probability distribution.	required
q	ndarray	The model probability distribution.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import kl_div
>>> kl_div(p=np.array([0.1, 0.6, 0.1, 0.2]), q=np.array([0.2, 0.5, 0.2, 0.1]))
{'size': 4, 'kl_pq': 0.109..., 'kl_qp': 0.116...}

analora.metric.mean_absolute_error

mean_absolute_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the mean absolute error (MAE).

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import mean_absolute_error
>>> mean_absolute_error(y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5]))
{'count': 5, 'mean_absolute_error': 0.0}

analora.metric.mean_absolute_percentage_error

mean_absolute_percentage_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the mean absolute percentage error (MAPE).

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import mean_absolute_percentage_error
>>> mean_absolute_percentage_error(
...     y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'mean_absolute_percentage_error': 0.0}

analora.metric.mean_squared_error

mean_squared_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the mean squared error (MSE).

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import mean_squared_error
>>> mean_squared_error(y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5]))
{'count': 5, 'mean_squared_error': 0.0}

analora.metric.mean_squared_log_error

mean_squared_log_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the mean squared logarithmic error (MSLE).

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import mean_squared_log_error
>>> mean_squared_log_error(
...     y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'mean_squared_log_error': 0.0}

analora.metric.mean_tweedie_deviance

mean_tweedie_deviance(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    powers: Sequence[float] = (0,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the mean Tweedie deviance regression loss.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
powers	Sequence[float]	The Tweedie power parameter. The higher power the less weight is given to extreme deviations between true and predicted targets.	(0,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import mean_tweedie_deviance
>>> mean_tweedie_deviance(
...     y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'mean_tweedie_deviance_power_0': 0.0}

analora.metric.median_absolute_error

median_absolute_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the median absolute error.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import median_absolute_error
>>> median_absolute_error(
...     y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'median_absolute_error': 0.0}

analora.metric.multiclass_average_precision

multiclass_average_precision(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the average precision metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_average_precision
>>> metrics = multiclass_average_precision(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_score=np.array(
...         [
...             [0.7, 0.2, 0.1],
...             [0.4, 0.3, 0.3],
...             [0.1, 0.8, 0.1],
...             [0.2, 0.3, 0.5],
...             [0.4, 0.4, 0.2],
...             [0.1, 0.2, 0.7],
...         ]
...     ),
... )
>>> metrics
{'average_precision': array([0.833..., 0.75 , 0.75 ]),
 'count': 6,
 'macro_average_precision': 0.777...,
 'micro_average_precision': 0.75,
 'weighted_average_precision': 0.777...}

analora.metric.multiclass_confusion_matrix

multiclass_confusion_matrix(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the confusion matrix metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_confusion_matrix
>>> multiclass_confusion_matrix(
...     y_true=np.array([0, 1, 1, 2, 2, 2]), y_pred=np.array([0, 1, 1, 2, 2, 2])
... )
{'confusion_matrix': array([[1, 0, 0], [0, 2, 0], [0, 0, 3]]), 'count': 6}

analora.metric.multiclass_fbeta_score

multiclass_fbeta_score(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    betas: Sequence[float] = (1,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the F-beta metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_fbeta_score
>>> multiclass_fbeta_score(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_pred=np.array([0, 0, 1, 1, 2, 2]),
... )
{'count': 6,
 'f1': array([1., 1., 1.]),
 'macro_f1': 1.0,
 'micro_f1': 1.0,
 'weighted_f1': 1.0}

analora.metric.multiclass_jaccard

multiclass_jaccard(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Jaccard metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_jaccard
>>> multiclass_jaccard(
...     y_true=np.array([0, 0, 1, 1, 2, 2]), y_pred=np.array([0, 0, 1, 1, 2, 2])
... )
{'count': 6,
 'jaccard': array([1., 1., 1.]),
 'macro_jaccard': 1.0,
 'micro_jaccard': 1.0,
 'weighted_jaccard': 1.0}

analora.metric.multiclass_precision

multiclass_precision(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the precision metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_precision
>>> multiclass_precision(
...     y_true=np.array([0, 0, 1, 1, 2, 2]), y_pred=np.array([0, 0, 1, 1, 2, 2])
... )
{'count': 6,
 'macro_precision': 1.0,
 'micro_precision': 1.0,
 'precision': array([1., 1., 1.]),
 'weighted_precision': 1.0}

analora.metric.multiclass_recall

multiclass_recall(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the recall metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multiclass_recall
>>> multiclass_recall(
...     y_true=np.array([0, 0, 1, 1, 2, 2]), y_pred=np.array([0, 0, 1, 1, 2, 2])
... )
{'count': 6,
 'macro_recall': 1.0,
 'micro_recall': 1.0,
 'recall': array([1., 1., 1.]),
 'weighted_recall': 1.0}

analora.metric.multiclass_roc_auc

multiclass_roc_auc(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Area Under the Receiver Operating Characteristic Curve (ROC AUC) metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

analora.metric.multiclass_top_k_accuracy

multiclass_top_k_accuracy(
    y_true: ndarray,
    y_score: ndarray,
    *,
    k: Sequence[int] = (2,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Area Under the Top-k Accuracy classification metrics for multiclass labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_classes).	required
k	Sequence[int]	The numbers of most likely outcomes considered to find the correct label.	(2,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

analora.metric.multilabel_average_precision

multilabel_average_precision(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the average precision metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, n_classes).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_average_precision
>>> metrics = multilabel_average_precision(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_score=np.array([[2, -1, -1], [-1, 1, 2], [0, 2, 3], [3, -2, -4], [1, -3, -5]]),
... )
>>> metrics
{'average_precision': array([1. , 1. , 0.477...]),
 'count': 5,
 'macro_average_precision': 0.825...,
 'micro_average_precision': 0.588...,
 'weighted_average_precision': 0.804...}

analora.metric.multilabel_confusion_matrix

multilabel_confusion_matrix(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the confusion matrix metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels.	required
y_pred	ndarray	The predicted labels.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_confusion_matrix
>>> multilabel_confusion_matrix(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
... )
{'confusion_matrix': array([[[2, 0], [0, 3]],
                            [[3, 0], [0, 2]],
                            [[2, 0], [0, 3]]]),
 'count': 5}

analora.metric.multilabel_fbeta_score

multilabel_fbeta_score(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    betas: Sequence[float] = (1,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the F-beta metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,).	required
betas	Sequence[float]	The betas used to compute the F-beta scores.	(1,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_fbeta_score
>>> multilabel_fbeta_score(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
... )
{'count': 5,
 'f1': array([1., 1., 1.]),
 'macro_f1': 1.0,
 'micro_f1': 1.0,
 'weighted_f1': 1.0}

analora.metric.multilabel_jaccard

multilabel_jaccard(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Jaccard metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_jaccard
>>> multilabel_jaccard(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
... )
{'count': 5,
 'jaccard': array([1., 1., 1.]),
 'macro_jaccard': 1.0,
 'micro_jaccard': 1.0,
 'weighted_jaccard': 1.0}

analora.metric.multilabel_precision

multilabel_precision(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the precision metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_precision
>>> multilabel_precision(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
... )
{'count': 5,
 'macro_precision': 1.0,
 'micro_precision': 1.0,
 'precision': array([1., 1., 1.]),
 'weighted_precision': 1.0}

analora.metric.multilabel_recall

multilabel_recall(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the recall metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import multilabel_recall
>>> multilabel_recall(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
... )
{'count': 5,
 'macro_recall': 1.0,
 'micro_recall': 1.0,
 'recall': array([1., 1., 1.]),
 'weighted_recall': 1.0}

analora.metric.multilabel_roc_auc

multilabel_roc_auc(
    y_true: ndarray,
    y_score: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Area Under the Receiver Operating Characteristic Curve (ROC AUC) metrics for multilabel labels.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples, n_classes).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_classes).	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

analora.metric.ndcg

ndcg(
    y_true: ndarray,
    y_score: ndarray,
    *,
    k: int | None = None,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Normalized Discounted Cumulative Gain (NDCG) metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target targets of multilabel classification, or true scores of entities to be ranked. Negative values in y_true may result in an output that is not between 0 and 1. This input must be an array of shape (n_samples, n_labels).	required
y_score	ndarray	The predicted scores, can either be probability estimates, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples, n_labels).	required
k	int | None	Only consider the highest k scores in the ranking. If None, use all outputs.	None
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import ndcg
>>> ndcg(
...     y_true=np.array([[1, 0, 0], [1, 2, 0], [1, 1, 2], [0, 0, 1]]),
...     y_score=np.array(
...         [[2.0, 1.0, 0.0], [0.0, 1.0, -1.0], [0.0, 0.0, 1.0], [1.0, 2.0, 3.0]]
...     ),
... )
{'count': 4, 'ndcg': 1.0}

analora.metric.pearsonr

pearsonr(
    x: ndarray,
    y: ndarray,
    *,
    alternative: str = "two-sided",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Pearson correlation coefficient and p-value for testing non-correlation.

Parameters:

Name	Type	Description	Default
x	ndarray	The first input array.	required
y	ndarray	The second input array.	required
alternative	str	The alternative hypothesis. Default is 'two-sided'. The following options are available: - 'two-sided': the correlation is nonzero - 'less': the correlation is negative (less than zero) - 'greater': the correlation is positive (greater than zero)	'two-sided'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import pearsonr
>>> pearsonr(x=np.array([1, 2, 3, 4, 5]), y=np.array([1, 2, 3, 4, 5]))
{'count': 5, 'pearson_coeff': 1.0, 'pearson_pvalue': 0.0}

analora.metric.precision

precision(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the precision metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import precision
>>> # auto
>>> precision(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'precision': 1.0}
>>> # binary
>>> precision(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
...     label_type="binary",
... )
{'count': 5, 'precision': 1.0}
>>> # multiclass
>>> precision(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_pred=np.array([0, 0, 1, 1, 2, 2]),
...     label_type="multiclass",
... )
{'count': 6,
 'macro_precision': 1.0,
 'micro_precision': 1.0,
 'precision': array([1., 1., 1.]),
 'weighted_precision': 1.0}
>>> # multilabel
>>> precision(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     label_type="multilabel",
... )
{'count': 5,
 'macro_precision': 1.0,
 'micro_precision': 1.0,
 'precision': array([1., 1., 1.]),
 'weighted_precision': 1.0}

analora.metric.r2_score

r2_score(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the R^2 (coefficient of determination) regression score metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import r2_score
>>> r2_score(y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5]))
{'count': 5, 'r2_score': 1.0}

analora.metric.recall

recall(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the recall metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_pred	ndarray	The predicted labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import recall
>>> # auto
>>> recall(y_true=np.array([1, 0, 0, 1, 1]), y_pred=np.array([1, 0, 0, 1, 1]))
{'count': 5, 'recall': 1.0}
>>> # binary
>>> recall(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_pred=np.array([1, 0, 0, 1, 1]),
...     label_type="binary",
... )
{'count': 5, 'recall': 1.0}
>>> # multiclass
>>> recall(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_pred=np.array([0, 0, 1, 1, 2, 2]),
...     label_type="multiclass",
... )
{'count': 6,
 'macro_recall': 1.0,
 'micro_recall': 1.0,
 'recall': array([1., 1., 1.]),
 'weighted_recall': 1.0}
>>> # multilabel
>>> recall(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_pred=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     label_type="multilabel",
... )
{'count': 5,
 'macro_recall': 1.0,
 'micro_recall': 1.0,
 'recall': array([1., 1., 1.]),
 'weighted_recall': 1.0}

analora.metric.regression_errors

regression_errors(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the regression error metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import regression_errors
>>> regression_errors(y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5]))
{'count': 5,
 'mean_absolute_error': 0.0,
 'median_absolute_error': 0.0,
 'mean_squared_error': 0.0}

analora.metric.roc_auc

roc_auc(
    y_true: ndarray,
    y_score: ndarray,
    *,
    label_type: str = "auto",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Area Under the Receiver Operating Characteristic Curve (ROC AUC) metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. This input must be an array of shape (n_samples,) or (n_samples, n_classes).	required
label_type	str	The type of labels used to evaluate the metrics. The valid values are: 'binary', 'multiclass', and 'multilabel'. If 'binary' or 'multilabel', y_true values must be 0 and 1.	'auto'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import roc_auc
>>> # auto
>>> metrics = roc_auc(y_true=np.array([1, 0, 0, 1, 1]), y_score=np.array([2, -1, 0, 3, 1]))
>>> metrics
{'count': 5, 'roc_auc': 1.0}
>>> # binary
>>> metrics = roc_auc(
...     y_true=np.array([1, 0, 0, 1, 1]),
...     y_score=np.array([2, -1, 0, 3, 1]),
...     label_type="binary",
... )
>>> metrics
{'count': 5, 'roc_auc': 1.0}
>>> # multiclass
>>> metrics = roc_auc(
...     y_true=np.array([0, 0, 1, 1, 2, 2]),
...     y_score=np.array(
...         [
...             [0.7, 0.2, 0.1],
...             [0.4, 0.3, 0.3],
...             [0.1, 0.8, 0.1],
...             [0.2, 0.3, 0.5],
...             [0.4, 0.4, 0.2],
...             [0.1, 0.2, 0.7],
...         ]
...     ),
...     label_type="multiclass",
... )
>>> metrics
{'count': 6,
 'macro_roc_auc': 0.833...,
 'micro_roc_auc': 0.826...,
 'roc_auc': array([0.9375, 0.8125, 0.75  ]),
 'weighted_roc_auc': 0.833...}
>>> # multilabel
>>> metrics = roc_auc(
...     y_true=np.array([[1, 0, 1], [0, 1, 0], [0, 1, 0], [1, 0, 1], [1, 0, 1]]),
...     y_score=np.array([[2, -1, -1], [-1, 1, 2], [0, 2, 3], [3, -2, -4], [1, -3, -5]]),
...     label_type="multilabel",
... )
>>> metrics
{'count': 5,
 'macro_roc_auc': 0.666...,
 'micro_roc_auc': 0.544...,
 'roc_auc': array([1., 1., 0.]),
 'weighted_roc_auc': 0.625}

analora.metric.root_mean_squared_error

root_mean_squared_error(
    y_true: ndarray,
    y_pred: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the root mean squared error (RMSE).

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target values.	required
y_pred	ndarray	The predicted values.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import root_mean_squared_error
>>> root_mean_squared_error(
...     y_true=np.array([1, 2, 3, 4, 5]), y_pred=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'root_mean_squared_error': 0.0}

analora.metric.spearmanr

spearmanr(
    x: ndarray,
    y: ndarray,
    *,
    alternative: str = "two-sided",
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Spearman correlation coefficient and p-value for testing non-correlation.

Parameters:

Name	Type	Description	Default
x	ndarray	The first input array.	required
y	ndarray	The second input array.	required
alternative	str	The alternative hypothesis. Default is 'two-sided'. The following options are available: - 'two-sided': the correlation is nonzero - 'less': the correlation is negative (less than zero) - 'greater': the correlation is positive (greater than zero)	'two-sided'
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import spearmanr
>>> spearmanr(
...     x=np.array([1, 2, 3, 4, 5, 6, 7, 8, 9]),
...     y=np.array([1, 2, 3, 4, 5, 6, 7, 8, 9]),
... )
{'count': 9, 'spearman_coeff': 1.0, 'spearman_pvalue': 0.0}

analora.metric.top_k_accuracy

top_k_accuracy(
    y_true: ndarray,
    y_score: ndarray,
    *,
    k: Sequence[int] = (2,),
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float | ndarray]

Return the Area Under the Top-k Accuracy classification metrics.

Parameters:

Name	Type	Description	Default
y_true	ndarray	The ground truth target labels. This input must be an array of shape (n_samples,).	required
y_score	ndarray	The target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions. The binary case expects scores with shape (n_samples,) while the multiclass case expects scores with shape (n_samples, n_classes).	required
k	Sequence[int]	The numbers of most likely outcomes considered to find the correct label.	(2,)
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float | ndarray]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import top_k_accuracy
>>> # binary
>>> metrics = top_k_accuracy(
...     y_true=np.array([1, 0, 0, 1, 1]), y_score=np.array([2, -1, 0, 3, 1]), k=[1, 2]
... )
>>> metrics
{'count': 5, 'top_1_accuracy': 1.0, 'top_2_accuracy': 1.0}
>>> # multiclass
>>> metrics = top_k_accuracy(
...     y_true=np.array([0, 1, 2, 2]),
...     y_score=np.array(
...         [[0.5, 0.2, 0.2], [0.3, 0.4, 0.2], [0.2, 0.4, 0.3], [0.7, 0.2, 0.1]]
...     ),
...     k=[1, 2, 3],
... )
>>> metrics
{'count': 4, 'top_1_accuracy': 0.5, 'top_2_accuracy': 0.75, 'top_3_accuracy': 1.0}

analora.metric.wasserstein_distance

wasserstein_distance(
    u_values: ndarray,
    v_values: ndarray,
    *,
    prefix: str = "",
    suffix: str = "",
    nan_policy: str = "propagate"
) -> dict[str, float]

Return the Wasserstein distance between two 1D discrete distributions.

Parameters:

Name	Type	Description	Default
u_values	ndarray	An array that contains a sample from a probability distribution or the support (set of all possible values) of a probability distribution. Each element is an observation or possible value.	required
v_values	ndarray	An array that contains a sample from or the support of a second distribution.	required
prefix	str	The key prefix in the returned dictionary.	''
suffix	str	The key suffix in the returned dictionary.	''
nan_policy	str	The policy on how to handle NaN values in the input arrays. The following options are available: 'omit', 'propagate', and 'raise'.	'propagate'

Returns:

Type	Description
dict[str, float]	The computed metrics.

Example usage:

>>> import numpy as np
>>> from analora.metric import wasserstein_distance
>>> wasserstein_distance(
...     u_values=np.array([1, 2, 3, 4, 5]), v_values=np.array([1, 2, 3, 4, 5])
... )
{'count': 5, 'wasserstein_distance': 0.0}