Skip to content

karbonn.metric

karbonn.metric

Contain the metrics.

karbonn.metric.AbsoluteError

Bases: BaseStateMetric

Implement the absolute error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric
AbsoluteError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_err_")
{'abs_err_mean': 0.5,
 'abs_err_min': 0.0,
 'abs_err_max': 1.0,
 'abs_err_sum': 2.0,
 'abs_err_count': 4}
karbonn.metric.AbsoluteError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.AbsoluteRelativeError

Bases: BaseStateMetric

Implement the absolute relative error metric.

Parameters:

Name Type Description Default
eps float

An arbitrary small strictly positive number to avoid undefined results when the target is zero.

 1e-08
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric
AbsoluteRelativeError(
  (eps): 1e-08
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_rel_err_")
{'abs_rel_err_mean': 0.5,
 'abs_rel_err_min': 0.0,
 'abs_rel_err_max': 1.0,
 'abs_rel_err_sum': 2.0,
 'abs_rel_err_count': 4}
karbonn.metric.AbsoluteRelativeError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute percentage error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.Accuracy

Bases: BaseStateMetric

Implement a categorical accuracy metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, AccuracyState is instantiated.

 None
transform Module | dict | None

The transformation applied on the predictions to generate the predicted categorical labels. If None, the identity module is used. The transform module must take a single input tensor and output a single tensor.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric
Accuracy(
  (state): AccuracyState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
  (transform): Identity()
)
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0, 'count': 2}
>>> metric(torch.tensor([[0, 2]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.75, 'count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[1, 1]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.5, 'count': 2}
karbonn.metric.Accuracy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the accuracy metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels or the predictions. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type long or float. If the input is the predictions/scores, then the transform module should be set to transform the predictions/scores to categorical labels where the values are in {0, 1, ..., num_classes-1}.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric(torch.tensor([1, 2, 0, 1]), torch.tensor([1, 2, 0, 1]))
>>> metric.value()
{'accuracy': 1.0, 'count': 4}

karbonn.metric.BaseMetric

Bases: Module

Define the base class to implement a metric.

This class is used to register the metric using the metaclass factory. Child classes must implement the following methods:

- ``forward``
- ``get_records``
- ``reset``
- ``value``
karbonn.metric.BaseMetric.get_records abstractmethod 
get_records(
    prefix: str = "", suffix: str = ""
) -> tuple[BaseRecord, ...]

Get the records for the metrics associated to the current state.

Parameters:

Name Type Description Default
prefix str

The key prefix in the record names.

 ''
suffix str

The key suffix in the record names.

 ''

Returns:

Name Type Description
tuple tuple[BaseRecord, ...]

The records.

Example usage:

>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
 MinScalarRecord(name=error_min, max_size=10, size=0),
 MinScalarRecord(name=error_max, max_size=10, size=0),
 MinScalarRecord(name=error_sum, max_size=10, size=0))
karbonn.metric.BaseMetric.reset abstractmethod 
reset() -> None

Reset the metric.

karbonn.metric.BaseMetric.value abstractmethod 
value(prefix: str = '', suffix: str = '') -> dict

Evaluate the metric and return the results given all the examples previously seen.

Parameters:

Name Type Description Default
prefix str

The key prefix in the returned dictionary.

 ''
suffix str

The key suffix in the returned dictionary.

 ''

Returns:

Type Description
dict

The results of the metric.

karbonn.metric.BaseStateMetric

Bases: BaseMetric

Define a base class to implement a metric with a state.

Child classes must implement the following method:

- ``forward``

Parameters:

Name Type Description Default
state BaseState | dict

The metric state or its configuration.

 required

karbonn.metric.BinaryConfusionMatrix

Bases: BaseMetric

Implement a confusion tracker metric for binary labels.

Parameters:

Name Type Description Default
betas Sequence[int | float]

The betas used to compute the f-beta scores.

 (1)
tracker BinaryConfusionMatrixTracker | None

The value tracker. If None, a BinaryConfusionMatrixTracker object is initialized.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric
BinaryConfusionMatrix(
  (betas): (1,)
  (tracker): BinaryConfusionMatrixTracker(num_classes=2, count=0)
  (track_count): True
)
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 4,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 2,
 'f1_score': 1.0}
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 6,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 3,
 'true_positive_rate': 1.0,
 'true_positive': 3,
 'f1_score': 1.0}
>>> metric.reset()
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 2,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 1,
 'true_positive_rate': 1.0,
 'true_positive': 1,
 'f1_score': 1.0}
karbonn.metric.BinaryConfusionMatrix.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the confusion tracker metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required
target Tensor

The binary targets where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type bool or long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 4,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 2,
 'f1_score': 1.0}

karbonn.metric.CategoricalConfusionMatrix

Bases: BaseMetric

Implement a confusion tracker metric for multi-class labels.

Parameters:

Name Type Description Default
num_classes int

The number of classes.

 required
betas Sequence[int | float]

The betas used to compute the f-beta scores.

 (1)
tracker MulticlassConfusionMatrixTracker | None

The value tracker. If None, a BinaryConfusionMatrixTracker object is initialized.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric
CategoricalConfusionMatrix(
  (betas): (1,)
  (tracker): MulticlassConfusionMatrixTracker(num_classes=3, count=0)
  (track_count): True
)
>>> metric(
...     prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
...     target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
 'balanced_accuracy': 0.333333...,
 'count': 6,
 'macro_precision': 0.555555...,
 'macro_recall': 0.333333...,
 'macro_f1_score': 0.388888...,
 'micro_precision': 0.5,
 'micro_recall': 0.5,
 'micro_f1_score': 0.5,
 'weighted_precision': 0.833333...,
 'weighted_recall': 0.5,
 'weighted_f1_score': 0.583333...,
 'precision': tensor([0.6667, 0.0000, 1.0000]),
 'recall': tensor([0.6667, 0.0000, 0.3333]),
 'f1_score': tensor([0.6667, 0.0000, 0.5000])}
>>> metric(prediction=torch.tensor([1, 0]), target=torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 0.625,
 'balanced_accuracy': 0.694444...,
 'count': 8,
 'macro_precision': 0.694444...,
 'macro_recall': 0.694444...,
 'macro_f1_score': 0.583333...,
 'micro_precision': 0.625,
 'micro_recall': 0.625,
 'micro_f1_score': 0.625,
 'weighted_precision': 0.791666...,
 'weighted_recall': 0.625,
 'weighted_f1_score': 0.625,
 'precision': tensor([0.7500, 0.3333, 1.0000]),
 'recall': tensor([0.7500, 1.0000, 0.3333]),
 'f1_score': tensor([0.7500, 0.5000, 0.5000])}
>>> metric.reset()
>>> metric(prediction=torch.tensor([1, 0, 2]), target=torch.tensor([1, 0, 2]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'count': 3,
 'macro_precision': 1.0,
 'macro_recall': 1.0,
 'macro_f1_score': 1.0,
 'micro_precision': 1.0,
 'micro_recall': 1.0,
 'micro_f1_score': 1.0,
 'weighted_precision': 1.0,
 'weighted_recall': 1.0,
 'weighted_f1_score': 1.0,
 'precision': tensor([1., 1., 1.]),
 'recall': tensor([1., 1., 1.]),
 'f1_score': tensor([1., 1., 1.])}
karbonn.metric.CategoricalConfusionMatrix.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the confusion tracker metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required
target Tensor

The binary targets where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type bool or long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric(
...     prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
...     target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
 'balanced_accuracy': 0.333333...,
 'count': 6,
 'macro_precision': 0.555555...,
 'macro_recall': 0.333333...,
 'macro_f1_score': 0.388888...,
 'micro_precision': 0.5,
 'micro_recall': 0.5,
 'micro_f1_score': 0.5,
 'weighted_precision': 0.833333...,
 'weighted_recall': 0.5,
 'weighted_f1_score': 0.583333...,
 'precision': tensor([0.6667, 0.0000, 1.0000]),
 'recall': tensor([0.6667, 0.0000, 0.3333]),
 'f1_score': tensor([0.6667, 0.0000, 0.5000])}

karbonn.metric.CategoricalCrossEntropy

Bases: BaseStateMetric

Implement a metric to compute the categorical cross-entropy.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, MeanErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric
CategoricalCrossEntropy(
  (state): MeanErrorState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 0.861983..., 'count': 6}
>>> metric.reset()
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 1.098612..., 'count': 2}
karbonn.metric.CategoricalCrossEntropy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels as a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type float.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}

karbonn.metric.EmptyMetricError

Bases: Exception

Raised when an empty metric is evaluated.

karbonn.metric.LogCoshError

Bases: BaseStateMetric

Implement a metric to compute the logarithm of the hyperbolic cosine of the prediction error.

Parameters:

Name Type Description Default
scale float

The scale factor.

 1.0
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric
LogCoshError(
  (scale): 1.0
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.072296...,
 'min': 0.0,
 'max': 0.433780...,
 'sum': 0.867561...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("log_cosh_err_")
{'log_cosh_err_mean': 0.216890...,
 'log_cosh_err_min': 0.0,
 'log_cosh_err_max': 0.433780...,
 'log_cosh_err_sum': 0.867561...,
 'log_cosh_err_count': 4}
karbonn.metric.LogCoshError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.NormalizedMeanSquaredError

Bases: BaseStateMetric

Implement the normalized mean squared error (NMSE) metric.

Note: this metric does not work if all the targets are zero.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, NormalizedMeanSquaredErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric
NormalizedMeanSquaredError(
  (state): NormalizedMeanSquaredErrorState(
      (squared_errors): MeanTensorTracker(count=0, total=0.0)
      (squared_targets): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.5, 'count': 4}
karbonn.metric.NormalizedMeanSquaredError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the normalized mean squared error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}

karbonn.metric.RootMeanSquaredError

Bases: SquaredError

Implement the squared error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, RootMeanErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import RootMeanSquaredError
>>> metric = RootMeanSquaredError()
>>> metric
RootMeanSquaredError(
  (state): RootMeanErrorState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.408248..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.707106..., 'count': 4}

karbonn.metric.SquaredAsinhError

Bases: BaseStateMetric

Implement a metric to compute the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric
SquaredAsinhError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.129469...,
 'min': 0.0,
 'max': 0.776819...,
 'sum': 1.553638...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_asinh_err_")
{'sq_asinh_err_mean': 0.388409...,
 'sq_asinh_err_min': 0.0,
 'sq_asinh_err_max': 0.776819...,
 'sq_asinh_err_sum': 1.553638...,
 'sq_asinh_err_count': 4}
karbonn.metric.SquaredAsinhError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.SquaredError

Bases: BaseStateMetric

Implement the squared error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric
SquaredError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666...,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_err_")
{'sq_err_mean': 0.5,
 'sq_err_min': 0.0,
 'sq_err_max': 1.0,
 'sq_err_sum': 2.0,
 'sq_err_count': 4}
karbonn.metric.SquaredError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared logarithmic error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.SquaredLogError

Bases: BaseStateMetric

Implement the squared logarithmic error (SLE) metric.

This metric is best to use when targets having exponential growth, such as population counts, average sales of a commodity over a span of years etc. Note that this metric penalizes an under-predicted estimate greater than an over-predicted estimate.

Note: this metric only works with positive value (0 included).

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric
SquaredLogError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.080075...,
 'min': 0.0,
 'max': 0.480453...,
 'sum': 0.960906...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_log_err_")
{'sq_log_err_mean': 0.240226...,
 'sq_log_err_min': 0.0,
 'sq_log_err_max': 0.480453...,
 'sq_log_err_sum': 0.960906...,
 'sq_log_err_count': 4}
karbonn.metric.SquaredLogError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared logarithmic error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.SymmetricAbsoluteRelativeError

Bases: BaseStateMetric

Implement the symmetric absolute relative error (SARE) metric.

This metric tracks the mean, maximum and minimum absolute relative error values.

Parameters:

Name Type Description Default
eps float

An arbitrary small strictly positive number to avoid undefined results when the target is zero.

 1e-08
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric
SymmetricAbsoluteRelativeError(
  (eps): 1e-08
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.3333333333333333,
 'min': 0.0,
 'max': 2.0,
 'sum': 4.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sym_abs_rel_err_")
{'sym_abs_rel_err_mean': 1.0,
 'sym_abs_rel_err_min': 0.0,
 'sym_abs_rel_err_max': 2.0,
 'sym_abs_rel_err_sum': 4.0,
 'sym_abs_rel_err_count': 4}
karbonn.metric.SymmetricAbsoluteRelativeError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute percentage error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.TopKAccuracy

Bases: BaseMetric

Implement the accuracy at k metric a.k.a. top-k accuracy.

Parameters:

Name Type Description Default
topk Sequence[int]

The k values used to evaluate the top-k accuracy metric.

 (1, 5)

Example usage:

>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric
TopKAccuracy(
  (topk): (1,)
  (states):
    (1): AccuracyState(
          (tracker): MeanTensorTracker(count=0, total=0.0)
          (track_count): True
        )
)
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value()
{'top_1_accuracy': 0.75, 'top_1_count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value("acc_")
{'acc_top_1_accuracy': 0.5, 'acc_top_1_count': 2}
karbonn.metric.TopKAccuracy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the accuracy metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels as a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type long or float.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}

karbonn.metric.setup_metric 

setup_metric(metric: BaseMetric | dict) -> BaseMetric

Set up the metric.

Parameters:

Name Type Description Default
metric BaseMetric | dict

The metric or its configuration.

 required

Returns:

Type Description
BaseMetric

The instantiated metric.

karbonn.metric.classification

Contain classification metrics.

karbonn.metric.classification.Accuracy

Bases: BaseStateMetric

Implement a categorical accuracy metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, AccuracyState is instantiated.

 None
transform Module | dict | None

The transformation applied on the predictions to generate the predicted categorical labels. If None, the identity module is used. The transform module must take a single input tensor and output a single tensor.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric
Accuracy(
  (state): AccuracyState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
  (transform): Identity()
)
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0, 'count': 2}
>>> metric(torch.tensor([[0, 2]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.75, 'count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[1, 1]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.5, 'count': 2}
karbonn.metric.classification.Accuracy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the accuracy metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels or the predictions. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type long or float. If the input is the predictions/scores, then the transform module should be set to transform the predictions/scores to categorical labels where the values are in {0, 1, ..., num_classes-1}.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric(torch.tensor([1, 2, 0, 1]), torch.tensor([1, 2, 0, 1]))
>>> metric.value()
{'accuracy': 1.0, 'count': 4}

karbonn.metric.classification.BinaryConfusionMatrix

Bases: BaseMetric

Implement a confusion tracker metric for binary labels.

Parameters:

Name Type Description Default
betas Sequence[int | float]

The betas used to compute the f-beta scores.

 (1)
tracker BinaryConfusionMatrixTracker | None

The value tracker. If None, a BinaryConfusionMatrixTracker object is initialized.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric
BinaryConfusionMatrix(
  (betas): (1,)
  (tracker): BinaryConfusionMatrixTracker(num_classes=2, count=0)
  (track_count): True
)
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 4,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 2,
 'f1_score': 1.0}
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 6,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 3,
 'true_positive_rate': 1.0,
 'true_positive': 3,
 'f1_score': 1.0}
>>> metric.reset()
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 2,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 1,
 'true_positive_rate': 1.0,
 'true_positive': 1,
 'f1_score': 1.0}
karbonn.metric.classification.BinaryConfusionMatrix.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the confusion tracker metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required
target Tensor

The binary targets where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type bool or long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'false_negative_rate': 0.0,
 'false_negative': 0,
 'false_positive_rate': 0.0,
 'false_positive': 0,
 'jaccard_index': 1.0,
 'count': 4,
 'precision': 1.0,
 'recall': 1.0,
 'true_negative_rate': 1.0,
 'true_negative': 2,
 'true_positive_rate': 1.0,
 'true_positive': 2,
 'f1_score': 1.0}

karbonn.metric.classification.CategoricalConfusionMatrix

Bases: BaseMetric

Implement a confusion tracker metric for multi-class labels.

Parameters:

Name Type Description Default
num_classes int

The number of classes.

 required
betas Sequence[int | float]

The betas used to compute the f-beta scores.

 (1)
tracker MulticlassConfusionMatrixTracker | None

The value tracker. If None, a BinaryConfusionMatrixTracker object is initialized.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric
CategoricalConfusionMatrix(
  (betas): (1,)
  (tracker): MulticlassConfusionMatrixTracker(num_classes=3, count=0)
  (track_count): True
)
>>> metric(
...     prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
...     target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
 'balanced_accuracy': 0.333333...,
 'count': 6,
 'macro_precision': 0.555555...,
 'macro_recall': 0.333333...,
 'macro_f1_score': 0.388888...,
 'micro_precision': 0.5,
 'micro_recall': 0.5,
 'micro_f1_score': 0.5,
 'weighted_precision': 0.833333...,
 'weighted_recall': 0.5,
 'weighted_f1_score': 0.583333...,
 'precision': tensor([0.6667, 0.0000, 1.0000]),
 'recall': tensor([0.6667, 0.0000, 0.3333]),
 'f1_score': tensor([0.6667, 0.0000, 0.5000])}
>>> metric(prediction=torch.tensor([1, 0]), target=torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 0.625,
 'balanced_accuracy': 0.694444...,
 'count': 8,
 'macro_precision': 0.694444...,
 'macro_recall': 0.694444...,
 'macro_f1_score': 0.583333...,
 'micro_precision': 0.625,
 'micro_recall': 0.625,
 'micro_f1_score': 0.625,
 'weighted_precision': 0.791666...,
 'weighted_recall': 0.625,
 'weighted_f1_score': 0.625,
 'precision': tensor([0.7500, 0.3333, 1.0000]),
 'recall': tensor([0.7500, 1.0000, 0.3333]),
 'f1_score': tensor([0.7500, 0.5000, 0.5000])}
>>> metric.reset()
>>> metric(prediction=torch.tensor([1, 0, 2]), target=torch.tensor([1, 0, 2]))
>>> metric.value()
{'accuracy': 1.0,
 'balanced_accuracy': 1.0,
 'count': 3,
 'macro_precision': 1.0,
 'macro_recall': 1.0,
 'macro_f1_score': 1.0,
 'micro_precision': 1.0,
 'micro_recall': 1.0,
 'micro_f1_score': 1.0,
 'weighted_precision': 1.0,
 'weighted_recall': 1.0,
 'weighted_f1_score': 1.0,
 'precision': tensor([1., 1., 1.]),
 'recall': tensor([1., 1., 1.]),
 'f1_score': tensor([1., 1., 1.])}
karbonn.metric.classification.CategoricalConfusionMatrix.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the confusion tracker metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required
target Tensor

The binary targets where the values are 0 or 1. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type bool or long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric(
...     prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
...     target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
 'balanced_accuracy': 0.333333...,
 'count': 6,
 'macro_precision': 0.555555...,
 'macro_recall': 0.333333...,
 'macro_f1_score': 0.388888...,
 'micro_precision': 0.5,
 'micro_recall': 0.5,
 'micro_f1_score': 0.5,
 'weighted_precision': 0.833333...,
 'weighted_recall': 0.5,
 'weighted_f1_score': 0.583333...,
 'precision': tensor([0.6667, 0.0000, 1.0000]),
 'recall': tensor([0.6667, 0.0000, 0.3333]),
 'f1_score': tensor([0.6667, 0.0000, 0.5000])}

karbonn.metric.classification.CategoricalCrossEntropy

Bases: BaseStateMetric

Implement a metric to compute the categorical cross-entropy.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, MeanErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric
CategoricalCrossEntropy(
  (state): MeanErrorState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 0.861983..., 'count': 6}
>>> metric.reset()
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 1.098612..., 'count': 2}
karbonn.metric.classification.CategoricalCrossEntropy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels as a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type float.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}

karbonn.metric.classification.TopKAccuracy

Bases: BaseMetric

Implement the accuracy at k metric a.k.a. top-k accuracy.

Parameters:

Name Type Description Default
topk Sequence[int]

The k values used to evaluate the top-k accuracy metric.

 (1, 5)

Example usage:

>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric
TopKAccuracy(
  (topk): (1,)
  (states):
    (1): AccuracyState(
          (tracker): MeanTensorTracker(count=0, total=0.0)
          (track_count): True
        )
)
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value()
{'top_1_accuracy': 0.75, 'top_1_count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value("acc_")
{'acc_top_1_accuracy': 0.5, 'acc_top_1_count': 2}
karbonn.metric.classification.TopKAccuracy.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the accuracy metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predicted labels as a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, num_classes) and type long or float.

 required
target Tensor

The categorical targets. The values have to be in {0, 1, ..., num_classes-1}. This input must be a torch.Tensor of shape (d0, d1, ..., dn) or (d0, d1, ..., dn, 1) and type long or float.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}

karbonn.metric.regression

Contain the regression metrics.

karbonn.metric.regression.AbsoluteError

Bases: BaseStateMetric

Implement the absolute error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric
AbsoluteError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_err_")
{'abs_err_mean': 0.5,
 'abs_err_min': 0.0,
 'abs_err_max': 1.0,
 'abs_err_sum': 2.0,
 'abs_err_count': 4}
karbonn.metric.regression.AbsoluteError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.AbsoluteRelativeError

Bases: BaseStateMetric

Implement the absolute relative error metric.

Parameters:

Name Type Description Default
eps float

An arbitrary small strictly positive number to avoid undefined results when the target is zero.

 1e-08
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric
AbsoluteRelativeError(
  (eps): 1e-08
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_rel_err_")
{'abs_rel_err_mean': 0.5,
 'abs_rel_err_min': 0.0,
 'abs_rel_err_max': 1.0,
 'abs_rel_err_sum': 2.0,
 'abs_rel_err_count': 4}
karbonn.metric.regression.AbsoluteRelativeError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute percentage error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.LogCoshError

Bases: BaseStateMetric

Implement a metric to compute the logarithm of the hyperbolic cosine of the prediction error.

Parameters:

Name Type Description Default
scale float

The scale factor.

 1.0
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric
LogCoshError(
  (scale): 1.0
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.072296...,
 'min': 0.0,
 'max': 0.433780...,
 'sum': 0.867561...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("log_cosh_err_")
{'log_cosh_err_mean': 0.216890...,
 'log_cosh_err_min': 0.0,
 'log_cosh_err_max': 0.433780...,
 'log_cosh_err_sum': 0.867561...,
 'log_cosh_err_count': 4}
karbonn.metric.regression.LogCoshError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.NormalizedMeanSquaredError

Bases: BaseStateMetric

Implement the normalized mean squared error (NMSE) metric.

Note: this metric does not work if all the targets are zero.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, NormalizedMeanSquaredErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric
NormalizedMeanSquaredError(
  (state): NormalizedMeanSquaredErrorState(
      (squared_errors): MeanTensorTracker(count=0, total=0.0)
      (squared_targets): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.5, 'count': 4}
karbonn.metric.regression.NormalizedMeanSquaredError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the normalized mean squared error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}

karbonn.metric.regression.RootMeanSquaredError

Bases: SquaredError

Implement the squared error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, RootMeanErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import RootMeanSquaredError
>>> metric = RootMeanSquaredError()
>>> metric
RootMeanSquaredError(
  (state): RootMeanErrorState(
      (tracker): MeanTensorTracker(count=0, total=0.0)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.408248..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.707106..., 'count': 4}

karbonn.metric.regression.SquaredAsinhError

Bases: BaseStateMetric

Implement a metric to compute the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric
SquaredAsinhError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.129469...,
 'min': 0.0,
 'max': 0.776819...,
 'sum': 1.553638...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_asinh_err_")
{'sq_asinh_err_mean': 0.388409...,
 'sq_asinh_err_min': 0.0,
 'sq_asinh_err_max': 0.776819...,
 'sq_asinh_err_sum': 1.553638...,
 'sq_asinh_err_count': 4}
karbonn.metric.regression.SquaredAsinhError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.SquaredError

Bases: BaseStateMetric

Implement the squared error metric.

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric
SquaredError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666...,
 'min': 0.0,
 'max': 1.0,
 'sum': 2.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_err_")
{'sq_err_mean': 0.5,
 'sq_err_min': 0.0,
 'sq_err_max': 1.0,
 'sq_err_sum': 2.0,
 'sq_err_count': 4}
karbonn.metric.regression.SquaredError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared logarithmic error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.SquaredLogError

Bases: BaseStateMetric

Implement the squared logarithmic error (SLE) metric.

This metric is best to use when targets having exponential growth, such as population counts, average sales of a commodity over a span of years etc. Note that this metric penalizes an under-predicted estimate greater than an over-predicted estimate.

Note: this metric only works with positive value (0 included).

Parameters:

Name Type Description Default
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric
SquaredLogError(
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.080075...,
 'min': 0.0,
 'max': 0.480453...,
 'sum': 0.960906...,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_log_err_")
{'sq_log_err_mean': 0.240226...,
 'sq_log_err_min': 0.0,
 'sq_log_err_max': 0.480453...,
 'sq_log_err_sum': 0.960906...,
 'sq_log_err_count': 4}
karbonn.metric.regression.SquaredLogError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the squared logarithmic error metric given a mini- batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.regression.SymmetricAbsoluteRelativeError

Bases: BaseStateMetric

Implement the symmetric absolute relative error (SARE) metric.

This metric tracks the mean, maximum and minimum absolute relative error values.

Parameters:

Name Type Description Default
eps float

An arbitrary small strictly positive number to avoid undefined results when the target is zero.

 1e-08
state BaseState | dict | None

The metric state or its configuration. If None, ErrorState is instantiated.

 None

Example usage:

>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric
SymmetricAbsoluteRelativeError(
  (eps): 1e-08
  (state): ErrorState(
      (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
      (track_count): True
    )
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.3333333333333333,
 'min': 0.0,
 'max': 2.0,
 'sum': 4.0,
 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sym_abs_rel_err_")
{'sym_abs_rel_err_mean': 1.0,
 'sym_abs_rel_err_min': 0.0,
 'sym_abs_rel_err_max': 2.0,
 'sym_abs_rel_err_sum': 4.0,
 'sym_abs_rel_err_count': 4}
karbonn.metric.regression.SymmetricAbsoluteRelativeError.forward 
forward(prediction: Tensor, target: Tensor) -> None

Update the mean absolute percentage error metric given a mini-batch of examples.

Parameters:

Name Type Description Default
prediction Tensor

The predictions as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required
target Tensor

The target tensor as a torch.Tensor of shape (d0, d1, ..., dn) and type float or long.

 required

Example usage:

>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
 'min': 0.0,
 'max': 0.0,
 'sum': 0.0,
 'count': 8}

karbonn.metric.state

Contain the metric states.

karbonn.metric.state.AccuracyState

Bases: BaseState

Implement a metric state to compute the accuracy.

This state has a constant space complexity.

Parameters:

Name Type Description Default
tracker MeanTensorTracker | None

The mean value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import AccuracyState
>>> state = AccuracyState()
>>> state
AccuracyState(
  (tracker): MeanTensorTracker(count=0, total=0.0)
  (track_count): True
)
>>> state.get_records()
(MaxScalarRecord(name=accuracy, max_size=10, size=0),)
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25, 'count': 16}
karbonn.metric.state.AccuracyState.update 
update(correct: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
correct Tensor

A tensor that indicates the correct predictions. 1 indicates a correct prediction and 0 indicates a bad prediction.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import AccuracyState
>>> state = AccuracyState()
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25, 'count': 16}

karbonn.metric.state.BaseState

Bases: ABC

Define a base class to implement a metric state.

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
  (track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
 MinScalarRecord(name=error_min, max_size=10, size=0),
 MinScalarRecord(name=error_max, max_size=10, size=0),
 MinScalarRecord(name=error_sum, max_size=10, size=0))
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
 'error_min': 0.0,
 'error_max': 5.0,
 'error_sum': 15.0,
 'error_count': 6}
karbonn.metric.state.BaseState.count abstractmethod property 
count: int

The number of predictions in the state.

karbonn.metric.state.BaseState.clone 
clone() -> BaseState

Create a copy of the current state.

Returns:

Type Description
BaseState

A copy of the current state.

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state_cloned = state.clone()
>>> state.update(torch.ones(3))
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=9, total=18.0, min_value=0, max_value=5)
  (track_count): True
)
>>> state_cloned
ErrorState(
  (tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0.0, max_value=5.0)
  (track_count): True
)
karbonn.metric.state.BaseState.equal abstractmethod 
equal(other: Any) -> bool

Indicate if two metric states are equal or not.

Parameters:

Name Type Description Default
other Any

The other metric state to compare.

 required

Returns:

Type Description
bool

True if the two metric states are equal, otherwise False.

 
Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> from karbonn.utils.tracker import ScalableTensorTracker
>>> state = ErrorState()
>>> state.equal(ErrorState())
True
>>> state.equal(
...     ErrorState(ScalableTensorTracker(count=4, total=10.0, min_value=0.0, max_value=5.0))
... )
False
karbonn.metric.state.BaseState.get_records abstractmethod 
get_records(
    prefix: str = "", suffix: str = ""
) -> tuple[BaseRecord, ...]

Get the records for the metrics associated to the current state.

Parameters:

Name Type Description Default
prefix str

The key prefix in the record names.

 ''
suffix str

The key suffix in the record names.

 ''

Returns:

Name Type Description
tuple tuple[BaseRecord, ...]

The records.

Example usage:

>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
 MinScalarRecord(name=error_min, max_size=10, size=0),
 MinScalarRecord(name=error_max, max_size=10, size=0),
 MinScalarRecord(name=error_sum, max_size=10, size=0))
karbonn.metric.state.BaseState.reset abstractmethod 
reset() -> None

Reset the state.

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0, max_value=5)
  (track_count): True
)
>>> state.reset()
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
  (track_count): True
)
karbonn.metric.state.BaseState.update abstractmethod 
update(*args: Any, **kwargs: Any) -> None

Update the metric state.

The exact signature for this method depends on each metric state implementation.

Parameters:

Name Type Description Default
*args Any

Variable length argument list.

 ()
**kwargs Any

Arbitrary keyword arguments.

 {}

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0, max_value=5)
  (track_count): True
)
karbonn.metric.state.BaseState.value abstractmethod 
value(
    prefix: str = "", suffix: str = ""
) -> dict[str, float]

Compute the metrics given the current state.

Parameters:

Name Type Description Default
prefix str

The key prefix in the returned dictionary.

 ''
suffix str

The key suffix in thenreturned dictionary.

 ''

Returns:

Type Description
dict[str, float]

The metric values.

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
 'error_min': 0.0,
 'error_max': 5.0,
 'error_sum': 15.0,
 'error_count': 6}

karbonn.metric.state.ErrorState

Bases: BaseState

Implement a metric state to capture some metrics about the errors.

This state has a constant space complexity.

Parameters:

Name Type Description Default
tracker ScalableTensorTracker | None

The value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
  (track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
 MinScalarRecord(name=error_min, max_size=10, size=0),
 MinScalarRecord(name=error_max, max_size=10, size=0),
 MinScalarRecord(name=error_sum, max_size=10, size=0))
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
 'error_min': 0.0,
 'error_max': 5.0,
 'error_sum': 15.0,
 'error_count': 6}
karbonn.metric.state.ErrorState.update 
update(error: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
error Tensor

A tensor of errors.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
 'error_min': 0.0,
 'error_max': 5.0,
 'error_sum': 15.0,
 'error_count': 6}

karbonn.metric.state.ExtendedAccuracyState

Bases: BaseState

Implement a metric state to compute the accuracy and other metrics.

This state has a constant space complexity.

Parameters:

Name Type Description Default
tracker MeanTensorTracker | None

The mean value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import ExtendedAccuracyState
>>> state = ExtendedAccuracyState()
>>> state
ExtendedAccuracyState(
  (tracker): MeanTensorTracker(count=0, total=0.0)
  (track_count): True
)
>>> state.get_records()
(MaxScalarRecord(name=accuracy, max_size=10, size=0),
 MinScalarRecord(name=error, max_size=10, size=0),
 MaxScalarRecord(name=count_correct, max_size=10, size=0),
 MinScalarRecord(name=count_incorrect, max_size=10, size=0))
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25,
 'error': 0.75,
 'count_correct': 4,
 'count_incorrect': 12,
 'count': 16}
karbonn.metric.state.ExtendedAccuracyState.update 
update(correct: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
correct Tensor

A tensor that indicates the correct predictions. 1 indicates a correct prediction and 0 indicates a bad prediction.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import ExtendedAccuracyState
>>> state = ExtendedAccuracyState()
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25,
 'error': 0.75,
 'count_correct': 4,
 'count_incorrect': 12,
 'count': 16}

karbonn.metric.state.ExtendedErrorState

Bases: BaseState

Implement a metric state to capture some metrics about the errors.

This state stores all the error values, so it does not scale to large datasets. This state has a linear space complexity.

Parameters:

Name Type Description Default
quantiles Tensor | Sequence[float]

The quantile values to evaluate.

 ()
tracker TensorTracker | None

The value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import ExtendedErrorState
>>> state = ExtendedErrorState(quantiles=[0.5, 0.9])
>>> state
ExtendedErrorState(
  (quantiles): tensor([0.5000, 0.9000])
  (tracker): TensorTracker(count=0)
  (track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
 MinScalarRecord(name=error_median, max_size=10, size=0),
 MinScalarRecord(name=error_min, max_size=10, size=0),
 MinScalarRecord(name=error_max, max_size=10, size=0),
 MinScalarRecord(name=error_sum, max_size=10, size=0),
 MinScalarRecord(name=error_quantile_0.5, max_size=10, size=0),
 MinScalarRecord(name=error_quantile_0.9, max_size=10, size=0))
>>> state.update(torch.arange(11))
>>> state.value("error_")
{'error_mean': 5.0,
 'error_median': 5,
 'error_min': 0,
 'error_max': 10,
 'error_sum': 55,
 'error_std': 3.316...,
 'error_quantile_0.5': 5.0,
 'error_quantile_0.9': 9.0,
 'error_count': 11}
karbonn.metric.state.ExtendedErrorState.update 
update(error: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
error Tensor

A tensor of errors.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import ExtendedErrorState
>>> state = ExtendedErrorState(quantiles=[0.5, 0.9])
>>> state.update(torch.arange(11))
>>> state.value("error_")
{'error_mean': 5.0,
 'error_median': 5,
 'error_min': 0,
 'error_max': 10,
 'error_sum': 55,
 'error_std': 3.316...,
 'error_quantile_0.5': 5.0,
 'error_quantile_0.9': 9.0,
 'error_count': 11}

karbonn.metric.state.MeanErrorState

Bases: BaseState

Implement a metric state to capture the mean error value.

This state has a constant space complexity.

Parameters:

Name Type Description Default
tracker MeanTensorTracker | None

The mean value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import MeanErrorState
>>> state = MeanErrorState()
>>> state
MeanErrorState(
  (tracker): MeanTensorTracker(count=0, total=0.0)
  (track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5, 'error_count': 6}
karbonn.metric.state.MeanErrorState.update 
update(error: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
error Tensor

A tensor of errors.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import MeanErrorState
>>> state = MeanErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5, 'error_count': 6}

karbonn.metric.state.NormalizedMeanSquaredErrorState

Bases: BaseState

Implement a metric state to capture the normalized mean squared error value.

This state has a constant space complexity.

Parameters:

Name Type Description Default
squared_errors MeanTensorTracker | None

The value tracker for squared errors.

 None
squared_targets MeanTensorTracker | None

The value tracker for squared targets.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import NormalizedMeanSquaredErrorState
>>> state = NormalizedMeanSquaredErrorState()
>>> state
NormalizedMeanSquaredErrorState(
  (squared_errors): MeanTensorTracker(count=0, total=0.0)
  (squared_targets): MeanTensorTracker(count=0, total=0.0)
  (track_count): True
)
>>> state.get_records("nmse_")
(MinScalarRecord(name=nmse_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6), torch.ones(6))
>>> state.value("nmse_")
{'nmse_mean': 9.166..., 'nmse_count': 6}
karbonn.metric.state.NormalizedMeanSquaredErrorState.update 
update(error: Tensor, target: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
error Tensor

A tensor of errors.

 required
target Tensor

A tensor with the target values.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import NormalizedMeanSquaredErrorState
>>> state = NormalizedMeanSquaredErrorState()
>>> state.update(torch.arange(6), torch.ones(6))
>>> state.value("nmse_")
{'nmse_mean': 9.166..., 'nmse_count': 6}

karbonn.metric.state.RootMeanErrorState

Bases: BaseState

Implement a metric state to capture the root mean error value.

This state has a constant space complexity.

Parameters:

Name Type Description Default
tracker MeanTensorTracker | None

The mean value tracker.

 None
track_count bool

If True, the state tracks and returns the number of predictions.

 True

Example usage:

>>> import torch
>>> from karbonn.metric.state import RootMeanErrorState
>>> state = RootMeanErrorState()
>>> state
RootMeanErrorState(
  (tracker): MeanTensorTracker(count=0, total=0.0)
  (track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 1.581..., 'error_count': 6}
karbonn.metric.state.RootMeanErrorState.update 
update(error: Tensor) -> None

Update the metric state with a new tensor of errors.

Parameters:

Name Type Description Default
error Tensor

A tensor of errors.

 required

Example usage:

>>> import torch
>>> from karbonn.metric.state import RootMeanErrorState
>>> state = RootMeanErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 1.581..., 'error_count': 6}

karbonn.metric.state.is_state_config 

is_state_config(config: dict) -> bool

Indicate if the input configuration is a configuration for a BaseState.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name Type Description Default
config dict

The configuration to check.

 required

Returns:

Type Description
bool

True if the input configuration is a configuration for a BaseState object, otherwise False.

Example usage:

>>> from karbonn.metric.state import is_state_config
>>> is_state_config({"_target_": "karbonn.metric.state.ErrorState"})
True

karbonn.metric.state.setup_state 

setup_state(state: BaseState | dict) -> BaseState

Set up a BaseState object.

Parameters:

Name Type Description Default
state BaseState | dict

The state or its configuration.

 required

Returns:

Type Description
BaseState

The instantiated BaseState object.

Example usage:

>>> from karbonn.metric.state import setup_state
>>> state = setup_state({"_target_": "karbonn.metric.state.ErrorState"})
>>> state
ErrorState(
  (tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
  (track_count): True
)