karbonn.metric¶
karbonn.metric ¶
Contain the metrics.
karbonn.metric.AbsoluteError ¶
Bases: BaseStateMetric
Implement the absolute error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric
AbsoluteError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_err_")
{'abs_err_mean': 0.5,
'abs_err_min': 0.0,
'abs_err_max': 1.0,
'abs_err_sum': 2.0,
'abs_err_count': 4}
karbonn.metric.AbsoluteError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.AbsoluteRelativeError ¶
Bases: BaseStateMetric
Implement the absolute relative error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
eps
|
float
|
An arbitrary small strictly positive number to avoid undefined results when the target is zero. |
1e-08
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric
AbsoluteRelativeError(
(eps): 1e-08
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_rel_err_")
{'abs_rel_err_mean': 0.5,
'abs_rel_err_min': 0.0,
'abs_rel_err_max': 1.0,
'abs_rel_err_sum': 2.0,
'abs_rel_err_count': 4}
karbonn.metric.AbsoluteRelativeError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute percentage error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.Accuracy ¶
Bases: BaseStateMetric
Implement a categorical accuracy metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
transform
|
Module | dict | None
|
The transformation applied on the predictions to
generate the predicted categorical labels. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric
Accuracy(
(state): AccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
(transform): Identity()
)
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0, 'count': 2}
>>> metric(torch.tensor([[0, 2]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.75, 'count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[1, 1]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.5, 'count': 2}
karbonn.metric.Accuracy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the accuracy metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels or the predictions.
This input must be a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric(torch.tensor([1, 2, 0, 1]), torch.tensor([1, 2, 0, 1]))
>>> metric.value()
{'accuracy': 1.0, 'count': 4}
karbonn.metric.BaseMetric ¶
Bases: Module
Define the base class to implement a metric.
This class is used to register the metric using the metaclass factory. Child classes must implement the following methods:
- ``forward``
- ``get_records``
- ``reset``
- ``value``
karbonn.metric.BaseMetric.get_records
abstractmethod
¶
get_records(
prefix: str = "", suffix: str = ""
) -> tuple[BaseRecord, ...]
Get the records for the metrics associated to the current state.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prefix
|
str
|
The key prefix in the record names. |
''
|
suffix
|
str
|
The key suffix in the record names. |
''
|
Returns:
Name | Type | Description |
---|---|---|
tuple |
tuple[BaseRecord, ...]
|
The records. |
Example usage:
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
MinScalarRecord(name=error_min, max_size=10, size=0),
MinScalarRecord(name=error_max, max_size=10, size=0),
MinScalarRecord(name=error_sum, max_size=10, size=0))
karbonn.metric.BaseMetric.value
abstractmethod
¶
value(prefix: str = '', suffix: str = '') -> dict
Evaluate the metric and return the results given all the examples previously seen.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prefix
|
str
|
The key prefix in the returned dictionary. |
''
|
suffix
|
str
|
The key suffix in the returned dictionary. |
''
|
Returns:
Type | Description |
---|---|
dict
|
The results of the metric. |
karbonn.metric.BaseStateMetric ¶
Bases: BaseMetric
Define a base class to implement a metric with a state.
Child classes must implement the following method:
- ``forward``
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict
|
The metric state or its configuration. |
required |
karbonn.metric.BinaryConfusionMatrix ¶
Bases: BaseMetric
Implement a confusion tracker metric for binary labels.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
betas
|
Sequence[int | float]
|
The betas used to compute the f-beta scores. |
(1)
|
tracker
|
BinaryConfusionMatrixTracker | None
|
The value tracker. If |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric
BinaryConfusionMatrix(
(betas): (1,)
(tracker): BinaryConfusionMatrixTracker(num_classes=2, count=0)
(track_count): True
)
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 4,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 2,
'true_positive_rate': 1.0,
'true_positive': 2,
'f1_score': 1.0}
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 6,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 3,
'true_positive_rate': 1.0,
'true_positive': 3,
'f1_score': 1.0}
>>> metric.reset()
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 2,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 1,
'true_positive_rate': 1.0,
'true_positive': 1,
'f1_score': 1.0}
karbonn.metric.BinaryConfusionMatrix.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the confusion tracker metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels where the values are |
required |
target
|
Tensor
|
The binary targets where the values are |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 4,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 2,
'true_positive_rate': 1.0,
'true_positive': 2,
'f1_score': 1.0}
karbonn.metric.CategoricalConfusionMatrix ¶
Bases: BaseMetric
Implement a confusion tracker metric for multi-class labels.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
num_classes
|
int
|
The number of classes. |
required |
betas
|
Sequence[int | float]
|
The betas used to compute the f-beta scores. |
(1)
|
tracker
|
MulticlassConfusionMatrixTracker | None
|
The value tracker. If |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric
CategoricalConfusionMatrix(
(betas): (1,)
(tracker): MulticlassConfusionMatrixTracker(num_classes=3, count=0)
(track_count): True
)
>>> metric(
... prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
... target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
'balanced_accuracy': 0.333333...,
'count': 6,
'macro_precision': 0.555555...,
'macro_recall': 0.333333...,
'macro_f1_score': 0.388888...,
'micro_precision': 0.5,
'micro_recall': 0.5,
'micro_f1_score': 0.5,
'weighted_precision': 0.833333...,
'weighted_recall': 0.5,
'weighted_f1_score': 0.583333...,
'precision': tensor([0.6667, 0.0000, 1.0000]),
'recall': tensor([0.6667, 0.0000, 0.3333]),
'f1_score': tensor([0.6667, 0.0000, 0.5000])}
>>> metric(prediction=torch.tensor([1, 0]), target=torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 0.625,
'balanced_accuracy': 0.694444...,
'count': 8,
'macro_precision': 0.694444...,
'macro_recall': 0.694444...,
'macro_f1_score': 0.583333...,
'micro_precision': 0.625,
'micro_recall': 0.625,
'micro_f1_score': 0.625,
'weighted_precision': 0.791666...,
'weighted_recall': 0.625,
'weighted_f1_score': 0.625,
'precision': tensor([0.7500, 0.3333, 1.0000]),
'recall': tensor([0.7500, 1.0000, 0.3333]),
'f1_score': tensor([0.7500, 0.5000, 0.5000])}
>>> metric.reset()
>>> metric(prediction=torch.tensor([1, 0, 2]), target=torch.tensor([1, 0, 2]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'count': 3,
'macro_precision': 1.0,
'macro_recall': 1.0,
'macro_f1_score': 1.0,
'micro_precision': 1.0,
'micro_recall': 1.0,
'micro_f1_score': 1.0,
'weighted_precision': 1.0,
'weighted_recall': 1.0,
'weighted_f1_score': 1.0,
'precision': tensor([1., 1., 1.]),
'recall': tensor([1., 1., 1.]),
'f1_score': tensor([1., 1., 1.])}
karbonn.metric.CategoricalConfusionMatrix.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the confusion tracker metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels where the values are |
required |
target
|
Tensor
|
The binary targets where the values are |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric(
... prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
... target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
'balanced_accuracy': 0.333333...,
'count': 6,
'macro_precision': 0.555555...,
'macro_recall': 0.333333...,
'macro_f1_score': 0.388888...,
'micro_precision': 0.5,
'micro_recall': 0.5,
'micro_f1_score': 0.5,
'weighted_precision': 0.833333...,
'weighted_recall': 0.5,
'weighted_f1_score': 0.583333...,
'precision': tensor([0.6667, 0.0000, 1.0000]),
'recall': tensor([0.6667, 0.0000, 0.3333]),
'f1_score': tensor([0.6667, 0.0000, 0.5000])}
karbonn.metric.CategoricalCrossEntropy ¶
Bases: BaseStateMetric
Implement a metric to compute the categorical cross-entropy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric
CategoricalCrossEntropy(
(state): MeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 0.861983..., 'count': 6}
>>> metric.reset()
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 1.098612..., 'count': 2}
karbonn.metric.CategoricalCrossEntropy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels as a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
karbonn.metric.EmptyMetricError ¶
Bases: Exception
Raised when an empty metric is evaluated.
karbonn.metric.LogCoshError ¶
Bases: BaseStateMetric
Implement a metric to compute the logarithm of the hyperbolic cosine of the prediction error.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
scale
|
float
|
The scale factor. |
1.0
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric
LogCoshError(
(scale): 1.0
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.072296...,
'min': 0.0,
'max': 0.433780...,
'sum': 0.867561...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("log_cosh_err_")
{'log_cosh_err_mean': 0.216890...,
'log_cosh_err_min': 0.0,
'log_cosh_err_max': 0.433780...,
'log_cosh_err_sum': 0.867561...,
'log_cosh_err_count': 4}
karbonn.metric.LogCoshError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.NormalizedMeanSquaredError ¶
Bases: BaseStateMetric
Implement the normalized mean squared error (NMSE) metric.
Note: this metric does not work if all the targets are zero.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric
NormalizedMeanSquaredError(
(state): NormalizedMeanSquaredErrorState(
(squared_errors): MeanTensorTracker(count=0, total=0.0)
(squared_targets): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.5, 'count': 4}
karbonn.metric.NormalizedMeanSquaredError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the normalized mean squared error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
karbonn.metric.RootMeanSquaredError ¶
Bases: SquaredError
Implement the squared error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import RootMeanSquaredError
>>> metric = RootMeanSquaredError()
>>> metric
RootMeanSquaredError(
(state): RootMeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.408248..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.707106..., 'count': 4}
karbonn.metric.SquaredAsinhError ¶
Bases: BaseStateMetric
Implement a metric to compute the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric
SquaredAsinhError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.129469...,
'min': 0.0,
'max': 0.776819...,
'sum': 1.553638...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_asinh_err_")
{'sq_asinh_err_mean': 0.388409...,
'sq_asinh_err_min': 0.0,
'sq_asinh_err_max': 0.776819...,
'sq_asinh_err_sum': 1.553638...,
'sq_asinh_err_count': 4}
karbonn.metric.SquaredAsinhError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.SquaredError ¶
Bases: BaseStateMetric
Implement the squared error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric
SquaredError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666...,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_err_")
{'sq_err_mean': 0.5,
'sq_err_min': 0.0,
'sq_err_max': 1.0,
'sq_err_sum': 2.0,
'sq_err_count': 4}
karbonn.metric.SquaredError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared logarithmic error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.SquaredLogError ¶
Bases: BaseStateMetric
Implement the squared logarithmic error (SLE) metric.
This metric is best to use when targets having exponential growth, such as population counts, average sales of a commodity over a span of years etc. Note that this metric penalizes an under-predicted estimate greater than an over-predicted estimate.
Note: this metric only works with positive value (0 included).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric
SquaredLogError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.080075...,
'min': 0.0,
'max': 0.480453...,
'sum': 0.960906...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_log_err_")
{'sq_log_err_mean': 0.240226...,
'sq_log_err_min': 0.0,
'sq_log_err_max': 0.480453...,
'sq_log_err_sum': 0.960906...,
'sq_log_err_count': 4}
karbonn.metric.SquaredLogError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared logarithmic error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.SymmetricAbsoluteRelativeError ¶
Bases: BaseStateMetric
Implement the symmetric absolute relative error (SARE) metric.
This metric tracks the mean, maximum and minimum absolute relative error values.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
eps
|
float
|
An arbitrary small strictly positive number to avoid undefined results when the target is zero. |
1e-08
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric
SymmetricAbsoluteRelativeError(
(eps): 1e-08
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.3333333333333333,
'min': 0.0,
'max': 2.0,
'sum': 4.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sym_abs_rel_err_")
{'sym_abs_rel_err_mean': 1.0,
'sym_abs_rel_err_min': 0.0,
'sym_abs_rel_err_max': 2.0,
'sym_abs_rel_err_sum': 4.0,
'sym_abs_rel_err_count': 4}
karbonn.metric.SymmetricAbsoluteRelativeError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute percentage error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.TopKAccuracy ¶
Bases: BaseMetric
Implement the accuracy at k metric a.k.a. top-k accuracy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
topk
|
Sequence[int]
|
The k values used to evaluate the top-k accuracy metric. |
(1, 5)
|
Example usage:
>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric
TopKAccuracy(
(topk): (1,)
(states):
(1): AccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value()
{'top_1_accuracy': 0.75, 'top_1_count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value("acc_")
{'acc_top_1_accuracy': 0.5, 'acc_top_1_count': 2}
karbonn.metric.TopKAccuracy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the accuracy metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels as a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
karbonn.metric.setup_metric ¶
setup_metric(metric: BaseMetric | dict) -> BaseMetric
Set up the metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
metric
|
BaseMetric | dict
|
The metric or its configuration. |
required |
Returns:
Type | Description |
---|---|
BaseMetric
|
The instantiated metric. |
karbonn.metric.classification ¶
Contain classification metrics.
karbonn.metric.classification.Accuracy ¶
Bases: BaseStateMetric
Implement a categorical accuracy metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
transform
|
Module | dict | None
|
The transformation applied on the predictions to
generate the predicted categorical labels. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric
Accuracy(
(state): AccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
(transform): Identity()
)
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0, 'count': 2}
>>> metric(torch.tensor([[0, 2]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.75, 'count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[1, 1]]), torch.tensor([1, 2]))
>>> metric.value()
{'accuracy': 0.5, 'count': 2}
karbonn.metric.classification.Accuracy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the accuracy metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels or the predictions.
This input must be a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import Accuracy
>>> metric = Accuracy()
>>> metric(torch.tensor([1, 2, 0, 1]), torch.tensor([1, 2, 0, 1]))
>>> metric.value()
{'accuracy': 1.0, 'count': 4}
karbonn.metric.classification.BinaryConfusionMatrix ¶
Bases: BaseMetric
Implement a confusion tracker metric for binary labels.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
betas
|
Sequence[int | float]
|
The betas used to compute the f-beta scores. |
(1)
|
tracker
|
BinaryConfusionMatrixTracker | None
|
The value tracker. If |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric
BinaryConfusionMatrix(
(betas): (1,)
(tracker): BinaryConfusionMatrixTracker(num_classes=2, count=0)
(track_count): True
)
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 4,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 2,
'true_positive_rate': 1.0,
'true_positive': 2,
'f1_score': 1.0}
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 6,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 3,
'true_positive_rate': 1.0,
'true_positive': 3,
'f1_score': 1.0}
>>> metric.reset()
>>> metric(torch.tensor([1, 0]), torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 2,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 1,
'true_positive_rate': 1.0,
'true_positive': 1,
'f1_score': 1.0}
karbonn.metric.classification.BinaryConfusionMatrix.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the confusion tracker metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels where the values are |
required |
target
|
Tensor
|
The binary targets where the values are |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import BinaryConfusionMatrix
>>> metric = BinaryConfusionMatrix()
>>> metric(torch.tensor([0, 1, 0, 1]), torch.tensor([0, 1, 0, 1]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'false_negative_rate': 0.0,
'false_negative': 0,
'false_positive_rate': 0.0,
'false_positive': 0,
'jaccard_index': 1.0,
'count': 4,
'precision': 1.0,
'recall': 1.0,
'true_negative_rate': 1.0,
'true_negative': 2,
'true_positive_rate': 1.0,
'true_positive': 2,
'f1_score': 1.0}
karbonn.metric.classification.CategoricalConfusionMatrix ¶
Bases: BaseMetric
Implement a confusion tracker metric for multi-class labels.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
num_classes
|
int
|
The number of classes. |
required |
betas
|
Sequence[int | float]
|
The betas used to compute the f-beta scores. |
(1)
|
tracker
|
MulticlassConfusionMatrixTracker | None
|
The value tracker. If |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric
CategoricalConfusionMatrix(
(betas): (1,)
(tracker): MulticlassConfusionMatrixTracker(num_classes=3, count=0)
(track_count): True
)
>>> metric(
... prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
... target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
'balanced_accuracy': 0.333333...,
'count': 6,
'macro_precision': 0.555555...,
'macro_recall': 0.333333...,
'macro_f1_score': 0.388888...,
'micro_precision': 0.5,
'micro_recall': 0.5,
'micro_f1_score': 0.5,
'weighted_precision': 0.833333...,
'weighted_recall': 0.5,
'weighted_f1_score': 0.583333...,
'precision': tensor([0.6667, 0.0000, 1.0000]),
'recall': tensor([0.6667, 0.0000, 0.3333]),
'f1_score': tensor([0.6667, 0.0000, 0.5000])}
>>> metric(prediction=torch.tensor([1, 0]), target=torch.tensor([1, 0]))
>>> metric.value()
{'accuracy': 0.625,
'balanced_accuracy': 0.694444...,
'count': 8,
'macro_precision': 0.694444...,
'macro_recall': 0.694444...,
'macro_f1_score': 0.583333...,
'micro_precision': 0.625,
'micro_recall': 0.625,
'micro_f1_score': 0.625,
'weighted_precision': 0.791666...,
'weighted_recall': 0.625,
'weighted_f1_score': 0.625,
'precision': tensor([0.7500, 0.3333, 1.0000]),
'recall': tensor([0.7500, 1.0000, 0.3333]),
'f1_score': tensor([0.7500, 0.5000, 0.5000])}
>>> metric.reset()
>>> metric(prediction=torch.tensor([1, 0, 2]), target=torch.tensor([1, 0, 2]))
>>> metric.value()
{'accuracy': 1.0,
'balanced_accuracy': 1.0,
'count': 3,
'macro_precision': 1.0,
'macro_recall': 1.0,
'macro_f1_score': 1.0,
'micro_precision': 1.0,
'micro_recall': 1.0,
'micro_f1_score': 1.0,
'weighted_precision': 1.0,
'weighted_recall': 1.0,
'weighted_f1_score': 1.0,
'precision': tensor([1., 1., 1.]),
'recall': tensor([1., 1., 1.]),
'f1_score': tensor([1., 1., 1.])}
karbonn.metric.classification.CategoricalConfusionMatrix.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the confusion tracker metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels where the values are |
required |
target
|
Tensor
|
The binary targets where the values are |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalConfusionMatrix
>>> metric = CategoricalConfusionMatrix(num_classes=3)
>>> metric(
... prediction=torch.tensor([0, 1, 2, 0, 0, 1]),
... target=torch.tensor([2, 2, 2, 0, 0, 0]),
... )
>>> metric.value()
{'accuracy': 0.5,
'balanced_accuracy': 0.333333...,
'count': 6,
'macro_precision': 0.555555...,
'macro_recall': 0.333333...,
'macro_f1_score': 0.388888...,
'micro_precision': 0.5,
'micro_recall': 0.5,
'micro_f1_score': 0.5,
'weighted_precision': 0.833333...,
'weighted_recall': 0.5,
'weighted_f1_score': 0.583333...,
'precision': tensor([0.6667, 0.0000, 1.0000]),
'recall': tensor([0.6667, 0.0000, 0.3333]),
'f1_score': tensor([0.6667, 0.0000, 0.5000])}
karbonn.metric.classification.CategoricalCrossEntropy ¶
Bases: BaseStateMetric
Implement a metric to compute the categorical cross-entropy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric
CategoricalCrossEntropy(
(state): MeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 0.861983..., 'count': 6}
>>> metric.reset()
>>> metric(torch.ones(2, 3), torch.ones(2))
>>> metric.value()
{'mean': 1.098612..., 'count': 2}
karbonn.metric.classification.CategoricalCrossEntropy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels as a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import CategoricalCrossEntropy
>>> metric = CategoricalCrossEntropy()
>>> metric(torch.eye(4), torch.arange(4))
>>> metric.value()
{'mean': 0.743668..., 'count': 4}
karbonn.metric.classification.TopKAccuracy ¶
Bases: BaseMetric
Implement the accuracy at k metric a.k.a. top-k accuracy.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
topk
|
Sequence[int]
|
The k values used to evaluate the top-k accuracy metric. |
(1, 5)
|
Example usage:
>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric
TopKAccuracy(
(topk): (1,)
(states):
(1): AccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value()
{'top_1_accuracy': 0.75, 'top_1_count': 4}
>>> metric.reset()
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 2]))
>>> metric.value("acc_")
{'acc_top_1_accuracy': 0.5, 'acc_top_1_count': 2}
karbonn.metric.classification.TopKAccuracy.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the accuracy metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predicted labels as a |
required |
target
|
Tensor
|
The categorical targets. The values have to be in
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric import TopKAccuracy
>>> metric = TopKAccuracy(topk=(1,))
>>> metric(torch.tensor([[0.0, 2.0, 1.0], [2.0, 1.0, 0.0]]), torch.tensor([1, 0]))
>>> metric.value()
{'top_1_accuracy': 1.0, 'top_1_count': 2}
karbonn.metric.regression ¶
Contain the regression metrics.
karbonn.metric.regression.AbsoluteError ¶
Bases: BaseStateMetric
Implement the absolute error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric
AbsoluteError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_err_")
{'abs_err_mean': 0.5,
'abs_err_min': 0.0,
'abs_err_max': 1.0,
'abs_err_sum': 2.0,
'abs_err_count': 4}
karbonn.metric.regression.AbsoluteError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteError
>>> metric = AbsoluteError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.AbsoluteRelativeError ¶
Bases: BaseStateMetric
Implement the absolute relative error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
eps
|
float
|
An arbitrary small strictly positive number to avoid undefined results when the target is zero. |
1e-08
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric
AbsoluteRelativeError(
(eps): 1e-08
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.16666666666666666,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value(prefix="abs_rel_err_")
{'abs_rel_err_mean': 0.5,
'abs_rel_err_min': 0.0,
'abs_rel_err_max': 1.0,
'abs_rel_err_sum': 2.0,
'abs_rel_err_count': 4}
karbonn.metric.regression.AbsoluteRelativeError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute percentage error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import AbsoluteRelativeError
>>> metric = AbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.LogCoshError ¶
Bases: BaseStateMetric
Implement a metric to compute the logarithm of the hyperbolic cosine of the prediction error.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
scale
|
float
|
The scale factor. |
1.0
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric
LogCoshError(
(scale): 1.0
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.072296...,
'min': 0.0,
'max': 0.433780...,
'sum': 0.867561...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("log_cosh_err_")
{'log_cosh_err_mean': 0.216890...,
'log_cosh_err_min': 0.0,
'log_cosh_err_max': 0.433780...,
'log_cosh_err_sum': 0.867561...,
'log_cosh_err_count': 4}
karbonn.metric.regression.LogCoshError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import LogCoshError
>>> metric = LogCoshError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.NormalizedMeanSquaredError ¶
Bases: BaseStateMetric
Implement the normalized mean squared error (NMSE) metric.
Note: this metric does not work if all the targets are zero.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric
NormalizedMeanSquaredError(
(state): NormalizedMeanSquaredErrorState(
(squared_errors): MeanTensorTracker(count=0, total=0.0)
(squared_targets): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.5, 'count': 4}
karbonn.metric.regression.NormalizedMeanSquaredError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the normalized mean squared error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import NormalizedMeanSquaredError
>>> metric = NormalizedMeanSquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
karbonn.metric.regression.RootMeanSquaredError ¶
Bases: SquaredError
Implement the squared error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import RootMeanSquaredError
>>> metric = RootMeanSquaredError()
>>> metric
RootMeanSquaredError(
(state): RootMeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0, 'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.408248..., 'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.707106..., 'count': 4}
karbonn.metric.regression.SquaredAsinhError ¶
Bases: BaseStateMetric
Implement a metric to compute the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric
SquaredAsinhError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.129469...,
'min': 0.0,
'max': 0.776819...,
'sum': 1.553638...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_asinh_err_")
{'sq_asinh_err_mean': 0.388409...,
'sq_asinh_err_min': 0.0,
'sq_asinh_err_max': 0.776819...,
'sq_asinh_err_sum': 1.553638...,
'sq_asinh_err_count': 4}
karbonn.metric.regression.SquaredAsinhError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared error on the inverse hyperbolic sine (arcsinh) transformed predictions and targets given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredAsinhError
>>> metric = SquaredAsinhError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.SquaredError ¶
Bases: BaseStateMetric
Implement the squared error metric.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric
SquaredError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.166666...,
'min': 0.0,
'max': 1.0,
'sum': 2.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_err_")
{'sq_err_mean': 0.5,
'sq_err_min': 0.0,
'sq_err_max': 1.0,
'sq_err_sum': 2.0,
'sq_err_count': 4}
karbonn.metric.regression.SquaredError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared logarithmic error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredError
>>> metric = SquaredError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.SquaredLogError ¶
Bases: BaseStateMetric
Implement the squared logarithmic error (SLE) metric.
This metric is best to use when targets having exponential growth, such as population counts, average sales of a commodity over a span of years etc. Note that this metric penalizes an under-predicted estimate greater than an over-predicted estimate.
Note: this metric only works with positive value (0 included).
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric
SquaredLogError(
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.080075...,
'min': 0.0,
'max': 0.480453...,
'sum': 0.960906...,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sq_log_err_")
{'sq_log_err_mean': 0.240226...,
'sq_log_err_min': 0.0,
'sq_log_err_max': 0.480453...,
'sq_log_err_sum': 0.960906...,
'sq_log_err_count': 4}
karbonn.metric.regression.SquaredLogError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the squared logarithmic error metric given a mini- batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SquaredLogError
>>> metric = SquaredLogError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.regression.SymmetricAbsoluteRelativeError ¶
Bases: BaseStateMetric
Implement the symmetric absolute relative error (SARE) metric.
This metric tracks the mean, maximum and minimum absolute relative error values.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
eps
|
float
|
An arbitrary small strictly positive number to avoid undefined results when the target is zero. |
1e-08
|
state
|
BaseState | dict | None
|
The metric state or its configuration. If |
None
|
Example usage:
>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric
SymmetricAbsoluteRelativeError(
(eps): 1e-08
(state): ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
)
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value()
{'mean': 0.3333333333333333,
'min': 0.0,
'max': 2.0,
'sum': 4.0,
'count': 12}
>>> metric.reset()
>>> metric(torch.eye(2), torch.ones(2, 2))
>>> metric.value("sym_abs_rel_err_")
{'sym_abs_rel_err_mean': 1.0,
'sym_abs_rel_err_min': 0.0,
'sym_abs_rel_err_max': 2.0,
'sym_abs_rel_err_sum': 4.0,
'sym_abs_rel_err_count': 4}
karbonn.metric.regression.SymmetricAbsoluteRelativeError.forward ¶
forward(prediction: Tensor, target: Tensor) -> None
Update the mean absolute percentage error metric given a mini-batch of examples.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prediction
|
Tensor
|
The predictions as a |
required |
target
|
Tensor
|
The target tensor as a |
required |
Example usage:
>>> import torch
>>> from karbonn.metric import SymmetricAbsoluteRelativeError
>>> metric = SymmetricAbsoluteRelativeError()
>>> metric(torch.ones(2, 4), torch.ones(2, 4))
>>> metric.value()
{'mean': 0.0,
'min': 0.0,
'max': 0.0,
'sum': 0.0,
'count': 8}
karbonn.metric.state ¶
Contain the metric states.
karbonn.metric.state.AccuracyState ¶
Bases: BaseState
Implement a metric state to compute the accuracy.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracker
|
MeanTensorTracker | None
|
The mean value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import AccuracyState
>>> state = AccuracyState()
>>> state
AccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
>>> state.get_records()
(MaxScalarRecord(name=accuracy, max_size=10, size=0),)
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25, 'count': 16}
karbonn.metric.state.AccuracyState.update ¶
update(correct: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
correct
|
Tensor
|
A tensor that indicates the correct predictions.
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import AccuracyState
>>> state = AccuracyState()
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25, 'count': 16}
karbonn.metric.state.BaseState ¶
Bases: ABC
Define a base class to implement a metric state.
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
MinScalarRecord(name=error_min, max_size=10, size=0),
MinScalarRecord(name=error_max, max_size=10, size=0),
MinScalarRecord(name=error_sum, max_size=10, size=0))
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
'error_min': 0.0,
'error_max': 5.0,
'error_sum': 15.0,
'error_count': 6}
karbonn.metric.state.BaseState.count
abstractmethod
property
¶
count: int
The number of predictions in the state.
karbonn.metric.state.BaseState.clone ¶
clone() -> BaseState
Create a copy of the current state.
Returns:
Type | Description |
---|---|
BaseState
|
A copy of the current state. |
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state_cloned = state.clone()
>>> state.update(torch.ones(3))
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=9, total=18.0, min_value=0, max_value=5)
(track_count): True
)
>>> state_cloned
ErrorState(
(tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0.0, max_value=5.0)
(track_count): True
)
karbonn.metric.state.BaseState.equal
abstractmethod
¶
equal(other: Any) -> bool
Indicate if two metric states are equal or not.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
other
|
Any
|
The other metric state to compare. |
required |
Returns:
Type | Description |
---|---|
bool
|
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> from karbonn.utils.tracker import ScalableTensorTracker
>>> state = ErrorState()
>>> state.equal(ErrorState())
True
>>> state.equal(
... ErrorState(ScalableTensorTracker(count=4, total=10.0, min_value=0.0, max_value=5.0))
... )
False
karbonn.metric.state.BaseState.get_records
abstractmethod
¶
get_records(
prefix: str = "", suffix: str = ""
) -> tuple[BaseRecord, ...]
Get the records for the metrics associated to the current state.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prefix
|
str
|
The key prefix in the record names. |
''
|
suffix
|
str
|
The key suffix in the record names. |
''
|
Returns:
Name | Type | Description |
---|---|---|
tuple |
tuple[BaseRecord, ...]
|
The records. |
Example usage:
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
MinScalarRecord(name=error_min, max_size=10, size=0),
MinScalarRecord(name=error_max, max_size=10, size=0),
MinScalarRecord(name=error_sum, max_size=10, size=0))
karbonn.metric.state.BaseState.reset
abstractmethod
¶
reset() -> None
Reset the state.
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0, max_value=5)
(track_count): True
)
>>> state.reset()
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
karbonn.metric.state.BaseState.update
abstractmethod
¶
update(*args: Any, **kwargs: Any) -> None
Update the metric state.
The exact signature for this method depends on each metric state implementation.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
*args
|
Any
|
Variable length argument list. |
()
|
**kwargs
|
Any
|
Arbitrary keyword arguments. |
{}
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=6, total=15.0, min_value=0, max_value=5)
(track_count): True
)
karbonn.metric.state.BaseState.value
abstractmethod
¶
value(
prefix: str = "", suffix: str = ""
) -> dict[str, float]
Compute the metrics given the current state.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
prefix
|
str
|
The key prefix in the returned dictionary. |
''
|
suffix
|
str
|
The key suffix in thenreturned dictionary. |
''
|
Returns:
Type | Description |
---|---|
dict[str, float]
|
The metric values. |
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
'error_min': 0.0,
'error_max': 5.0,
'error_sum': 15.0,
'error_count': 6}
karbonn.metric.state.ErrorState ¶
Bases: BaseState
Implement a metric state to capture some metrics about the errors.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracker
|
ScalableTensorTracker | None
|
The value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
MinScalarRecord(name=error_min, max_size=10, size=0),
MinScalarRecord(name=error_max, max_size=10, size=0),
MinScalarRecord(name=error_sum, max_size=10, size=0))
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
'error_min': 0.0,
'error_max': 5.0,
'error_sum': 15.0,
'error_count': 6}
karbonn.metric.state.ErrorState.update ¶
update(error: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
error
|
Tensor
|
A tensor of errors. |
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import ErrorState
>>> state = ErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5,
'error_min': 0.0,
'error_max': 5.0,
'error_sum': 15.0,
'error_count': 6}
karbonn.metric.state.ExtendedAccuracyState ¶
Bases: BaseState
Implement a metric state to compute the accuracy and other metrics.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracker
|
MeanTensorTracker | None
|
The mean value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import ExtendedAccuracyState
>>> state = ExtendedAccuracyState()
>>> state
ExtendedAccuracyState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
>>> state.get_records()
(MaxScalarRecord(name=accuracy, max_size=10, size=0),
MinScalarRecord(name=error, max_size=10, size=0),
MaxScalarRecord(name=count_correct, max_size=10, size=0),
MinScalarRecord(name=count_incorrect, max_size=10, size=0))
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25,
'error': 0.75,
'count_correct': 4,
'count_incorrect': 12,
'count': 16}
karbonn.metric.state.ExtendedAccuracyState.update ¶
update(correct: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
correct
|
Tensor
|
A tensor that indicates the correct predictions.
|
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import ExtendedAccuracyState
>>> state = ExtendedAccuracyState()
>>> state.update(torch.eye(4))
>>> state.value()
{'accuracy': 0.25,
'error': 0.75,
'count_correct': 4,
'count_incorrect': 12,
'count': 16}
karbonn.metric.state.ExtendedErrorState ¶
Bases: BaseState
Implement a metric state to capture some metrics about the errors.
This state stores all the error values, so it does not scale to large datasets. This state has a linear space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
quantiles
|
Tensor | Sequence[float]
|
The quantile values to evaluate. |
()
|
tracker
|
TensorTracker | None
|
The value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import ExtendedErrorState
>>> state = ExtendedErrorState(quantiles=[0.5, 0.9])
>>> state
ExtendedErrorState(
(quantiles): tensor([0.5000, 0.9000])
(tracker): TensorTracker(count=0)
(track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),
MinScalarRecord(name=error_median, max_size=10, size=0),
MinScalarRecord(name=error_min, max_size=10, size=0),
MinScalarRecord(name=error_max, max_size=10, size=0),
MinScalarRecord(name=error_sum, max_size=10, size=0),
MinScalarRecord(name=error_quantile_0.5, max_size=10, size=0),
MinScalarRecord(name=error_quantile_0.9, max_size=10, size=0))
>>> state.update(torch.arange(11))
>>> state.value("error_")
{'error_mean': 5.0,
'error_median': 5,
'error_min': 0,
'error_max': 10,
'error_sum': 55,
'error_std': 3.316...,
'error_quantile_0.5': 5.0,
'error_quantile_0.9': 9.0,
'error_count': 11}
karbonn.metric.state.ExtendedErrorState.update ¶
update(error: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
error
|
Tensor
|
A tensor of errors. |
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import ExtendedErrorState
>>> state = ExtendedErrorState(quantiles=[0.5, 0.9])
>>> state.update(torch.arange(11))
>>> state.value("error_")
{'error_mean': 5.0,
'error_median': 5,
'error_min': 0,
'error_max': 10,
'error_sum': 55,
'error_std': 3.316...,
'error_quantile_0.5': 5.0,
'error_quantile_0.9': 9.0,
'error_count': 11}
karbonn.metric.state.MeanErrorState ¶
Bases: BaseState
Implement a metric state to capture the mean error value.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracker
|
MeanTensorTracker | None
|
The mean value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import MeanErrorState
>>> state = MeanErrorState()
>>> state
MeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5, 'error_count': 6}
karbonn.metric.state.MeanErrorState.update ¶
update(error: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
error
|
Tensor
|
A tensor of errors. |
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import MeanErrorState
>>> state = MeanErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 2.5, 'error_count': 6}
karbonn.metric.state.NormalizedMeanSquaredErrorState ¶
Bases: BaseState
Implement a metric state to capture the normalized mean squared error value.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
squared_errors
|
MeanTensorTracker | None
|
The value tracker for squared errors. |
None
|
squared_targets
|
MeanTensorTracker | None
|
The value tracker for squared targets. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import NormalizedMeanSquaredErrorState
>>> state = NormalizedMeanSquaredErrorState()
>>> state
NormalizedMeanSquaredErrorState(
(squared_errors): MeanTensorTracker(count=0, total=0.0)
(squared_targets): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
>>> state.get_records("nmse_")
(MinScalarRecord(name=nmse_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6), torch.ones(6))
>>> state.value("nmse_")
{'nmse_mean': 9.166..., 'nmse_count': 6}
karbonn.metric.state.NormalizedMeanSquaredErrorState.update ¶
update(error: Tensor, target: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
error
|
Tensor
|
A tensor of errors. |
required |
target
|
Tensor
|
A tensor with the target values. |
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import NormalizedMeanSquaredErrorState
>>> state = NormalizedMeanSquaredErrorState()
>>> state.update(torch.arange(6), torch.ones(6))
>>> state.value("nmse_")
{'nmse_mean': 9.166..., 'nmse_count': 6}
karbonn.metric.state.RootMeanErrorState ¶
Bases: BaseState
Implement a metric state to capture the root mean error value.
This state has a constant space complexity.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tracker
|
MeanTensorTracker | None
|
The mean value tracker. |
None
|
track_count
|
bool
|
If |
True
|
Example usage:
>>> import torch
>>> from karbonn.metric.state import RootMeanErrorState
>>> state = RootMeanErrorState()
>>> state
RootMeanErrorState(
(tracker): MeanTensorTracker(count=0, total=0.0)
(track_count): True
)
>>> state.get_records("error_")
(MinScalarRecord(name=error_mean, max_size=10, size=0),)
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 1.581..., 'error_count': 6}
karbonn.metric.state.RootMeanErrorState.update ¶
update(error: Tensor) -> None
Update the metric state with a new tensor of errors.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
error
|
Tensor
|
A tensor of errors. |
required |
Example usage:
>>> import torch
>>> from karbonn.metric.state import RootMeanErrorState
>>> state = RootMeanErrorState()
>>> state.update(torch.arange(6))
>>> state.value("error_")
{'error_mean': 1.581..., 'error_count': 6}
karbonn.metric.state.is_state_config ¶
is_state_config(config: dict) -> bool
Indicate if the input configuration is a configuration for a
BaseState
.
This function only checks if the value of the key _target_
is valid. It does not check the other values. If _target_
indicates a function, the returned type hint is used to check
the class.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
config
|
dict
|
The configuration to check. |
required |
Returns:
Type | Description |
---|---|
bool
|
|
Example usage:
>>> from karbonn.metric.state import is_state_config
>>> is_state_config({"_target_": "karbonn.metric.state.ErrorState"})
True
karbonn.metric.state.setup_state ¶
Set up a BaseState
object.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
state
|
BaseState | dict
|
The state or its configuration. |
required |
Returns:
Type | Description |
---|---|
BaseState
|
The instantiated |
Example usage:
>>> from karbonn.metric.state import setup_state
>>> state = setup_state({"_target_": "karbonn.metric.state.ErrorState"})
>>> state
ErrorState(
(tracker): ScalableTensorTracker(count=0, total=0.0, min_value=inf, max_value=-inf)
(track_count): True
)