arkas.reporter

arkas.reporter

Contain reporters.

arkas.reporter.BaseReporter

Bases: ABC

Define the base class to generate a HTML report.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> from grizz.ingestor import Ingestor
>>> from grizz.transformer import SequentialTransformer
>>> from arkas.reporter import EvalReporter
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     reporter = EvalReporter(
...         ingestor=Ingestor(
...             pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
...         ),
...         transformer=SequentialTransformer(transformers=[]),
...         evaluator=AccuracyEvaluator(y_true="target", y_pred="pred"),
...         report_path=Path(tmpdir).joinpath("report.html"),
...     )
...     reporter.generate()
...

arkas.reporter.BaseReporter.generate abstractmethod

generate() -> None

Generate a HTML report.

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> from grizz.ingestor import Ingestor
>>> from grizz.transformer import SequentialTransformer
>>> from arkas.reporter import EvalReporter
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     reporter = EvalReporter(
...         ingestor=Ingestor(
...             pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
...         ),
...         transformer=SequentialTransformer(transformers=[]),
...         evaluator=AccuracyEvaluator(y_true="target", y_pred="pred"),
...         report_path=Path(tmpdir).joinpath("report.html"),
...     )
...     reporter.generate()
...

arkas.reporter.EvalReporter

Bases: BaseReporter

Implement a simple reporter that evaluates results and writes them in a HTML file.

Parameters:

Name	Type	Description	Default
ingestor	BaseIngestor | dict	The ingestor or its configuration.	required
transformer	BaseTransformer | dict	The data transformer or its configuration.	required
evaluator	BaseEvaluator | dict	The evaluator or its configuration.	required
report_path	Path | str	The path where to save the HTML report.	required
max_toc_depth	int	The maximum level to show in the table of content.	6

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> import polars as pl
>>> from arkas.evaluator import AccuracyEvaluator
>>> from grizz.ingestor import Ingestor
>>> from grizz.transformer import SequentialTransformer
>>> from arkas.reporter import EvalReporter
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     reporter = EvalReporter(
...         ingestor=Ingestor(
...             pl.DataFrame({"pred": [3, 2, 0, 1, 0, 1], "target": [3, 2, 0, 1, 0, 1]})
...         ),
...         transformer=SequentialTransformer(transformers=[]),
...         evaluator=AccuracyEvaluator(y_true="target", y_pred="pred"),
...         report_path=Path(tmpdir).joinpath("report.html"),
...     )
...     reporter.generate()
...

arkas.reporter.Reporter

Bases: BaseReporter

Implement a simple reporter that generates a HTML file and save it.

Parameters:

Name	Type	Description	Default
report_path	Path | str	The path where to save the HTML report.	required
max_toc_depth	int	The maximum level to show in the table of content.	6

Example usage:

>>> import tempfile
>>> from pathlib import Path
>>> import numpy as np
>>> from arkas.content import AccuracyContentGenerator
>>> from arkas.evaluator2 import AccuracyEvaluator
>>> from arkas.state import AccuracyState
>>> from arkas.reporter import Reporter
>>> generator = AccuracyContentGenerator(
...     AccuracyEvaluator(
...         AccuracyState(
...             y_true=np.array([1, 0, 0, 1, 1]),
...             y_pred=np.array([1, 0, 0, 1, 1]),
...             y_true_name="target",
...             y_pred_name="pred",
...         )
...     )
... )
>>> with tempfile.TemporaryDirectory() as tmpdir:
...     reporter = Reporter(
...         report_path=Path(tmpdir).joinpath("report.html"),
...     )
...     reporter.generate(generator)
...

arkas.reporter.is_reporter_config

is_reporter_config(config: dict) -> bool

Indicate if the input configuration is a configuration for a BaseReporter.

This function only checks if the value of the key _target_ is valid. It does not check the other values. If _target_ indicates a function, the returned type hint is used to check the class.

Parameters:

Name	Type	Description	Default
config	dict	The configuration to check.	required

Returns:

Name	Type	Description
bool	bool	True if the input configuration is a configuration for a BaseReporter object.

Example usage:

>>> from arkas.reporter import is_reporter_config
>>> is_reporter_config(
...     {
...         "_target_": "arkas.reporter.EvalReporter",
...         "ingestor": {
...             "_target_": "grizz.ingestor.CsvIngestor",
...             "path": "/path/to/data.csv",
...         },
...         "transformer": {"_target_": "grizz.transformer.DropDuplicate"},
...         "evaluator": {
...             "_target_": "arkas.evaluator.AccuracyEvaluator",
...             "y_true": "target",
...             "y_pred": "pred",
...         },
...         "report_path": "/path/to/report.html",
...     }
... )
True

arkas.reporter.setup_reporter

setup_reporter(
    reporter: BaseReporter | dict,
) -> BaseReporter

Set up a reporter.

The reporter is instantiated from its configuration by using the BaseReporter factory function.

Parameters:

Name	Type	Description	Default
reporter	BaseReporter | dict	A reporter or its configuration.	required

Returns:

Type	Description
BaseReporter	An instantiated reporter.

Example usage:

>>> from arkas.reporter import setup_reporter
>>> reporter = setup_reporter(
...     {
...         "_target_": "arkas.reporter.EvalReporter",
...         "ingestor": {
...             "_target_": "grizz.ingestor.CsvIngestor",
...             "path": "/path/to/data.csv",
...         },
...         "transformer": {"_target_": "grizz.transformer.DropDuplicate"},
...         "evaluator": {
...             "_target_": "arkas.evaluator.AccuracyEvaluator",
...             "y_true": "target",
...             "y_pred": "pred",
...         },
...         "report_path": "/path/to/report.html",
...     }
... )
>>> reporter
EvalReporter(
  (ingestor): CsvIngestor(path=/path/to/data.csv)
  (transformer): DropDuplicateTransformer(columns=None, exclude_columns=(), missing_policy='raise')
  (evaluator): AccuracyEvaluator(y_true='target', y_pred='pred', drop_nulls=True, nan_policy='propagate')
  (report_path): /path/to/report.html
  (max_toc_depth): 6
)