veil.evaluator

Classes

ConfusionCounts([tp, fp, fn])
Evaluator(config)

class veil.evaluator.ConfusionCounts(tp: 'int' = 0, fp: 'int' = 0, fn: 'int' = 0)

Bases: object

Parameters:

• tp (int)

• fp (int)

• fn (int)

fn: int = 0

fp: int = 0

to_dict()

Return type:

Dict[str, int]

tp: int = 0

class veil.evaluator.Evaluator(config)

Bases: object

Parameters:

config (EvaluatorConfig)

evaluate_document(*, document, mask_result, component_spans, metric_store, component_supported_types=None)

Compute confusion matrices for a single document and send to metric store.

Returns only variant-based evaluations under “variants”:

• “exact” (if enabled)

• “iou@THRESH” for each threshold in config.report_iou_thresholds

Parameters:

• document (Document)

• mask_result (MaskResult)

• component_spans (Dict[str, List[Span]] | None)

• metric_store (Optional['MetricStore'])

• component_supported_types (Optional[Dict[str, List[str]]])

Return type:

Optional[Dict[str, Any]]