What metrics evaluate document extraction accuracy?medium

Type
conceptual
Topic
metrics-do-you-use-to-evaluate-extraction-accuracy-in-inte
Frequency
common
Tags
mlops, what, metrics, you, use, evaluate
Answer

Field-level exact match (does extracted value match ground truth?), partial match / token F1 for variable-format fields like dates, overall document accuracy (all fields correct), null rate (fields model failed to extrac

Explanation

Field-level exact match (does extracted value match ground truth?), partial match / token F1 for variable-format fields like dates, overall document accuracy (all fields correct), null rate (fields model failed to extract), and false positive rate (model extracted something for an absent field). Ground truth: manually annotated 200 representative documents by a domain expert.

Follow-upHow would you monitor this in production?