How should 98% extraction accuracy be measured?medium
Answer
Measured on a held-out test set of ~100 documents not seen during prompt development.
Explanation
Measured on a held-out test set of ~100 documents not seen during prompt development. Ground truth annotated by a domain expert. Metric: field-level exact match after normalizing dates and numeric formats. 98% = 98% of extracted fields matched ground truth across all documents and fields. Errors were mostly edge cases — unusual swap structures or corrupted PDFs.
Follow-upHow would you monitor this in production?