Extraction review
How it works
- The extraction pipeline flags a fact it cannot trust, for example: the quoted passage was not found in the document, confidence is low, two models disagreed, a page was image-only, or a value regressed.
- The flag becomes a review item. Its flag class names who owns it, the plain question the reviewer is asked, and the actions the reviewer may take.
- The reviewer opens the document with the cited passage highlighted, answers the question with one of the allowed actions, and records the disposition under their name.
- The platform applies the action (table below) and, where the fact can be addressed, records a verification or a correction.
human (a person can settle it), by the machine (the pipeline could not read the value, so only a re-extraction settles it), or is informational (an audit record of a disagreement the pipeline already resolved; it goes to no queue). The allowed actions are accept_value, enter_value, map_lob (map a label printed in the document to a code in the document profile’s vocabulary), not_stated, not_applicable, reject, and request_reextraction. For an item with a registered flag class, the server refuses any action the class does not allow.
Review item state model
Transitions:
pending to approved, corrected, rejected, reextraction_requested or superseded; approved, corrected or rejected back to pending by reopening.
What each action changes for downstream readers
Which fields can be corrected, and which of the three paths above each one takes, is fixed in code for each document profile. The original extracted value is kept alongside a correction. Corrections are stored in their own ledger and applied as an overlay each time records are projected, so a rebuild keeps them.
A verification is bound to its evidence: the value, the quoted passage, the page, the document’s content hash and the extractor’s schema version. If a later extraction changes the value or the evidence, or drops the fact, the verification is demoted. A reviewer can also dispute a verified fact, which demotes it and reopens its resolved item in one step.
Verification tiers
Every delivered record row carries the strength of confirmation behind it. The tier says how much stands behind a figure, never that the figure is right.
A person’s verification outranks a model agreement at the same fact.
Re-extraction spend cap
Re-extraction calls a model, so it costs money. Requests are capped per connection per day. A request over the cap is refused until spend inside the window falls; nothing is queued silently. An accepted request reports its projected cost.Review sessions and sampling
A review session is one reviewer’s sitting over one connection. It has one of three modes:
A
sample or custom session draws 1 to 2,000 documents; a flagged session walks every document with pending items. The draw happens once when the session is created, so a half-finished session is never reshuffled. Sessions are workflow only: every choice inside one writes its own disposition and verification.
Weekly blind sample
For every connection where Nexio has armed delivery, the platform draws a random sample of served records each week and places them in the review queue under theblind_sample flag class. The draw is blind: a record’s confidence, flags and dollar value play no part in which records are chosen, so the sample measures accuracy instead of confirming what the pipeline already doubted. The draw is due Monday at 06:30 UTC. The share of sampled records that pass is the measured accuracy reported with every delivery; until a sample has been fully reviewed, a delivery says accuracy is not yet measured. See Transformation and delivery.
Run review
Run review is a portal surface for people in your organization. It is available today.Roles
See Team and roles for how roles are assigned.
How it works
- A moderator sends a run to review and assigns it to a reviewer.
- The reviewer opens Review in the sidebar. The page has four tabs: Inbox, Snoozed, Done and All.
- The reviewer reads the run, adds annotations, and finishes the item with a disposition, or snoozes it.
- Annotations are stored on the run. They do not change served data.
Review item state model
A reviewed item cannot be snoozed. Finishing or snoozing an item that is reviewed, or snoozed with time still to run, is refused. A reviewer who is not a moderator can act only on items assigned to them.
Outcomes and annotations
Record annotations on runs through the API.
Evaluation
How annotations become evaluation sets.