Derivations
A derivation is one declared table the platform produces from sources it reads: your warehouse, your document library, reference data, or an external provider. Each derivation declares the checks its output must pass, such as a minimum group size so no row describes too few people or accounts. Nexio arms each derivation per connection with a schedule, and its output reaches you through the reads and deliveries built on it. A derivation run’s result is published as async.run_completed platform event with run_kind derivation:
Managed delivery
Managed delivery writes your served document data, with its evidence and verification labels, to storage you own. It is available today. The destination is your storage, at the root path and table prefix you choose; Nexio runs the delivery job and writes there, you read, and nothing is pulled from your side. Destinations are a fixed set of supported types. Today the supported destination type is Azure Blob Storage, written over SFTP.How it works
- One snapshot. Every table in a delivery is read inside one read-only, serializable database transaction. The transaction’s timestamp is the delivery’s watermark, stamped on every row as
SNAPSHOT_AT. A join across delivered tables cannot reach a row that did not exist when another was read. - Release check. Before any file is written, the delivery evaluates its release rules against the snapshot. A failed rule refuses the whole delivery: nothing is written and your storage keeps pointing at the previous delivery. The one exception is the verification rule, which labels every row with its verification tier instead of refusing.
- Hold open items. In the record families a delivery holds (fixed in code for each document profile), a record with an open review item (a pending review at the record, or an unresolved disagreement between two model readings anywhere in its source document) is left out and listed in the exclusion list with the reason.
- Write each file atomically. Each file is uploaded under a temporary name and renamed only after all its bytes are sent. You never see a partial file under a delivered name.
- Move the pointer.
deliveries/LATESTis rewritten only after every file has been read back and its size and SHA-256 match what was sent. A delivery that fails before that point leavesLATESTnaming the previous delivery; a partial directory it wrote is left in place under an id nothing points at. - Copy the documents. After
LATESTmoves, every document in the connection’s document library is copied underdocuments/, named by its content hash. A failure here does not undo the tables:LATESTalready names the new delivery, and a document it cites can be missing until the next run copies it.
What is written to your storage
A delivery contains the data tables built from the served records of the connection’s document profile, a document catalog table, and the files below. The set of data tables is fixed in code for each document profile.- The delivery id is the watermark to the minute (
YYYY-MM-DDTHHMMZ), so a directory listing is in time order. <PREFIX>is the table-name prefix you chose for your organization (see What you provide). It appears in file names, and in the table, stage and file format names the load kit creates.- Each data table is written as CSV and Parquet from one column declaration, so the two formats and the load kit’s table definitions cannot disagree.
LOAD_KIT/holds numbered SQL scripts: session setup, file format and stage, control tables, one script per table, and a final verification script.
Column groups
Data tables carry these groups in this order, with the table’s own columns between Identity and Evidence. A table whose records belong to a counterparty carries a counterparty group between Identity and its own columns. The document catalog table, and any table its document profile defines as catalog-shaped, carry only the identity group, their own columns, andVERSION, SNAPSHOT_AT, DELIVERY_ID and DELIVERY_SCOPE.
CONFIDENCE is a reading confidence from 0 to 1, not a probability that the value is correct. An empty value is never a zero.
Versioning
Rows are versions.VERSION starts at 1 and increases each time a record is re-projected. An empty EFFECTIVE_TO means the row is current. The load kit’s load step closes the prior version (sets EFFECTIVE_TO and SUPERSEDED_BY_VERSION) when it loads a newer one. A row of a held family whose source document was removed from your library is delivered already closed, with EFFECTIVE_TO set to that delivery’s SNAPSHOT_AT.
The verification column
VERIFICATION says how much confirmation stands behind the row’s critical figure, never that the figure is right.
Tables with no per-field verification deliver
single. See Review for how each tier is earned.
The exclusion list
The exclusion list names documents whose records are not delivered as current rows, with a status and a plain reason. It is read in the same snapshot as the tables.
A document is not listed because of the verification tier of its rows: the
VERIFICATION column is the statement about them.
The delivery record
_summary.json is the machine-readable record of one delivery:
To confirm a delivery is complete: read
LATEST, open that directory’s _summary.json, and check each file’s row count against what you loaded. The load kit’s verification script does this in Snowflake.
Schedule
Delivery runs at 06:00 UTC for each connection where Nexio has armed it: daily by default, or every few whole days. Scheduling is off by default and cannot be turned on until one manual delivery to your storage has completed cleanly.What you provide
- A storage container with an SFTP endpoint and a login Nexio writes with.
- Where Nexio writes (the root path) and the table prefix you want.
Review
How verification tiers and measured accuracy are earned.
Served data
Read the same records through the API.