Skip to main content
POST
Start an on-demand eval run

Behavior

The run is queued and executes in the background; the response is 201 with status running. Poll Get an eval run until the status is passed, failed, or error. The state model is on Annotations and evaluation.

Authorizations

Authorization
string
header
required

Send the key as Authorization: Bearer <key>. Two kinds of key exist.

Organization keys are issued in the portal (Settings, then API keys), each bound to one environment, shaped nx_<environment slug>_<64 hex>. They carry no capabilities and pass every capability check, with one exception: routes under /api/v1/records, /api/v1/engines/{id}/opportunities, /api/v1/graph and /api/v1/catalog/documents accept an organization key only when its environment is live, and refuse any other with 403 scoped_key_required. Revocation takes effect within 60 seconds.

Scoped keys are issued by Nexio on request, shaped nxsk_v1_<24 hex key id>_<43 character secret>. Each is bound to one org, one environment, a set of engines and a set of capabilities. A malformed, unknown or revoked nxsk_ key fails with 401 and is never retried as an organization key. Revocation takes effect on the next request. A scoped key without a route's capability gets 403 insufficient_capability; a scoped key not bound to the engine gets 403 engine_binding_forbidden.

Key-grantable capabilities: engines:read, runs:write, runs:read, runs:defensibility:read, runs:test, catalog:read, catalog:documents:read, webhooks:manage, conversations:use, conversations:export, records:read, records:opportunities:run, actions:write, actions:read, graph:read, records:analyze.

Routes that accept organization keys only (every scoped key gets 403 insufficient_capability): environment management, engine create, update, configuration and publish, and conversation instance authoring. Each operation description names the capability a scoped key needs.

Path Parameters

instance_slug
string
required

Conversation instance identifier slug (e.g. platform-assistant).

Body

application/json
suite
enum<string>
required
Available options:
gate,
workflows,
adversarial,
smoke

Response

The queued run.

One eval run, from a publish gate or started on demand.

id
string<uuid>
required
suite
enum<string>
required
Available options:
gate,
workflows,
adversarial,
smoke
status
enum<string>
required

running while queued or executing. On an on-demand run, passed when every scenario passed and failed when at least one failed. On a publish gate run, passed means no regression against the prior version's run (a first gate run, or failures that were already failing, still pass; read fail_count), and failed means a scenario newly fails or a new scenario fails. waived for a publish gate run whose regression was waived; error when the run cannot complete: at once for a permanent failure (the instance, its archived configuration, a stored scenario, or the eval executor is unavailable or unreadable), or after 5 attempts when execution failed transiently each time.

Available options:
running,
passed,
failed,
waived,
error
triggered_by
enum<string>
required
Available options:
publish,
manual,
schedule,
event
pass_count
integer
required
fail_count
integer
required
created_at
string<date-time>
required
version
string

The released instance version the run measured, as an integer string (for example "4"). Absent when not recorded.

error
string

Why the run ended in error. Absent otherwise.

Last modified on September 25, 2026