curl --request GET \
--url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run"
headers = {"Authorization": "Bearer <token>"}
response = requests.get(url, headers=headers)
print(response.text)const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"version": "<string>",
"config_version_hash": "<string>",
"run": {
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"status": "running",
"pass_count": 123,
"fail_count": 123,
"created_at": "2023-11-07T05:31:56Z",
"waived_by": "<string>",
"waive_reason": "<string>"
},
"results": [
{
"scenario_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"scenario_name": "<string>",
"passed": true,
"scores": {}
}
],
"diff": {
"prior_version": "<string>",
"newly_failing": [
"<string>"
],
"new_failing": [
"<string>"
],
"newly_passing": [
"<string>"
]
}
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "unauthorized",
"message": "Missing or invalid API key"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "rate_limited",
"message": "Rate limit exceeded"
}{
"code": "internal_error",
"message": "Internal server error"
}{
"code": "auth_unavailable",
"message": "API key authentication is temporarily unavailable"
}{
"code": "request_timeout",
"message": "Request timed out"
}Get a version's eval run
Read the latest eval run recorded for a version, from its publish gate or a later on-demand run, with its diff against the previous version.
curl --request GET \
--url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run"
headers = {"Authorization": "Bearer <token>"}
response = requests.get(url, headers=headers)
print(response.text)const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/versions/{version}/eval-run', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"version": "<string>",
"config_version_hash": "<string>",
"run": {
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"status": "running",
"pass_count": 123,
"fail_count": 123,
"created_at": "2023-11-07T05:31:56Z",
"waived_by": "<string>",
"waive_reason": "<string>"
},
"results": [
{
"scenario_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"scenario_name": "<string>",
"passed": true,
"scores": {}
}
],
"diff": {
"prior_version": "<string>",
"newly_failing": [
"<string>"
],
"new_failing": [
"<string>"
],
"newly_passing": [
"<string>"
]
}
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "unauthorized",
"message": "Missing or invalid API key"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "rate_limited",
"message": "Rate limit exceeded"
}{
"code": "internal_error",
"message": "Internal server error"
}{
"code": "auth_unavailable",
"message": "API key authentication is temporarily unavailable"
}{
"code": "request_timeout",
"message": "Request timed out"
}Authorizations
Send the key as Authorization: Bearer <key>. Two kinds of key exist.
Organization keys are issued in the portal (Settings, then API keys),
each bound to one environment, shaped nx_<environment slug>_<64 hex>. They carry
no capabilities and pass every capability check, with one exception:
routes under /api/v1/records, /api/v1/engines/{id}/opportunities,
/api/v1/graph and /api/v1/catalog/documents accept an
organization key only when its environment is live, and refuse any
other with 403 scoped_key_required. Revocation takes effect within 60
seconds.
Scoped keys are issued by Nexio on request, shaped
nxsk_v1_<24 hex key id>_<43 character secret>. Each is bound to one
org, one environment, a set of engines and a set of capabilities. A
malformed, unknown or revoked nxsk_ key fails with 401 and is never
retried as an organization key. Revocation takes effect on the next
request. A scoped key without a route's capability gets 403
insufficient_capability; a scoped key not bound to the engine gets 403
engine_binding_forbidden.
Key-grantable capabilities: engines:read, runs:write, runs:read,
runs:defensibility:read, runs:test, catalog:read,
catalog:documents:read, webhooks:manage, conversations:use,
conversations:export, records:read, records:opportunities:run,
actions:write, actions:read, graph:read, records:analyze.
Routes that accept organization keys only (every scoped key gets 403
insufficient_capability): environment management, engine create,
update, configuration and publish, and conversation instance
authoring. Each operation description names the capability a scoped
key needs.
Path Parameters
Conversation instance identifier slug (e.g. platform-assistant).
Released instance version (a positive integer, e.g. 3).
x >= 1Response
The version's recorded eval run.
A released version's recorded eval run with per-scenario verdicts and the diff vs the prior version's run.
The released version as an integer string, for example "4".
The archived config version the run measured.
Hide child attributes
Hide child attributes
The run's status. The latest run associated with the version
can be the publish gate run or a later on-demand run, so an
on-demand run still executing reads running and one that
broke reads error. waived means a blocking regression was
explicitly waived.
running, passed, failed, waived, error Who waived a blocking regression; omitted when not waived.
Why the regression was waived; omitted when not waived.
Omitted when the version has no prior version with a recorded run (baseline).
Hide child attributes
Hide child attributes
The previous released version as an integer string.
Regressions; scenario names that passed on the prior version's run and fail on this one.
Scenario names with no prior result that fail on this version (they block like regressions).