curl --request GET \
--url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff"
headers = {"Authorization": "Bearer <token>"}
response = requests.get(url, headers=headers)
print(response.text)const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"baseline_run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"diff": {
"newly_failing": [
"<string>"
],
"newly_passing": [
"<string>"
],
"still_failing": [
"<string>"
],
"still_passing": [
"<string>"
],
"added": [
"<string>"
],
"removed": [
"<string>"
]
}
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "unauthorized",
"message": "Missing or invalid API key"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "rate_limited",
"message": "Rate limit exceeded"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "auth_unavailable",
"message": "API key authentication is temporarily unavailable"
}{
"code": "request_timeout",
"message": "Request timed out"
}Diff eval runs
Compare an eval run with a baseline run or the previous completed run of its suite.
curl --request GET \
--url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff \
--header 'Authorization: Bearer <token>'import requests
url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff"
headers = {"Authorization": "Bearer <token>"}
response = requests.get(url, headers=headers)
print(response.text)const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};
fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-runs/{run_id}/diff', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"baseline_run_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"diff": {
"newly_failing": [
"<string>"
],
"newly_passing": [
"<string>"
],
"still_failing": [
"<string>"
],
"still_passing": [
"<string>"
],
"added": [
"<string>"
],
"removed": [
"<string>"
]
}
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "unauthorized",
"message": "Missing or invalid API key"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "rate_limited",
"message": "Rate limit exceeded"
}{
"code": "<string>",
"message": "<string>",
"details": "<unknown>"
}{
"code": "auth_unavailable",
"message": "API key authentication is temporarily unavailable"
}{
"code": "request_timeout",
"message": "Request timed out"
}Authorizations
Send the key as Authorization: Bearer <key>. Two kinds of key exist.
Organization keys are issued in the portal (Settings, then API keys),
each bound to one environment, shaped nx_<environment slug>_<64 hex>. They carry
no capabilities and pass every capability check, with one exception:
routes under /api/v1/records, /api/v1/engines/{id}/opportunities,
/api/v1/graph and /api/v1/catalog/documents accept an
organization key only when its environment is live, and refuse any
other with 403 scoped_key_required. Revocation takes effect within 60
seconds.
Scoped keys are issued by Nexio on request, shaped
nxsk_v1_<24 hex key id>_<43 character secret>. Each is bound to one
org, one environment, a set of engines and a set of capabilities. A
malformed, unknown or revoked nxsk_ key fails with 401 and is never
retried as an organization key. Revocation takes effect on the next
request. A scoped key without a route's capability gets 403
insufficient_capability; a scoped key not bound to the engine gets 403
engine_binding_forbidden.
Key-grantable capabilities: engines:read, runs:write, runs:read,
runs:defensibility:read, runs:test, catalog:read,
catalog:documents:read, webhooks:manage, conversations:use,
conversations:export, records:read, records:opportunities:run,
actions:write, actions:read, graph:read, records:analyze.
Routes that accept organization keys only (every scoped key gets 403
insufficient_capability): environment management, engine create,
update, configuration and publish, and conversation instance
authoring. Each operation description names the capability a scoped
key needs.
Path Parameters
Conversation instance identifier slug (e.g. platform-assistant).
Query Parameters
The run to compare against. Omit to use the previous completed run of the same suite.
Response
The diff.
A run compared with a baseline run, by scenario, reported as scenario names.
Hide child attributes
Hide child attributes
Passed in the baseline, fail now.
Failed in the baseline, pass now.
In this run, not in the baseline.
In the baseline, not in this run.