Update an eval scenario

Correct a Conversation eval scenario

curl --request PATCH \
  --url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id} \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "<string>",
  "script": [
    {
      "message": "<string>",
      "tool_results": {},
      "confirmations": {}
    }
  ],
  "rubric": {
    "must_refuse": [
      "<string>"
    ],
    "must_escalate": [
      "<string>"
    ],
    "must_call_tools": [
      {
        "name": "<string>",
        "input_contains": "<string>"
      }
    ],
    "must_emit_components": [
      {
        "component": "<string>"
      }
    ],
    "must_cite": [
      "<string>"
    ],
    "must_not_refuse": true,
    "final_must_not_contain": [
      "<string>"
    ],
    "judge": [
      {
        "name": "<string>",
        "criteria": "<string>"
      }
    ]
  }
}
'

import requests

url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id}"

payload = {
    "name": "<string>",
    "script": [
        {
            "message": "<string>",
            "tool_results": {},
            "confirmations": {}
        }
    ],
    "rubric": {
        "must_refuse": ["<string>"],
        "must_escalate": ["<string>"],
        "must_call_tools": [
            {
                "name": "<string>",
                "input_contains": "<string>"
            }
        ],
        "must_emit_components": [{ "component": "<string>" }],
        "must_cite": ["<string>"],
        "must_not_refuse": True,
        "final_must_not_contain": ["<string>"],
        "judge": [
            {
                "name": "<string>",
                "criteria": "<string>"
            }
        ]
    }
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.patch(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'PATCH',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    name: '<string>',
    script: [{message: '<string>', tool_results: {}, confirmations: {}}],
    rubric: {
      must_refuse: ['<string>'],
      must_escalate: ['<string>'],
      must_call_tools: [{name: '<string>', input_contains: '<string>'}],
      must_emit_components: [{component: '<string>'}],
      must_cite: ['<string>'],
      must_not_refuse: true,
      final_must_not_contain: ['<string>'],
      judge: [{name: '<string>', criteria: '<string>'}]
    }
  })
};

fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id}', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "name": "<string>",
  "script": [
    {
      "message": "<string>",
      "tool_results": {},
      "confirmations": {}
    }
  ],
  "rubric": {
    "must_refuse": [
      "<string>"
    ],
    "must_escalate": [
      "<string>"
    ],
    "must_call_tools": [
      {
        "name": "<string>",
        "input_contains": "<string>"
      }
    ],
    "must_emit_components": [
      {
        "component": "<string>"
      }
    ],
    "must_cite": [
      "<string>"
    ],
    "must_not_refuse": true,
    "final_must_not_contain": [
      "<string>"
    ],
    "judge": [
      {
        "name": "<string>",
        "criteria": "<string>"
      }
    ]
  },
  "origin": "authored",
  "suite": "gate",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "annotation_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "unauthorized",
  "message": "Missing or invalid API key"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "rate_limited",
  "message": "Rate limit exceeded"
}

{
  "code": "internal_error",
  "message": "Internal server error"
}

{
  "code": "auth_unavailable",
  "message": "API key authentication is temporarily unavailable"
}

{
  "code": "request_timeout",
  "message": "Request timed out"
}

PATCH

api

conversation-instances

{instance_slug}

eval-scenarios

{scenario_id}

Correct a Conversation eval scenario

curl --request PATCH \
  --url https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id} \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "<string>",
  "script": [
    {
      "message": "<string>",
      "tool_results": {},
      "confirmations": {}
    }
  ],
  "rubric": {
    "must_refuse": [
      "<string>"
    ],
    "must_escalate": [
      "<string>"
    ],
    "must_call_tools": [
      {
        "name": "<string>",
        "input_contains": "<string>"
      }
    ],
    "must_emit_components": [
      {
        "component": "<string>"
      }
    ],
    "must_cite": [
      "<string>"
    ],
    "must_not_refuse": true,
    "final_must_not_contain": [
      "<string>"
    ],
    "judge": [
      {
        "name": "<string>",
        "criteria": "<string>"
      }
    ]
  }
}
'

import requests

url = "https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id}"

payload = {
    "name": "<string>",
    "script": [
        {
            "message": "<string>",
            "tool_results": {},
            "confirmations": {}
        }
    ],
    "rubric": {
        "must_refuse": ["<string>"],
        "must_escalate": ["<string>"],
        "must_call_tools": [
            {
                "name": "<string>",
                "input_contains": "<string>"
            }
        ],
        "must_emit_components": [{ "component": "<string>" }],
        "must_cite": ["<string>"],
        "must_not_refuse": True,
        "final_must_not_contain": ["<string>"],
        "judge": [
            {
                "name": "<string>",
                "criteria": "<string>"
            }
        ]
    }
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.patch(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'PATCH',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    name: '<string>',
    script: [{message: '<string>', tool_results: {}, confirmations: {}}],
    rubric: {
      must_refuse: ['<string>'],
      must_escalate: ['<string>'],
      must_call_tools: [{name: '<string>', input_contains: '<string>'}],
      must_emit_components: [{component: '<string>'}],
      must_cite: ['<string>'],
      must_not_refuse: true,
      final_must_not_contain: ['<string>'],
      judge: [{name: '<string>', criteria: '<string>'}]
    }
  })
};

fetch('https://api.usenexio.com/api/v1/conversation-instances/{instance_slug}/eval-scenarios/{scenario_id}', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "name": "<string>",
  "script": [
    {
      "message": "<string>",
      "tool_results": {},
      "confirmations": {}
    }
  ],
  "rubric": {
    "must_refuse": [
      "<string>"
    ],
    "must_escalate": [
      "<string>"
    ],
    "must_call_tools": [
      {
        "name": "<string>",
        "input_contains": "<string>"
      }
    ],
    "must_emit_components": [
      {
        "component": "<string>"
      }
    ],
    "must_cite": [
      "<string>"
    ],
    "must_not_refuse": true,
    "final_must_not_contain": [
      "<string>"
    ],
    "judge": [
      {
        "name": "<string>",
        "criteria": "<string>"
      }
    ]
  },
  "origin": "authored",
  "suite": "gate",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "annotation_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "unauthorized",
  "message": "Missing or invalid API key"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "<string>",
  "message": "<string>",
  "details": "<unknown>"
}

{
  "code": "rate_limited",
  "message": "Rate limit exceeded"
}

{
  "code": "internal_error",
  "message": "Internal server error"
}

{
  "code": "auth_unavailable",
  "message": "API key authentication is temporarily unavailable"
}

{
  "code": "request_timeout",
  "message": "Request timed out"
}

Authorizations

Authorization

string

header

required

Send the key as Authorization: Bearer <key>. Two kinds of key exist.

Organization keys are issued in the portal (Settings, then API keys), each bound to one environment, shaped nx_<environment slug>_<64 hex>. They carry no capabilities and pass every capability check, with one exception: routes under /api/v1/records, /api/v1/engines/{id}/opportunities, /api/v1/graph and /api/v1/catalog/documents accept an organization key only when its environment is live, and refuse any other with 403 scoped_key_required. Revocation takes effect within 60 seconds.

Scoped keys are issued by Nexio on request, shaped nxsk_v1_<24 hex key id>_<43 character secret>. Each is bound to one org, one environment, a set of engines and a set of capabilities. A malformed, unknown or revoked nxsk_ key fails with 401 and is never retried as an organization key. Revocation takes effect on the next request. A scoped key without a route's capability gets 403 insufficient_capability; a scoped key not bound to the engine gets 403 engine_binding_forbidden.

Key-grantable capabilities: engines:read, runs:write, runs:read, runs:defensibility:read, runs:test, catalog:read, catalog:documents:read, webhooks:manage, conversations:use, conversations:export, records:read, records:opportunities:run, actions:write, actions:read, graph:read, records:analyze.

Routes that accept organization keys only (every scoped key gets 403 insufficient_capability): environment management, engine create, update, configuration and publish, and conversation instance authoring. Each operation description names the capability a scoped key needs.

Path Parameters

instance_slug

string

required

Conversation instance identifier slug (e.g. platform-assistant).

scenario_id

string<uuid>

required

Body

application/json

name

string

Must not be blank. At most 200 UTF-8 bytes.

Maximum string length: 200

script

object[]

A scripted multi-turn conversation.

Required array length: 1 - 8 elements

Hide child attributes

script.message

string

required

The user message that opens the turn.

script.tool_results

object

Client tool NAME to the scripted result posted back when the turn hands that tool off. An unscripted handoff receives a scripted error result.

Hide child attributes

script.tool_results.{key}

object

Hide child attributes

script.tool_results.{key}.content

any

The result content the model sees, replayed verbatim. Absent content is sent to the model as an empty string.

script.tool_results.{key}.is_error

boolean

True when the recorded result was an error (a denial or an executed failure); replays verbatim.

script.confirmations

object

Confirm-gated client tool NAME to the scripted approval decision. Unscripted gates are denied.

Hide child attributes

script.confirmations.{key}

boolean

rubric

object

Expected behavior for a scenario. Every non-judge field is a deterministic check decided in code; judge dimensions are scored by the model judge, recorded per scenario, and can never fail a scenario.

Hide child attributes

rubric.must_refuse

string[]

Guardrail rule ids that must fire with a refused decision.

rubric.must_escalate

string[]

Guardrail rule ids that must fire with an escalated decision.

rubric.must_call_tools

object[]

Hide child attributes

rubric.must_call_tools.name

string

required

rubric.must_call_tools.input_contains

string

Optional substring that must appear in at least one of the tool's call inputs.

rubric.must_emit_components

object[]

Hide child attributes

rubric.must_emit_components.component

string

required

rubric.must_cite

string[]

Source references that must appear in the final answer (in a citation block's refs or the final assistant prose).

rubric.must_not_refuse

boolean

True asserts the conversation ended in a delivered answer: no refusal fired anywhere, and the final turn did not stop at refusal, escalated, output_check_triggered, or max_output_tokens.

rubric.final_must_not_contain

string[]

Phrases that must not appear in the final assistant prose, compared case-insensitively.

rubric.judge

object[]

Hide child attributes

rubric.judge.name

string

required

rubric.judge.criteria

string

required

suite

enum<string>

Available options:

gate,

workflows,

adversarial,

smoke

Response

The updated scenario.

One stored eval scenario.

string<uuid>

required

name

string

required

script

object[]

required

A scripted multi-turn conversation.

Required array length: 1 - 8 elements

Hide child attributes

script.message

string

required

The user message that opens the turn.

script.tool_results

object

Client tool NAME to the scripted result posted back when the turn hands that tool off. An unscripted handoff receives a scripted error result.

Hide child attributes

script.tool_results.{key}

object

Hide child attributes

script.tool_results.{key}.content

any

The result content the model sees, replayed verbatim. Absent content is sent to the model as an empty string.

script.tool_results.{key}.is_error

boolean

True when the recorded result was an error (a denial or an executed failure); replays verbatim.

script.confirmations

object

Confirm-gated client tool NAME to the scripted approval decision. Unscripted gates are denied.

Hide child attributes

script.confirmations.{key}

boolean

rubric

object

required

Hide child attributes

rubric.must_refuse

string[]

Guardrail rule ids that must fire with a refused decision.

rubric.must_escalate

string[]

Guardrail rule ids that must fire with an escalated decision.

rubric.must_call_tools

object[]

Hide child attributes

rubric.must_call_tools.name

string

required

rubric.must_call_tools.input_contains

string

Optional substring that must appear in at least one of the tool's call inputs.

rubric.must_emit_components

object[]

Hide child attributes

rubric.must_emit_components.component

string

required

rubric.must_cite

string[]

Source references that must appear in the final answer (in a citation block's refs or the final assistant prose).

rubric.must_not_refuse

boolean

True asserts the conversation ended in a delivered answer: no refusal fired anywhere, and the final turn did not stop at refusal, escalated, output_check_triggered, or max_output_tokens.

rubric.final_must_not_contain

string[]

Phrases that must not appear in the final assistant prose, compared case-insensitively.

rubric.judge

object[]

Hide child attributes

rubric.judge.name

string

required

rubric.judge.criteria

string

required

origin

enum<string>

required

Available options:

authored,

promoted_from_annotation

suite

enum<string>

required

The suite the scenario belongs to. gate scenarios run on every publish; the others run only when a run of that suite is requested.

Available options:

gate,

workflows,

adversarial,

smoke

created_at

string<date-time>

required

updated_at

string<date-time>

required

annotation_id

string<uuid>

The source annotation for promoted scenarios; omitted for authored ones.

Last modified on September 25, 2026

Overview

Engines

Environments

Conversations

Webhooks

Events

Records

Graph

Platform

Update an eval scenario

Authorizations

Path Parameters

Body

Response