Update an eval case

Partial update — omitted fields are left as they are. Supplying steps REPLACES the case’s definition rather than merging into it. matchOptions and checks accept null to clear the case-level override and fall back to the suite’s.

PATCH /projects/{projectId}/eval-suites/{suiteId}/cases/{caseId}

Update an eval case

cURL

curl --request PATCH \
  --url https://app.mcpjam.com/api/v1/projects/{projectId}/eval-suites/{suiteId}/cases/{caseId} \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "declaredSuiteId": "<string>",
  "title": "<string>",
  "steps": [
    {
      "id": "<string>",
      "prompt": "<string>",
      "serverName": "<string>",
      "toolName": "<string>",
      "arguments": {},
      "action": {},
      "assertion": {}
    }
  ],
  "expectedOutput": "<string>",
  "iterations": 5,
  "repetitions": 5,
  "passThreshold": 0.5,
  "isNegative": true,
  "scenario": "<string>",
  "intent": "<string>",
  "models": [
    {
      "model": "<string>",
      "provider": "<string>"
    }
  ],
  "matchOptions": {},
  "checks": {
    "list": [
      {}
    ]
  },
  "import": {
    "sourceCaseKey": "<string>",
    "note": "<string>"
  }
}
'

200

Example

{ "id": "<string>", "title": "<string>", "steps": [ { "id": "<string>", "kind": "prompt", "prompt": "<string>", "serverName": "<string>", "toolName": "<string>", "arguments": {}, "action": {}, "assertion": {} } ], "iterations": 5, "isNegative": true, "models": [ { "model": "<string>", "provider": "<string>" } ], "declaredId": "<string>", "expectedOutput": "<string>", "repetitions": 2, "passThreshold": 0.5, "scenario": "<string>", "intent": "<string>", "kind": "capability", "matchOptions": {}, "checks": { "mode": "inherit", "list": [ {} ] }, "import": { "status": "exact", "sourceCaseKey": "<string>", "note": "<string>" }, "source": { "format": "markdown", "method": "ai", "fileName": "<string>", "fileHash": "<string>", "excerpt": "<string>", "startLine": 2, "endLine": 2, "extractorVersion": "<string>" }, "createdAt": 123, "updatedAt": 123 }

Authorizations

Authorization · string · header · required

MCPJam API key (sk_…). Create one at Settings → API keys. Guest sessions cannot use the API, and API keys cannot manage other API keys.

Path Parameters

projectId · string · required

ID of the hosted project that contains the server.

suiteId · string · required

Eval suite ID, as returned by POST /eval-runs.

caseId · string · required

Eval case id.

Query Parameters

declaredSuiteId · string

The suite file's own suite.id, when this request is that file syncing itself. A CI-owned suite (see EvalSuiteDetail.managedBy) refuses configuration writes with 409 and details.reason: "CI_OWNED_SUITE_READ_ONLY"; naming the suite's own declared id is how the file writing itself is allowed through. Naming any other id refuses exactly as loudly as naming none, so it is not a capability — omit it for an ordinary edit.

A QUERY PARAMETER on every route that takes it, never a body field. These request bodies are strict, here and on every Inspector that predates the CI-owned lock, so a body field is a 400 against an older deployment — which would break mcpjam cloud eval run --file for anyone whose CLI is newer than their Inspector. A query parameter is read by the deployments that know it and ignored by those that do not, which is the right degradation: an Inspector with no lock has no exception to make.

Maximum string length: 128

Body

application/json

Partial update. Every field is optional; omitted fields are left as they are. steps REPLACES the definition rather than merging into it.

declaredSuiteId · string

The suite file's own suite.id, when THIS request is a suite-file sync.

A CI-owned suite (see EvalSuiteDetail.managedBy) refuses configuration writes with 409 and details.reason: "CI_OWNED_SUITE_READ_ONLY". Naming the suite's own declared id is how the file writing itself is allowed through — the platform permits the write only when the id matches, so naming any other id refuses exactly as loudly as naming none. Omit it for ordinary edits.

PREFER THE declaredSuiteId QUERY PARAMETER on this route. This body field still works here, but these request bodies are strict on every Inspector that predates the CI-owned lock, so a body field is a 400 against an older deployment while an unknown query parameter is simply ignored. The MCPJam SDK and CLI send the query parameter.

Maximum string length: 128

title · string

Minimum string length: 1

steps · object[]

REPLACES the case's test definition wholesale when provided.

Minimum array length: 1

Show child attributes

expectedOutput · string

iterations · integer

Required range: 1 <= x <= 10

repetitions · integer

Per-case repetitions; omitted cases inherit the suite default. VERDICT POLICY 2 ONLY — on a legacy suite this is a VALIDATION_ERROR naming the upgrade, because the legacy resolver takes the trial count from iterations and the suite's minimumIterations and would never read this. Capped at 10, the same hosted ceiling as iterations; the suite-file contract's larger MAX_REPETITIONS is what a file may DECLARE, not what a hosted run accepts.

Required range: 1 <= x <= 10

passThreshold · number

Per-case pass threshold fraction. VERDICT POLICY 2 ONLY — on a legacy suite this is a VALIDATION_ERROR naming the upgrade, for the same reason as repetitions.

Required range: 0 <= x <= 1

isNegative · boolean

scenario · string

intent · string | null

Optional authored analytics grouping label. Must be already trimmed; omitted leaves the stored value unchanged and null clears it.

Required string length: 1 - 64

Pattern: ^\S(?:[\s\S]*\S)?$

kind · enum<string> | null

Authored case kind. Omitted leaves the stored value unchanged; null clears it.

Available options: capability, regression, null

models · object[]

Show child attributes

matchOptions · object | null

null clears the case override and inherits the suite's.

checks · object | null

null clears the case override.

Show child attributes

import · object | null

The converter's claim for this case, or null to remove one. Omitted leaves the stored claim unchanged; null removes it. The two are different requests, and a PATCH that conflated them would erase provenance on every unrelated edit.

Show child attributes

Response

200

application/json

The updated case.

A persisted eval case, in the public steps-first shape. Note this is NOT EvalTestCase, which is the INLINE authoring shape accepted by suite creation.

id · string · required

title · string · required

steps · object[] · required

Ordered test steps. A prompt step is a model turn; a single model-free toolCall step is a render-check; assert steps hold the expectations.

Minimum array length: 1

Show child attributes

iterations · integer · required

Required range: 1 <= x <= 10

isNegative · boolean · required

When true, the case passes if NO tools are called.

models · object[] · required

Show child attributes

declaredId · string

The case's effective declared id. Absent on cases authored before declared identity existed.

expectedOutput · string

repetitions · integer

Trials this case runs under verdict policy 2, overriding the suite default. Absent means the case inherits it. NOT a second spelling of iterations: that one is the legacy count, which the legacy resolver reads as a FLOOR (max(iterations, suite.minimumIterations)) and which a policy-2 case still reports for compatibility. This one is exact.

Required range: x >= 1

passThreshold · number

Fraction of this case's trials that must pass, overriding the suite default. Absent means the case inherits it. Never derived from the suite's minimumAccuracy, which is a PERCENT under a different resolver.

Required range: 0 <= x <= 1

scenario · string

intent · string

Optional authored analytics grouping label. Must be already trimmed; absent means unlabelled.

Required string length: 1 - 64

Pattern: ^\S(?:[\s\S]*\S)?$

kind · enum<string>

Authored case kind for the simple editor. Absent means the editor derives it from matchOptions.

Available options: capability, regression

matchOptions · object

Absent when the case sets none — omitted from the response rather than sent as null.

checks · object

Absent when the case sets none — omitted from the response rather than sent as null.

Show child attributes

import · object

What a converter CLAIMED about one imported case. exact is CONVERTER-CLAIMED exact — the converter says it applied a structural mapping rule, cited in note. MCPJam has NOT verified semantic equivalence, so user-facing copy must say "claimed exact", never "verified" or "accepted". Claim-only: who approved an approximation, when, and why is a PER-RUN decision frozen on the run (ImportEligibility.approvedApproximationReceipts), never stored on the case — an approval that lived on a case would outlive the run it was granted for and the edit that invalidated it. Approval and internal keys are rejected with 400, never stripped.

Show child attributes

source · object

Where an AI-assisted Markdown case was authored from. Provenance only: the file, its hash, and the line range the extractor read, so a reader can trace a case back to its source document. It is not an import claim (see EvalCaseImportClaim) and carries no approval semantics.

Show child attributes

createdAt · number | null

updatedAt · number | null