Evaluate

Typed decisions from an evaluation model.

01Overview

Evaluation models return choices, scores, and boolean probabilities instead of generated text. Send the shared `state` plus a map of typed `questions`; every question is evaluated independently and returned under its own id. Question types are boolean, noul, choice, or score. The upstream path and protocol are configured on the channel, so both Vercel AI Gateway TypeSafe and TypeSafe AI Jev channels work.

POST/api/v1/evaluate

Base URL https://capi.minapp.xinAPI version v1Authentication Bearer YOUR_API_TOKEN

02Parameters

Name	Type	Description
model*	string	An enabled evaluation model ID, e.g. typesafe-ai/jev.
state*	string \| object \| array	The shared input every question is evaluated against.
questions*	object	Map of question id to a question of type boolean, noul, choice, or score. Up to 20 per request.

03Request body

JSON

{
  "model": "typesafe-ai/jev",
  "state": "I was charged twice for my subscription this month.",
  "questions": {
    "refund": { "type": "boolean", "instructions": "Is the customer asking for money back?" },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["Low, no impact", "Medium, degraded experience", "High, blocking with financial loss"]
    }
  }
}

04Notes

Requires the llm.evaluate scope on the API key.
Evaluation models are not language models: /api/v1/chat/completions rejects them.
Billed from the reported usage like any other model; some evaluation models charge input tokens only.

Example

Evaluate

curl -X POST https://capi.minapp.xin/api/v1/evaluate \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model":"typesafe-ai/jev","state":"I was charged twice for my subscription.","questions":{"refund":{"type":"boolean","instructions":"Is the customer asking for money back?"}}}'

HTTP 200· OK

{
  "model": "typesafe-ai/jev",
  "answers": {
    "refund": { "type": "boolean", "probability": 0.98 },
    "urgency": { "type": "score", "score": 2.1, "probabilities": { "0": 0.05, "1": 0.2, "2": 0.75 } }
  },
  "usage": { "inputTokens": 275, "outputTokens": 20 },
  "cost": { "amount": 0.0001, "currency": "USD" }
}

All endpoints →