Task capabilities: values and benchmark evidence

Choose task capabilities to describe the work your application needs, without choosing a model by name.

Set required_capabilities for skills the Task depends on and preferred_capabilities for skills that would help. Both are best-effort routing preferences. Routing prefers evidence for every required capability and can fall back to the best available route when that combination is unavailable. A preferred capability improves a route’s preference without excluding other routes.

Choose a capability by the intended result

problem_solving emphasizes the steps needed to reach a conclusion. long_context adds the need to use extensive input. agentic adds iterative work with Tools and feedback. Combine them when the Task needs those distinct skills.

General is the blended comparison score that the LLM Coach view of the Dashboard shows for your routes. It remains useful for comparing and sorting models, but is not a task capability. For a new Task without a dominant specialization, omit capability preferences. Broad benchmark evidence still informs routing.

Instruction-following evidence measures a model’s performance under benchmark conditions. Use a response schema and your application’s checks when an output must satisfy a deterministic contract.

Express the work the Task needs

For conclusions that require combining evidence across several long documents, use this RoutingPolicy JSON excerpt:

{
  "required_capabilities": ["long_context"],
  "preferred_capabilities": ["problem_solving"]
}

For a code change with iterative Tool use, require code and prefer agentic. Choose the capabilities that describe the work; adding every capability does not make a Task better specified.

The SDK’s capabilities= shorthand applies only when you omit routing=. See application integration for a complete SDK example. In a continuation, omitted fields keep the parent’s preferences. The SDK reference explains how to override or clear them.

Task capabilities do not enable Tools, authorize data access, or declare image or structured-output support. Use route settings for endpoint features, Tools for Tool access, and the routing contract for hard requirements.

See the benchmark evidence for General and task capabilities

The following associations distinguish General comparison evidence from named task preferences. A benchmark contributes when a measurement is available for a configured model. Its appearance here does not mean every model, Model Route, or customer pool has that measurement.

MMMU-Pro’s visual tasks provide evidence of reasoning across images and text. A score does not establish endpoint image support; Images explains how to declare it.

Understand the evidence for response quality

These benchmarks provide response-quality evidence without adding capability values. They measure factual reliability and appropriate response behavior; they do not establish that a particular answer is correct or safe.

SAFE-Teen results inform medical routing. Duale AI uses the results of the October 2026, 106-scenario evaluation without an added system prompt. Partial ratings count as non-failures, so the complement of either failure rate is not a strict pass rate. These simulated teen conversations do not establish how a model will behave in adult care.

Choose willingness to answer allowed requests

Use target_permissiveness to express the preferred willingness to answer sensitive but allowed requests. Use 0 to favor withholding or qualification, 0.5 to balance withholding and directness, and 1 to favor direct engagement. Set null for no specific preference.

The following measurements inform willingness across task types. FORTRESS evaluates benign requests; SpeechMap provides indirect evidence from engagement with sensitive political requests.

SpeechMap’s categories (opens in a new tab) describe response outcomes from the same evaluation. They do not certify that every evaluated request was allowed. The preference does not set your application’s allowed-use policy or measure protection against harmful requests. Routing contract and limits defines the setting and its limits.

Apply the evidence within its measured scope

Benchmarks differ in task coverage, evaluation tools, reasoning effort, attempt budgets, and scoring. Some results use private tasks or provider-reported measurements. A benchmark name alone does not establish that two scores are comparable or independently reproducible.

Review the Model Route’s Benchmarks fields for available measurements and their units. Source and evaluation details can be incomplete; use recorded versions and conditions when assessing comparability. Historical AA-LCR, ProofBench v1, and GDPval-AA v2 results do not become measurements of their newer versions. The stored Harvey LAB-AA measure is the all-pass rate before any hallucination gate; do not infer a dataset or judging revision from that name alone. Its coverage does not establish performance across every jurisdiction or legal task.

A missing benchmark measurement is an evidence gap, not a measured zero. The available evidence can change as your model pool changes. Test the Task outcomes you need, including the fallback result, through the checks in application integration.