Model routing contract and limits

Use the typed Task and terminal-outcome contract of the SDK and Platform versions you qualify. Your application owns the instruction, response schema, deadline, external-action controls, and business acceptance; routing chooses among the available Model Routes. Understand model routing explains how a route is chosen.

Stay within the configured pool

Every model turn of a Task uses only the routes enabled in the Tenant of the Task. This is the hard outer routing boundary. A routing policy cannot add a route from another Tenant or deployment, and Duale AI does not add a default route. Duale AI also runs its own analysis with separate models, outside your pool; Scope and limits describes that use.

A saved change applies to new provider requests after a short delay. If Duale AI cannot read the change, it can keep using the previous configuration for longer. When a cutoff must take effect at once, revoke the provider credential or block the network path.

Request compatibility is not a second hard boundary. When no route meets every check for remaining time, context size, and requested capabilities, Duale AI uses the closest routes rather than failing. It never extends the Task deadline. Images narrow the pool only when at least one route can carry them; otherwise the Task continues without the images, as Images explains. Qualify every enabled route for every reachable workload and define the safe result when required input is unavailable.

Route settings states what each provider type accepts and how you declare it.

Separate hard requirements from preferences

Classify a requirement before you configure routing:

If a workload needs one exact model version, use only standalone routes for that approved provider model and version. Detaching current fields does not make a preset-backed route a complete snapshot because other preset-owned fields can change. Keep several approved endpoints for the same version only when the account, region, credential, and endpoint boundaries permit them. Define what the application does when no approved route is available.

A configured provider model_id can refer to a rolling alias, gateway, or hosted deployment. An exact-version boundary also needs a pinned provider identity and a provider or operator version guarantee.

Set preferences inside the pool

Every RoutingPolicy field is a soft preference. A numeric value expresses relative emphasis inside the current pool. It is not a percentage target, service-level objective, route identifier, or control boundary.

The table describes the preferences you supply. Duale AI also applies its own needs for each part of the Task’s work, so a zero or omitted preference does not remove that emphasis on capability or latency.

Duale AI can add preferred capabilities that it infers from the Task’s latest message: the prompt of a new Task, or the user turn of a continuation. They add to the capabilities you set, never exclude a route, and are inferred again for each continuation. Set the capabilities you know the Task needs. Capability and benchmark evidence can change with the pool and published evidence; a capability label is not a certification or an allowed-use decision.

The SDK reference defines accepted values, omitted fields, and continuation overrides. For ask(), an explicit routing policy takes precedence over the capabilities= shorthand.

Task scheduling uses the separate top-level priority argument. It selects the priority class of the Task and the corresponding Task price. It does not replace speed_preference, change model ordering, or extend the deadline.

New Tasks use cost_sensitivity=0.1 when you omit that field. Explicit None or 0 removes the general preference for lower model prices; a positive value overrides the default. Continuations inherit omitted cost from their parent, and retries keep the value the Task started with. Some supporting model requests that Duale AI makes during a Task follow the preferences of Duale AI, so this preference does not apply to every model request.

What varies between two identical requests

The same policy can select different routes as the context, the pool, benchmark and speed evidence, or operating conditions change. Latency, retries, and estimated cost can also vary, including cached-input savings on routes that declare context caching. Plan for these limits:

• Do not assume a fixed traffic split, provider requests in sequence or in parallel, or a provider request to every enabled route.
• Output wording, reasoning, and Tool choice can differ. The Tool definitions available to a model can change between model turns.
• The SDK result does not identify the provider, model, order of provider requests, fallback, pool version, or the reason for the choice. Documented analytics can show provider and model later, but application logic must not depend on them.
• A continuation inherits routing fields it omits. The needs of each part of the work still apply, and inheritance does not freeze the model pool. Use the SDK reference above to override or clear individual preferences.
• Streamed content can arrive out of order, and a retried provider request can replace text already streamed. Follow Stream an answer and call model() for the terminal outcome. Stream exhaustion alone is not success.
• A configuration change does not cancel a provider request already in progress. Stopping a Task does not undo a provider request or Tool Call that has already started, or recover consumed cost.

Keep retries, display state, and business state changes independent of route identity and streamed text. Validate completed results against your application’s business rules.

Allow time for each provider request

Duale AI applies these bounds to every provider request, including a retry and a request to another route. It does not divide them by the number of candidate routes. An endpoint in your pool must meet them; when it exceeds one, that provider request fails, and Duale AI retries within the time that remains before the Task deadline.

The bound follows the provider request. A streaming Task can use a provider that returns its answer all at once; that request uses the 180-second non-streaming bound, and Duale AI then delivers the answer as a single chunk.

Generated text, reasoning, and Tool Call deltas count as progress. Keep-alives and metadata do not. A stream can last longer than 60 or 180 seconds while it continues to make progress and fits within the time that remains for the Task.

These bounds limit waiting on a model provider. They do not guarantee end-to-end delivery time to your application. Use the Task deadline and terminal outcome to handle the complete request.

When less Task time remains, after the time kept to finish and return the answer, that shorter time overrides these bounds. Retries keep the original Task deadline.

Duale AI starts new model, delegated, and Tool work only before the Task deadline. Final-answer writing has a separate window, and each Task, including a continuation Task, has its own. What happens near the deadline states the timing.

Keep a Task through a provider outage

When model providers are unavailable, Duale AI retries automatically within the time available under the Task’s deadline, allowing time to finish and return the answer. A temporary failure to read an image already in the Task history can also retry. Retries keep the original Task deadline.

An unresolved provider outage ends with LLM_PROVIDERS_UNAVAILABLE. A temporary image-read failure that cannot be retried further ends with retryable INTERNAL_ERROR. If every route refuses for a reason specific to the request, the provider requests end without waiting for an outage to recover: for example, LLM_CONFIG_CONTEXT_TOO_SMALL, LLM_CONFIG_UNAVAILABLE, or CONTENT_FILTERED.

Set the deadline from the business need. Show the Task as pending during retries and use Stop to end it. Automatic retries do not guarantee provider recovery or that delivery remains available under every retention, capacity, or deployment failure.

Handle pool failures

Task outcomes about the models in your pool have codes that start with LLM_. Task outcomes gives the meaning, the retry signal, and the next action for each code. Two of them need the configuration owner, not a retry: LLM_CONFIG_UNAVAILABLE and LLM_CONFIG_CONTEXT_TOO_SMALL mean that no route in the pool can process the request, so choose models for the work. LLM_PROVIDERS_UNAVAILABLE follows the automatic retries above.

A Task that ends with LLM_NO_PROGRESS fails instead of returning a best-effort final answer. LLM_PROVIDERS_UNAVAILABLE does not prove that every configured route ran: the Task’s deadline can end the provider requests earlier.