Control model changes across the production lifecycle

Treat a new model version as a new route with a distinct model_name. Use a standalone route and pin the provider identity where the provider contract permits it. Do not replace an existing route in place until you understand the technical and business effects.

The routing contract states the pool-wide selection and traffic-split limits. This page owns the evaluation, enablement, observation, rollback, emergency cutoff, and retirement sequence. Keep business approval and promotion controls in the customer’s change system.

Ownership and approvals maps each decision to the person who proposes, approves, applies, verifies, or stops it.

Classify the change

Classify what changed before you select reviewers. The class determines which boundary, customer outcome, and recovery evidence the approval must cover.

Different changes need different evidence:

A managed preset can change preset-owned fields, and a provider alias can move to another vendor revision. Include both in the same change process as tenant-owned route edits.

A preset-backed route is not a stable rollback snapshot. Detaching the fields that exist today pins only those paths; other or newly introduced preset-owned fields can still change. For a hard model, endpoint, or rollback boundary, keep a standalone route, a pinned provider version, and a secret-redacted effective snapshot in the customer change record.

Define acceptance before configuration

Write the acceptance thresholds, stop rule, and rollback trigger before you create the candidate route. Configuration evidence collected after enablement cannot replace a missing decision.

Build an evaluation set that covers:

• representative business cases, edge cases, languages, context, and input types;
• typed output, domain rules, tools, permissions, side effects, and recovery;
• latency, deadlines, throughput, quota, and cost limits;
• provider failure and exhausted-pool behavior; and
• security, privacy, contractual, regional, and human-review requirements.

Set acceptance thresholds, the authorized exception path, and the rollback trigger before the test. A schema-valid result is not sufficient business acceptance. Define both per-item controls that decide whether one result can be used and aggregate promotion thresholds for the route. Model-reported confidence cannot be the only acceptance signal.

Evaluate and introduce the route

This nine-step sequence keeps the candidate outside the production pool until the final enablement. Each step produces the evidence required by the next one.

1. Create an explicitly disabled route. For a standalone route, turn Enabled off before the first save; omission defaults to enabled. When you create from a preset, select Detach to edit for Enabled, then turn it off. Use a unique model_name and the approved identity, endpoint, version, limits, USD pricing, capabilities, and evidence. Saving validates shape but does not test the provider.
2. Protect the stable boundary. Keep the production tenant old-route-only. Its rollback routes must be standalone and use pinned provider identities; a preset or rolling alias is not a snapshot.
3. Qualify the candidate alone. In a separate evaluation tenant, enable only the candidate so the completion route is unambiguous. Use distinct credentials or a deployment when the evaluation needs account or physical isolation.
4. Qualify the planned target-pool composition. Test a non-production tenant that contains the candidate and every competing route. This tests compatibility and relative configuration inputs, not the target tenant’s exact identity, performance, cache state, or runtime order.
5. Run the application contract. Use the same request, strict response schema, tools, and business process as production. Accept route qualification only when await response.cache_hit() is False; None is not evidence of a provider call. Stop if the deployment handoff provides no supported way to verify a cache miss.
6. Review and approve the evidence. Include functional acceptance, data and contract review, performance, cost, resilience, recovery, open exceptions, named tenants, time window, thresholds, stop authority, and rollback action.
7. Ramp an application-owned cohort. Define a deterministic group of business items before submission, keep each root task in that group, then use a distinct SDK client and token for the evaluation tenant. Record the cohort version, tenant, business-item key, and task ID.
8. Observe customer outcomes. Make the application-owned record authoritative for attempted items, terminal failures, acceptance, tool execution, and full-workflow latency. Route analytics are aggregate, can be delayed, and do not show per-task, cohort, fallback, or how widely the change has taken effect.
9. Enable it in the target pool. After acceptance, apply the authorized target configuration. Every new task that reaches that tenant can rank the candidate. Preserve the stable tenant for the rollback window.

The change is ready only when the cohort record covers the intended workload, business results meet the declared thresholds, and technical evidence remains valid for its sample and time window. Record each analytics window and generation time. Missing data is not proof of success or absence.

Config API updates require the current version tag (ETag). Stop after a stale-write conflict rather than overwrite another change. The returned configuration hash is the resolved response’s current ETag; a preset update can change it without a tenant write. It is not an approval, immutable revision, signature, history, or applied-state proof. The current Dashboard list does not show every hash and writer needed by this process.

Do not enable the target pool when any gate lacks evidence. Return the candidate to the disabled state and keep the stable tenant unchanged until the rollback window closes.

Roll back or stop

For a cohort rollout, the deterministic rollback for new root tasks is to stop candidate admission and send the cohort to the stable tenant. Drain or stop candidate continuations; they use the original SDK and tenant. If work must continue, create a new stable root with only authorized reconstructed context.

If the shared production pool changed, re-enable the standalone rollback route before disabling the candidate. The two changes are not atomic, do not alter active work, and have no published propagation bound. Require cache_hit() is False for a live check, but do not treat one result as proof that the change reached every task, or that a route is retired.

For an urgent provider or credential cutoff, use the approved credential-revocation or network-block procedure as well as the configuration change. The submitting agent can request that an active task stop, but neither action undoes a provider or tool call that has already started. Verify the terminal stopped result separately.

Record a failed enablement or incident in the customer change system. Preserve secret-redacted configurations, available IDs and hashes, actor, time, affected workloads, decision, and evidence. Do not copy prompts or credentials into general logs. Product audit events support the record but do not replace it.

Retire the old route

Retire only after the replacement and its required fallback meet the acceptance criteria. Then:

• disable the route and verify the returned resolved configuration; do not claim runtime retirement without supported execution evidence;
• verify that no workload or exception still depends on it;
• revoke only credentials and quotas that no other route uses;
• update cost, provider, architecture, support, and incident records;
• preserve the approval and evaluation evidence for the required period;
• tell affected application and business owners what changed and when.

Delete the exact configuration only after the rollback and evidence-retention window, after concurrent edits are frozen. Deletion is hard and has no restore operation or write precondition. Do not delete a published preset key that other configurations can still reference.

Operate model routing and manage risk defines the health evidence and review cycle that decide whether the replacement stays enabled.