Changelog¶

All notable changes to Continuum Router are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased¶

Added¶

• Add per-backend request_extensions so an operator can set an OpenAI-compatible aggregator's routing preferences and attribution once in configuration instead of in every client request (#1631). A client could already send OpenRouter's provider, models, and transforms fields and its HTTP-Referer/X-Title headers, but there was no per-backend place to set a house policy, and a client that cannot be modified could not have one: request_params is a closed set of seven typed sampling parameters by design, and BackendConfig had no extra-headers field. request_extensions.body.defaults fills a key only when the client left it unset (client wins) and request_extensions.body.overrides replaces the client's value (config wins), both as a deep merge of JSON objects in which arrays, scalars, and type conflicts are decided wholesale at the key, so overrides.provider.data_collection: deny enforces one leaf while keeping the client's provider.sort. request_extensions.headers adds headers that replace a forwarded client header of the same name. The fragment is merged last into the fully built wire payload of the selected backend at every send site, on every retry and fallback hop (a hop to another backend carries that backend's extensions and never the previous one's), for chat completions, completions, responses, and embeddings, including the Anthropic Messages and Responses ingresses converted for an OpenAI-wire backend; headers also ride on model discovery, health probes, pre-warm, control-plane probes, and the realtime handshake. Extended in #1652 to one rule: every request that carries the backend's credential carries its extension headers. The guardrail backend: and smart-routing classifier calls through the typed backend executor, provider batch lifecycle calls, engine-statistics scrapes, backend type detection, Gemini context-cache management, the model auto-discovery a typed backend runs at construction, llama.cpp /props capability detection, and the typed Anthropic and Gemini connection pre-warm now carry them too, over both HTTP and Unix socket where the site has both. A consumer that outlives one configuration generation reads the block per send, so an Admin backend edit or a config-sync change reaches its next call, and a headers-only edit restarts an engine-statistics poll task so the next scrape carries the new values. The Admin backend-create type probe runs before the candidate is validated, so it attaches the candidate's headers only when they already pass header validation and otherwise probes without them, which keeps an unvalidated Authorization extension from replacing the key on the wire. The Unix-socket health probe still sends neither the credential nor the headers. Validation runs at load on every ingestion path with a named path in the error: body is a load error on anthropic, gemini, and bedrock, and headers on a Bedrock runtime/converse backend, whose SigV4-signed dispatch cannot carry them; request identity, stream, the request_params keys and their max_completion_tokens/max_output_tokens aliases, tool and output-shape fields, the response-shape fields text, logprobs, top_logprobs, encoding_format, and dimensions, router-managed fields, and any key the dispatch strips for that backend (including stop on a Codex OAuth backend) are reserved, while stop, seed, and logit_bias otherwise stay settable; auth-owned and router-owned header names are refused case-insensitively and with _ read as -; and the fragment (16 KiB, depth 8), header count (16), and header name and value sizes are bounded, with control characters refused. Tightened in #1650: background, include, include_reasoning, prompt_cache_key, and safety_identifier (Responses), modalities, audio, and prediction (Chat Completions), the legacy Completions echo, vLLM's prompt_logprobs, and vLLM's structured-output fields guided_json, guided_regex, guided_choice, guided_grammar, and their vLLM 0.12 replacement structured_outputs are now reserved for the same reason as the fields above, since each would change request content or the response shape a client parses, while service_tier, prompt_cache_retention, and prompt_cache_options stay settable alongside stop, seed, and logit_bias. Header values must also be visible ASCII (SP through ~): a value with a byte outside that range, such as café or Hangul, is now refused at load, because the wire client otherwise sends it as RFC 9110 obs-text that upstreams and intermediaries handle inconsistently. request_extensions has not shipped in a release yet, so this is a same-cycle tightening rather than a migration: a configuration written against the block as #1631 first added it, using one of these now-reserved body keys or a header value that is not visible ASCII, fails to load under this and later builds; drop the key or percent-encode the header value. Header values support ${ENV_VAR} and are treated as secrets whatever the header is called: masked on the Admin config and backend reads, export, history, and the WebUI with names left visible, preserved across a masked GET-then-PUT round trip, redacted by config diff and the MCP tools, never logged, and reported to Continuum Hub only as the request_extensions unsupported-field code on a local backend export. A change applies to the next request without a restart, a backend with a body fragment keys its response-cache entries by a fingerprint of the fragment so an edited fragment is never answered from entries stored under the previous one, and a configuration without the field serializes and dispatches exactly as before.

Fixed¶

• Stop charging cached prompt tokens twice on Continuum Hub usage records (#1642). The Hub prices input_tokens and cached_input_tokens as disjoint buckets, but the record sent the backend's prompt_tokens as input_tokens, and for OpenAI-shape responses that count already includes prompt_tokens_details.cached_tokens, so every cached token was billed at the input rate and again at the cached rate; local cache replays were billed the same way. prompt_tokens is now the inclusive prompt count for every producer (non-cached input plus cache reads plus cache writes) and the record carries input_tokens = prompt_tokens - cache reads, the same billable input router-local input_tpm and monthly enforcement already charged. Shapes that report an exclusive count are converted where they enter: the usage parser adds cache_read_input_tokens and cache_creation_input_tokens back onto an Anthropic-native input_tokens, the native /v1/messages stream tracker now also captures cache writes, and the Anthropic and Bedrock Converse transforms to /v1/chat/completions emit the inclusive prompt_tokens with prompt_tokens_details.cached_tokens and keep the top-level Anthropic cache keys even without a thinking block. Two visible consequences: prompt_tokens returned to /v1/chat/completions clients on an Anthropic or Bedrock cache hit is larger than before, and on those cache hits router-local input TPM and monthly token counters now charge the non-cached input plus cache writes (before, the cache read was subtracted from Anthropic's already-exclusive count, which could zero the charge), while Prometheus input-token counters, admin statistics, model experiment token totals, and capacity observations now count Anthropic and Bedrock cache reads and writes as prompt tokens. Anthropic streaming on /v1/chat/completions still reports no prompt usage until #1640. The reverse bridges follow the Anthropic convention: when /anthropic/v1/messages is served by an OpenAI-compatible or Responses API backend, the non-streaming body and the streaming message_start/message_delta usage now report input_tokens as the upstream prompt minus cache reads and writes (reading prompt_tokens_details.cached_tokens and input_tokens_details.cached_tokens as well as the top-level keys), so a client, or this same router re-parsing the bridged body (a non-streaming response, its own local cache replay, or a bridged stream it reconstructs itself), no longer counts the cached prompt twice; on a cache hit those clients see a smaller input_tokens. The native /v1/messages streaming tracker now also reads input_tokens, cache_read_input_tokens, and cache_creation_input_tokens from message_delta (the Anthropic Messages API repeats those cumulative counts there, and a bridged stream defers them there because its message_start carries input_tokens: 0), so a router metering another router's bridged stream records the real prompt instead of 0. /v1/responses served by an Anthropic backend now reports the Responses-convention inclusive input_tokens with input_token_details.cached_tokens and cache_write_tokens on both the non-streaming and streaming paths (streaming previously recorded Anthropic's exclusive count, or 0, and dropped both cache buckets), /v1/responses served by an OpenAI-compatible chat backend keeps prompt_tokens_details.cached_tokens as input_token_details.cached_tokens on the streaming path, and the usage parser reads the router's own singular input_token_details.cached_tokens, so a router metering another router's /v1/responses body, or replaying its own cached one, keeps the cached bucket instead of billing it at the input rate.

• Decode an OpenAI-compatible /v1/models catalog that omits the decorative object and owned_by fields, instead of failing the whole catalog with ModelFetchError::ParseError (#1629). ModelsResponse.object, Model.object, and Model.owned_by had no serde(default), so an aggregator such as OpenRouter, whose catalog carries none of the three, was classified as a permanent parse failure and left GET /v1/models empty for that backend forever, even though the same backend served chat traffic normally. object now defaults to "list"/"model" and an absent owned_by decodes as an empty string, which now reaches the owned_by repair in process_models_response (an openai backend fills it with "openai", except that a vendor/model id takes its vendor namespace, see #1630) instead of being rejected before that repair ever runs. An explicit JSON null on object, owned_by, or created, which servers that serialize unset optional fields as null emit, now decodes the same as an absent field instead of failing the catalog with a type error. id stays required, so an entry with a missing or null id still fails to decode. Both strict decode sites (handle_successful_response over HTTP and handle_unix_socket_response over a Unix socket) now share one decode_models_response helper, so a future change to this logic cannot fix one transport while missing the other. Making those fields optional also shrank the smallest accepted catalog entry from 36 bytes to 10 ({"id":""}), so a catalog inside the default 10 MiB max_response_size could decode to roughly a million entries before max_models_per_backend truncated it, a measured 657 MB peak per fetch against 175 MB for the densest catalog accepted before, and several such backends stack because fetch_all_models holds every result until the whole fetch pass finishes. Both decode sites now cap entries during decode itself instead of truncating afterward, keeping the first max_models_per_backend entries and still failing the catalog on a malformed entry past the cap, which brings the same body down to a 3 MB peak.

• Report an aggregator backend's model vendor and its upstream pricing, context window, and output cap in the model catalog (#1630). An OpenAI-compatible aggregator such as OpenRouter sends no owned_by on any catalog entry and namespaces every id by vendor, so the backend-type substitution in process_models_response stamped every model it listed, from anthropic/claude-haiku-4.5 to deepseek/deepseek-v4-flash, as owned_by: "openai", and the router ignored the context_length, top_provider.max_completion_tokens, and pricing the upstream sent, so each of those models read as unpriced with no known context window. When an upstream sends no owned_by for a vendor/model id, the first path segment is now the vendor. A real upstream value is still kept, and an explicit placeholder such as OpenAI's "system" still becomes the backend-type owner, as does a missing value on an id without a namespace; a per-model metadata owned_by still wins, and response_defaults.owned_by still applies only to a value that remains a placeholder. GET /v1/models keeps its OpenAI field set and changes only in that vendor. On GET /v1/models/extended and GET /v1/models/{model}, context_length joins vLLM's max_model_len as a context-window spelling (max_model_len wins when both are present), top_provider.max_completion_tokens sets limits.max_output, and therefore max_tokens, when it is within the entry's context length, and pricing.prompt/pricing.completion are converted from USD per token to pricing.input_tokens/pricing.output_tokens in USD per 1M tokens, with input_cache_read becoming cached_input_discount; the raw upstream fields keep passing through on the extended listing. Upstream values are treated as untrusted: a price must parse to a finite, non-negative number no larger than 100,000 USD per 1M tokens, so OpenRouter's "-1" for a variable price and an unparseable value publish no price rather than zero. Configured metadata from model-metadata.yaml, a model-metadata.d/ drop-in, backend model_configs, or the built-in OpenAI catalog still wins for limits and every other field, but not for the price, which is specific to the provider serving the model: an upstream price wins over pricing from the shared model-metadata.yaml, its drop-ins, and the built-in catalog however the entry was matched, so a shipped 0/0 self-hosting entry reached through vendor/ prefix stripping no longer makes a model such as minimax/minimax-m2 read as free, and only a price set in the serving backend's own backends[].model_configs wins over the upstream one. When several backends serve one id, the context window and output cap are the minimum across them, and the price folds one candidate per backend (its model_configs price if it pins one, otherwise its upstream price) into the component-wise maximum with the smallest discount. Both are collected per backend during deduplication the way #1456 collects max_model_len, so they do not depend on pool order; the published price is the highest any backend states, and a backend whose price is missing or "-1" is left out of the fold rather than bounded by it. A model kept listed because its backend's discovery failed now has its metadata recomputed from current configuration and the retained per-backend reports on every refresh, instead of republishing the folded context window, output cap, and price cached from the pass before, which could still reflect a backend no longer serving the id (this also applies to the #1456 context window). Two pre-existing passthrough hazards on the extended listing are closed as well. An upstream entry carrying a key the router writes itself (backends, tier, domains, or metadata; a type: continuumrouter upstream's own listing emits tier and domains) put that key into one JSON object twice, and such keys are now dropped from the passthrough. An upstream metadata object was decoded as router-trusted metadata, which let an upstream publish unvalidated pricing and failed the whole catalog on a shape mismatch; it is no longer decoded.

• Accept model ids containing / on GET /v1/models/{model} and GET /admin/smart-routing/model-profiles/{model} (#1647). Both routes were mounted as a single-segment axum capture, so a namespaced id such as OpenRouter's anthropic/claude-haiku-4.5 or vLLM's HF-style Qwen/Qwen3-32B 404'd in raw form and only resolved percent-encoded (anthropic%2Fclaude-haiku-4.5), which PR #1644 had to document as a workaround. Both are now mounted as an axum catch-all ({*model}), so the raw and percent-encoded forms return identical bodies; the reserved static segments extended, proxy, and refresh on /v1/models keep routing to their own handlers ahead of the catch-all, unchanged. A new model_id_from_models_path helper in src/http/middleware/model_extractor.rs parses the same decoded id that axum hands the single-model handler, and EnhancedRateLimitMiddleware::extract_model now uses it (as does the unmounted extract_model_middleware); the rate limiter previously read path.split('/')[3] off the raw, undecoded path and so keyed the per-model rate-limit dimension on anthropic rather than anthropic/claude-haiku-4.5 for a namespaced id, and on the literal extended for /v1/models/extended.

• Scope request de-duplication to the configuration snapshot each request runs under (#1648). The de-duplication key hashed only the endpoint, the request body, and the inbound headers, and a hit returns the earlier result before any backend is selected or dispatched to, so a byte-identical non-streaming request repeating within retry.timeout of an earlier one replayed that earlier answer across a configuration publish: a hot reload, an Admin API write, a control-plane config-sync install, or an AppProxy reconcile that changed backends[].request_extensions, backend membership, a backend's models list, URL, or credential, or selection_strategy took effect for everything except such a repeat. Each key now folds in a per-process random salt and the address of the Arc<Config> snapshot the request executes under, and every entry keeps a Weak handle to that allocation, which reserves the address while an entry is keyed on it and still lets the superseded configuration's contents drop with its last request. Within one published snapshot nothing changes: identical requests still coalesce onto one upstream call, and a request waiting on an in-flight one still receives its result. Any publish, including one that changes nothing routing-relevant, now starts a fresh window, costing at most one extra upstream call per distinct request per publish. Streaming is unaffected, because it is not de-duplicated. Expired entries are now reclaimed by a periodic sweep started with the server, because an entry keyed on a superseded snapshot is never looked up again and so never reaches the lazy expiry check that a lookup performs; without it, eviction at max_entries would be the only reclaim path. In AppProxy worker and ROUTER modes the effective window is shorter than retry.timeout, because every reconcile tick republishes the configuration whether or not anything changed, making it appproxy.reconcile_interval (default 15s). Library surface, for anyone using the public continuum_router::services::deduplication module directly rather than the documented serve_embedded entry point: DeduplicationEntry gains a public config_snapshot: Weak<Config> field, and DeduplicationManager::{generate_request_hash, mark_in_flight, cache_success_with_backend, cache_error} and EnhancedRetryHandler::execute_with_deduplication_attributed each take the snapshot as a new leading parameter.

• Re-validate the composed configuration when runtime backends join a hot reload (#1649). A reload validated the file on its own and never the result of merging in the backends created through POST /admin/backends and kept in the backends_persistence_file sidecar, so the router could publish an effective configuration that the next startup refuses: a reloaded tracing.headers name that a runtime backend already uses in request_extensions.headers, or a renamed file backend that duplicates a runtime backend's backend_id. The reload worker now composes a candidate and runs the shared validation gate on it before anything is published, and a failure takes the existing retry-and-retain path, so the previously published revision stays live. The ledger write that records which runtime entries a file shadows is deferred until publication, so a rejected reload cannot make a live runtime backend report configured and lose the next Admin update to it on restart.

• Escape upstream-authored text before it reaches a model-discovery log line (#1651). Every configured backend's /v1/models is untrusted input, and tracing-subscriber escapes ANSI/ESC in a log message but not a raw \r/\n, so an id or owned_by value carrying one could forge a second log record; process_models_response and eighteen sites across models::aggregation interpolated model.id/model.owned_by directly into debug!/trace! messages, reachable today from any backend whose catalog entries a client or operator does not fully control. Two new helpers in core::text_utils, escape_upstream_text (bound and escape control characters plus U+2028/U+2029) and describe_upstream_json_error (apply that to a serde_json::Error's message while keeping its line/column), are now used at every such site and at the three /v1/models//props parse-error paths, closing both the forging vector and the unbounded log growth an oversized upstream value could cause on a parse failure.

v1.28.0 - 2026-09-11¶

Forty-six commits since v1.27.0 completed KV-cache event ingestion end to end, put engine load and caller identity into routing decisions, and closed a run of provider-compatibility defects. A new continuum-kv-listener bridge carries vLLM, SGLang, and native TensorRT-LLM events into the router's cache index, a router-issued cache_salt takes detokenization out of that path, and Continuum Hub can manage and roll back KV routing as a synced configuration section. Smart routing gained an EngineLoad selection strategy, engine statistics in its load assessment, and when conditions that match on key tier, organization, language, and client headers, while its rule classifier learned code intent from wording and reached 100.0% domain accuracy on a labelled dataset grown to 59 cases. Fallback chains now run on the Anthropic Messages and Responses ingresses with a per-hop timeout multiplier, the semantic response cache has a serving path behind the hub gate, and the current Anthropic, OpenAI, and Google flagship models are in the catalog, with Fable 5.1 and Mythos 5.1 refusing forced tool use.

Added¶

• Run fallback.fallback_chains on /anthropic/v1/messages and /v1/responses (#1609). Both provider-shaped ingresses dispatched exactly once, so a chain keyed on the model a client sent to either of them never fired, the primary's failure was the client's answer, and no X-Fallback-* header could appear; PR #1598 had to record its X-Original-Model criterion as vacuous there. The two ingresses now run their own per-attempt selection, admission, and native dispatch through FallbackService::execute_with_fallback_snapshot, the executor the chat funnel uses, so the trigger conditions, max_fallback_attempts, and the per-backend hop dial bound are the ones chat applies. The primary attempt keeps its native wire, a hop is the same typed request with only the model name swapped and converted at dispatch by the selected backend's backend_type, and on /v1/responses this covers the passthrough and every convert strategy. Each attempt is judged from its shaped response: a 2xx commits it, a refused connect, timeout, unknown model, or no admissible backend hops under its own trigger class (carried by a marker the ingress error mappers store in the response extensions, never on the wire), and a provider status hops only when its code is in trigger_conditions.error_codes; on exhaustion the client receives the last attempt's own error in the ingress dialect. The streaming arms hop before the first byte only: a failure before the provider handshake restarts the native stream on the next chain entry, a committed handshake never hops, and mid-stream recovery stays on the OpenAI-compatible wire. The five X-Fallback-* headers are emitted on both ingresses honoring notify_on_fallback, with X-Original-Model naming the client's requested name after an alias rewrite (#1588), and a fallback-served request is counted in the supply telemetry at the same seam as chat. docs/en/architecture/model-fallback.md and docs/en/error-handling.md (and their docs/ko mirrors) now state the coverage per ingress and arm.

• Match smart_routing.routing_policies[].when on the caller's identity as well as on the content of the request (#1569). Four optional fields join complexity, domain, and requires under the existing semantics (AND across fields, OR within one, first match wins, evaluation order unchanged): key_tier matches the hub tier id, org the API key's organization_id, language a short BCP-47 primary subtag, and header a map of header name to accepted values. One auto alias can now send a paying tier to flagship models, pin a sandbox organization to the cheap tier, route Korean requests to a model that handles them, and honor an explicit client hint. Every field is serde(default), so an existing configuration keeps its exact meaning, and is_catch_all counts them, so a condition that sets only an identity field no longer satisfies the catch-all warning. Absence fails closed: a build without the control-plane feature resolves no key tier and neither does a non-hub key, so a when.key_tier policy never matches there rather than matching everything; an unauthenticated request carries no organization; and a letterless request carries no language and matches no language clause, while examined but unattributable text carries und. when.header reads client-controlled input, so only names a loaded policy references are captured, values are capped at 256 bytes and dropped rather than truncated (a longer value that merely starts with the configured one must not match), and no value reaches a metric label or a log line; gate anything privileged on key_tier or org instead. The tier reaches the handler through a new always-compiled KeyTierContext request extension that only the control-plane enforcement middleware populates, following the OptimizationDecision pattern, and the new RequestIdentity holds no HTTP types so the engine precomputes the header names any loaded policy reads and copies nothing else per request. POST /admin/smart-routing/simulate accepts key_tier, org, and headers next to payload, so a tier-gated policy is checkable without holding a key on that tier, and GET /admin/smart-routing/policies reports the new fields. config validate rejects an invalid when.header name or a blank when.language tag, and warns when a policy uses when.key_tier on a build that can never resolve one.

• Read code intent from wording rather than only from literal markup (#1604). The code domain was keyed entirely on fenced or inline code markup, so the classifier recognized a pasted snippet but not a request to write one: "Implement the rate limiting logic for this REST API." carries no backticks, fired no code signal, and fell through to general or multilingual, so every policy keyed on domain: [code] missed the unmarked half of its traffic. The same sentence in Korean had the identical gap, because the signal was keyed on markup rather than on language. A code keyword category joins the existing four on KeywordTable and its smart_routing.classifier.rule.keywords config mirror, with EN_CODE and KO_CODE shipped built in and merged the same additive way, so a Korean request embedding English technical vocabulary reaches the signal from either side of the mix. The signal needs two distinct matches rather than one, which is the existing house pattern (detect_analysis_markers already requires two) and is what lets the lists carry ordinary words like function and 수정 at all, since reaching the threshold takes a second independent piece of evidence from the same request. Requiring a bare action verb instead was tried and rejected: write fires on "Write a poem about autumn leaves" and flips every creative case to domain = code, so the lists mix actions with objects because each fails alone, bare actions taking creative writing and bare objects taking "What is a REST API?". Two entries are absent on purpose with a test pinning their absence, because a single-word keyword is a plain substring match: logic sits inside biological and technological, and api inside capital and rapid, so either would have reached the threshold beside an innocent second match and classified "Explain the biological function of mitochondria" as code. rest api carries the API case instead, and a multi-word entry is matched token-wise.

• Add continuum-kv-listener, a standalone bridge that turns engine KV-cache events into the per-backend SSE streams kv_cache_index.event_sources[] already consumes (#1561). Tier 4 of KV-aware routing has read one SSE stream per backend since the index was introduced, but vLLM and SGLang publish their KV events as msgpack over ZMQ rather than as SSE, so every deployment had to supply that adapter itself. The listener ships as a second binary with its own man page (continuum-kv-listener(1)) and Debian packaging. It subscribes to one ZMQ endpoint per backend, tracks bounded block chains, resolves each chain's text through the engine's own /detokenize endpoint, and computes the prefix hash through the same extract_prefix_key seam the router routes on, so the index and the router agree on the key by construction rather than by convention. One listener serves a fleet: /events/<backend_name> fans out to every subscribed router and replays retained history on reconnect through Last-Event-ID, so a router restart does not lose the blocks the engine reported while it was away. A sequence gap resets that source's tracked state instead of attributing later blocks to a chain the listener can no longer reconstruct. The publisher allowlist, the ZMQ bind or connect mode, the chain and tracked-block ceilings, and the prefix length are all configuration; config.kv-listener.yaml.example carries the shipped shape. The router side changes only in what event_sources[].endpoint points at: the engine container no longer needs an extra HTTP producer port, and the documented endpoint moves from http://vllm-1:8000/v1/kv_events to http://kv-listener.internal:7817/events/vllm-1.

• Add a native TensorRT-LLM source to continuum-kv-listener (#1563). A source declared engine: trtllm polls trtllm-serve's POST /kv_cache_events instead of subscribing to ZMQ, and normalizes its created, stored, and removed records through the same processor and SSE hub the ZMQ sources use, so the router consumes one stream shape regardless of engine. The poll interval, the request timeout, and the response body ceiling are each bounded and configurable, and the event ID rejects a duplicated response and clears stale state after a gap. trtllm-serve exposes no public detokenize endpoint, so this source resolves prefixes only through the router-issued cache salt: the first usable stored block must carry a salt matching the source's backend, later blocks inherit that resolved chain identity, and a chain whose salt is missing, malformed, or issued for another backend is discarded rather than attributed to a prefix. cache_level: 0 is read as GPU and higher levels as storage, an omitted stored level defaults to GPU, and a removal retains the tier it was tracked at. The URL must be the exact /kv_cache_events path on an explicitly allowlisted host, with no credentials, query, or fragment; redirects are never followed, and response bodies, token IDs, and cache salts stay out of logs and errors. Dynamo deployments that already publish the vLLM-compatible ZMQ event format should keep using the ZMQ source rather than configuring a second native HTTP source.

• Add prefix_routing.salt_echo, a default-off router-issued cache_salt that lets a listener attribute KV blocks without detokenizing them (#1562). With it enabled, a prefix-routed chat request to a vLLM, SGLang, or TensorRT-LLM backend carries a salt of the form cr1:<backend>:<prefix-hash>, and the engine echoes that salt back on the blocks it stores: vLLM in extra_keys, SGLang in metadata, TensorRT-LLM in blocks. A listener source configured identity: salt_echo then reads the prefix key straight off the event, which takes the /detokenize round trip out of the ingestion path and is the only way to attribute TensorRT-LLM blocks at all. A client-supplied cache_salt is preserved and never overwritten, a router-issued salt is bound to the selected backend name so it cannot be replayed against a different backend, and the default of false leaves request bodies byte-identical to previous releases.

• Advertise kv_routing as a second Hub-manageable configuration-sync section on control-plane builds (#1564). Alongside the existing request_params section, a Hub may now deliver selection_strategy, the prefix_routing.* leaves, and a bounded set of kv_cache_index.* routing leaves, so a fleet can turn KV-aware routing on, retune its scoring, and roll it back centrally instead of by editing every router's file. The section carries its own schema version and structured capabilities, its snapshot entries report presence and digest without values, and control_plane.config_sync.kv_routing_immutable: true pins the whole section locally the way request_params_immutable does. Delivered values go through the durable config-sync store beneath explicit local pins, and an unknown path, an out-of-range value, or an event-source endpoint carrying a secret is rejected rather than applied. A successful apply publishes through the existing hot-reload watch channel, so the effective snapshot reaches live reload consumers instead of waiting for a restart.

• Bridge Chat Completions requests that combine function tools with a reasoning effort through /v1/responses on the OpenAI models that refuse the combination (#1547). OpenAI answers /v1/chat/completions with 400 invalid_request_error and param: reasoning_effort when a request carries function tools and the effective reasoning_effort is not none, and the router forwarded that unchanged, so a client that pins an effort per model family and pins the Chat Completions endpoint for any custom base URL (n8n does both) could not use these models with tools at all. Two new model-metadata keys next to responses_only describe the upstream restriction as data on the model entry: chat_completions_tools_require_none_reasoning marks a model whose Chat Completions route refuses the combination, and the optional chat_completions_default_reasoning_effort records the effort upstream applies when the request omits the field, which is what separates the two rows of the restriction (the gpt-5.6 family defaults to medium upstream, so omitting the effort is refused there and accepted on gpt-5.4 and gpt-5.5). When the resolved model carries the flag, the request has a non-empty tools array, and the effective effort would be refused, that one request is dispatched through the existing responses_only bridge to /v1/responses, which accepts the combination, and the reply is translated back into the Chat Completions shape, streaming included. Everything else keeps going straight to /v1/chat/completions, reasoning_effort: "none" with tools and every request without tools included, so the cost and latency of the paths that already worked do not change. The requested effort rides through as reasoning.effort and an omitted effort stays omitted: the router does not rewrite the effort to none, which would silently change model behavior (#510). The keys ship set on gpt-5.6-sol (alias gpt-5.6), gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, and gpt-5.4-nano, in the built-in OpenAI registry and in model-metadata.yaml, with every row measured against the live API on 2026-09-03; the gpt-5.4 mini and nano tiers are included because live probing found them refused too, while gpt-5.2 and earlier accept the combination and are untouched, which is why the rule is per model rather than a family prefix check. Because these are ordinary metadata keys, a model-metadata.d/ drop-in clears them the day OpenAI lifts the restriction, without waiting for a release. Reasoning validation still runs before the bridge decision, so an effort outside the model's supported set is the router's own 400 and never reaches upstream. The dispatch log line and the new responses_bridge_total counter both carry a reason label separating tools_with_reasoning from an unconditionally bridged responses_only_model, and continuum-router config validate now warns when a backend model_configs entry sets responses_only: true on one of these models: that override is the pre-bridge workaround and remains the mitigation on released versions, at the cost of routing plain tool-free chat through the Responses API as well. Built-in OpenAI metadata now also resolves through a model's declared aliases, so gpt-5.6 inherits the gpt-5.6-sol entry's capability flags instead of resolving to no built-in metadata at all.

• Add catalog entries for the current Anthropic, OpenAI, and Google flagship releases: claude-fable-5-1 and claude-mythos-5-1, gpt-6-astra, and the Gemini Flash line from 3.5 through 3.8 (gemini-3.5-flash, gemini-3.5-flash-lite, gemini-3.6-flash, gemini-3.7-flash, gemini-3.8-flash). Each entry carries the published context window, max output, knowledge cutoff, and per-MTok pricing, so /v1/models metadata, smart-routing profile inference, and cost reporting stop falling back to name heuristics for them. Two of the additions are more than data. gpt-6-astra is also registered in the built-in OpenAI catalog (a new gpt6_family module) so a native type: openai backend's /v1/models discovery reads its real 1.05M context window and $10 / $50 price instead of the parser's fabricated defaults, and it ships chat_completions_tools_require_none_reasoning together with chat_completions_default_reasoning_effort: "medium": OpenAI refuses function tools on /v1/chat/completions for this model and, unlike the gpt-5.6 family, rejects reasoning_effort: "none" outright, so the "set reasoning_effort to none" remedy named in its own error text does not exist here and every tool-carrying request is bridged through /v1/responses. Its accepted effort set (low through xhigh, plus max on the Responses API) is registered in supported_reasoning_efforts, which turns a none request into the router's own 400 rather than an upstream one. claude-fable-5-1 and claude-mythos-5-1 are advertised by the Anthropic backend's default model list, and every existing capability gate already covers them because those gates key on the Mythos-class name and the parsed family version. Prompt cache reads are recorded at 2.5% of base input on both, against 10% elsewhere in the Claude family. The Gemini 3.6, 3.7, and 3.8 Flash entries record the promotional $0.75 / $3.75 per MTok rate that runs through 2026-12-31 rather than the $1.50 / $7.50 that replaces it on 2027-01-01, and they are deliberately absent from the Gemini supports_thinking_disable list, so reasoning_effort: "none" for them is refused by the router with a named-model error instead of being forwarded to a model documented to reject the minimal thinking level.

• Reject a forced tool_choice on Claude Fable 5.1 and Claude Mythos 5.1 before dispatch, instead of forwarding a request the provider answers with a 400. Both models removed forced tool use: {"type": "any"} and {"type": "tool", "name": ...} return tool_choice: type "tool" and "any" are not supported for this model., while auto and none are unaffected and disable_parallel_tool_use still works alongside auto. Claude Fable 5 and Claude Mythos 5 accept forced tool use, so the new model_rejects_forced_tool_choice gate parses the family version (>= 5.1) rather than keying on the Mythos-class name the way the disabled-thinking gate does, and every later Mythos-class minor inherits it automatically along with aliases, dated snapshots, and Bedrock-prefixed ids. The check runs on all three ingresses that can express a forced call: /v1/chat/completions streaming and non-streaming (where OpenAI's "required" and {"type": "function", ...} become Anthropic's any and tool), /anthropic/v1/messages, and the Responses-to-Anthropic converter, which unlike the disabled-thinking guard is reachable by an ordinary request because a Responses client can ask for a forced call today. /anthropic/v1/messages/count_tokens and Bedrock Converse passthrough stay ungated for the same reasons they are exempt from the disabled-thinking guard. The 400 names the model, echoes the spelling the client actually sent rather than the Anthropic translation of it, and lists the documented remedies (auto plus an instruction naming the tool, strict: true on the tool, or structured output); the router applies none of them, because downgrading a forced choice to auto lets the model answer with plain text while the caller still believes a tool call was guaranteed, and that failure surfaces far from the request that caused it.

• Add selection_strategy: EngineLoad, which ranks the whole eligible candidate set on engine-reported queue depth instead of only the holders of a prefix key (#1581, follow-up to #1447). The routing.engine_load scoring term reaches selection only through the KV overlap scorer, so a fleet with prefix routing off, or a cold prefix, collected engine statistics that never influenced a routing decision. The strategy ranks on engine truth only when every eligible candidate has a fresh waiting_requests statement and the spread clears the balance dead band; otherwise it defers to the new hot-reloading routing.engine_load.base_strategy and records why (strategy_stale_fallback, strategy_hysteresis_hold). Requiring a statement from every candidate is deliberate, because ranking a subset would send every request during an imbalance to whichever backends publish statistics and starve the ones that do not. Naming the strategy is the opt-in, so it does not read routing.engine_load.enabled; config validate warns instead when engine_stats.enabled is false, and base_strategy: EngineLoad is a load error with a second guard at the point of use so a programmatically built pool cannot recurse either. One shared comparison now serves both the strategy and the scorer term, which also makes the winner of an exact tie deterministic instead of dependent on HashMap iteration order.

• Feed engine-side kv_cache_usage and waiting_requests into smart-routing load assessment, so tier degradation reacts to GPU saturation (#1578). Router-local counters cannot see it: a backend whose KV cache is nearly full still accepts connections and still passes its health probe, and router latency only moves once the queue has already built. smart_routing.load_management.thresholds.{warning,critical} and any tier_thresholds override gain both fields, each opt-in per field, with kv_cache_usage range-checked to 0.0 through 1.0. Values are folded by maximum across backends with a fresh snapshot rather than averaged, because averaging would let three idle backends hide the one saturated backend the thresholds exist for. Freshness reuses the existing EngineStatsStore definition, so a deployment without engine_stats never fires the new thresholds and a stalled poller degrades them to silence rather than to a stale verdict. smart_routing_load_transitions_total gains a reason label drawn from a closed four-value set, and GET /admin/smart-routing/load-state, the status endpoint that embeds the same snapshot, and the WebUI panel now show the three engine numbers a degradation decision was made on, rendering absence as null rather than as a false zero.

• Serve semantic response-cache hits behind the Continuum Hub optimization gate (#1580). The hub decision has computed semantic_cache since #887 with nothing consuming it; the serving path adds a bounded in-process vector index, one embedding call per eligible request to an operator-pinned backend, and a third /v1/chat/completions lookup after the exact and prefix paths miss and after the guardrail input gate. Partitioning is the security boundary: the digest folds in the caller's identity, the model, the embedding model, every response-shaping field, and every message before the final user turn, so two API keys, system prompts, or tool sets never share an entry. response_cache.semantic is always compiled and validated whenever present, with backend required when enabled because vectors from two embedding implementations are not comparable. The lookup runs after the guardrail input gate so a blocked prompt is never sent to the embedder, the embedder call is bounded by embedding_timeout_ms and degrades to a miss on any failure, and the whole path is skipped before any embedding call when neither the operator threshold nor the hub similarity names one. Hits answer X-Cache: HIT with X-Cache-Mode: semantic and count under continuum_response_cache_semantic_total{result} and continuum_response_cache_semantic_entries. No new dependency, and the embed feature graph stays green.

Changed¶

• Behavior: Resolve the /v1/realtime model alias through the same exact-alias rule the chat, Anthropic Messages, and Responses ingresses use (proxy::alias_dispatch) instead of realtime's own fuzzy pipeline (#1589). resolve_audio_model used to run find_matching_config_slice's full six-phase matching (date-suffix stripping, quantization and format peeling, HuggingFace prefix stripping, wildcard aliases) and checked the metadata cache before backend model_configs, the opposite of the documented metadata priority; a realtime request for a fuzzy spelling such as a date- or quantization-suffixed id could therefore resolve to a different model than the identical spelling reached on the chat path, or resolve at all when the chat path would reject it. Only an exact id or exact alias, or a models/-prefixed spelling of one, now resolves, and capability lookup follows the same backend-model_configs-before-metadata-cache order as canonical_catalog_id. The per-request retry that only fired when literal candidate lookup came back empty is replaced by proxy::alias_dispatch's catalog-backed rewrite (canonical_catalog_id plus decide_dispatch_model), which reads the live catalog rather than the configured models: allowlist; this also fixes a configured-but-unserved alias, where a backend lists the alias in models: alongside the canonical id but live discovery answers only the canonical, a case the old retry could never reach because the configured alias always produced a candidate on its own.

• Behavior: Apply the alias-to-canonical dispatch rule on the remaining ingresses that pick a backend from a client-supplied model name (#1590). /v1/embeddings and its native Gemini multimodal sub-path, /v1/images/generations, /v1/images/edits, /v1/images/variations, /v1/responses/compact, and /anthropic/v1/messages/count_tokens each looked the raw name up and forwarded it verbatim, so gemini-embedding-2-preview (an alias of gemini-embedding-2) or nano-banana-2 (an alias of gemini-3.1-flash-image) reached Google as a 404 while the same alias worked on chat. Each now runs the shared proxy::alias_dispatch::dispatch_model_for_request after its own per-key gate and before backend selection, and carries the requested and dispatched names as a pair so a 404 or 403 raised after the rewrite still quotes what the client sent. A name any backend serves literally is still never touched, and no rule about which aliases resolve changed. Two consequences are worth knowing: /v1/embeddings now runs its per-key model check at the ingress rather than leaving it to the dispatch funnel, so the canonical id can be added to a non-empty allowed_models list only after the requested name was admitted; and the image edit and variations endpoints check their fixed list of accepted model names before the rewrite, so an alias reaches them only when it is itself one of those names. ACP stays literal-only, as its usage and architecture documents promise.

• Behavior: Answer a string or null in error.code on every OpenAI-shaped error body and SSE error event instead of repeating the HTTP status as an integer (#1610). The OpenAI Error schema types code as anyOf [string, null], and openai-python builds the field with construct_type, whose loose coercion hands an int straight through under an Optional[str] annotation, so err.code == "insufficient_quota" silently never matched and err.code.startswith(...) raised on a body the router considered valid. ErrorDetail.code is now Option<String>, filled from one table in src/errors.rs that maps every RouterError variant to a stable snake_case code (model_not_found, insufficient_permissions, rate_limit_exceeded, invalid_api_key, invalid_request, request_too_large, upstream_error, service_unavailable, timeout, internal_error), and a provider's own string code wins over the table whenever the upstream body carried one, which is how malformed_function_call (#1592) now reaches the client as error.code on both the non-streaming body and the streaming error event rather than only inside the message text. The status was and remains on the HTTP status line. Every route that used to format its own body (the admission 503, both rate limiters, the Responses handler, the auth refusals, the Chat Completions SSE error event) now builds it through one shared envelope, so message, type, param, and code are all present on every /v1 error with param and code serialized as null rather than omitted; the Responses error stream event takes the flat ResponseErrorEvent shape the spec describes, with code, message, param, and sequence_number beside type instead of nested under an error object. A client that parsed error.code as a number has to read the HTTP status line instead.

• Breaking (behavior): Accept xhigh as an output_config.effort value on the /anthropic/v1/messages ingress instead of treating it as unknown. xhigh is one of the five levels the Anthropic Messages API defines and the one Claude Code sends by default, but neither ingress transform listed it: transform_thinking_with_effort in src/http/handlers/anthropic/transform.rs and map_effort_string in src/http/handlers/anthropic/responses_transform.rs both matched max, high, medium, and low and sent everything else down the _ arm, which logs Unknown output_config.effort value and substitutes high. A client that asked for xhigh had its request downgraded one level, the only trace a warning in the router's own log, and on a backend whose vocabulary excludes high the downgrade turned a serviceable request into a 400. Observed against a vllm backend serving a Qwen3.8 FP8 template that accepts xhigh, medium, and low:

POST /anthropic/v1/messages  {"thinking":{"type":"adaptive"},"output_config":{"effort":"xhigh"}}
DEBUG Unknown output_config.effort value effort="xhigh"
DEBUG Mapped output_config.effort to reasoning_effort anthropic_effort="xhigh" reasoning_effort="high"
400 Invalid request sent to backend

With the arm added the same request maps xhigh to xhigh and returns 200. Both transforms now map xhigh to xhigh, the spelling every other level already uses. The _ arm and its warning are unchanged, so a genuinely unrecognized string still falls back to high, and the max to xhigh promotion is unchanged. Migration: a backend whose vocabulary lacks xhigh now receives xhigh where it used to receive high, and rejects it if it validates the field: vLLM chat templates that accept only low/high/max, for example, answer 400. This surfaces a mismatch the substitution used to hide, and the fix is to send the level that backend accepts, or to give it a model entry whose vocabulary the router already fits against. Targets that are fitted per model are unaffected: Gemini still downgrades xhigh to high through validate_reasoning_effort, and a cross-provider fallback hop still fits the level to the selected target as #1530 left it.

• Bound the model-list fan-out tail so one unreachable backend no longer costs every refresh its full retry budget (#1553, the follow-up that #1548 and #1551 deferred). ModelFetcher::fetch_all_models awaits one future per backend, so a refresh costs as much as its slowest backend, and a backend that never answers /v1/models spent (max_retries + 1) × request_timeout + max_retries × retry_delay on every refresh: 16 s at the defaults, the unit every convergence bound in #1548 and #1551 was denominated in. The fetcher now keeps a small per-backend failure memory across refreshes, shared through an Arc because the fetcher is cloned into every spawned revalidation, and a backend whose previous refresh failed with a transient error (a timeout, a connection failure, a 5xx) is probed with a single attempt at request_timeout on the next one. Measured with three fast backends and one that accepts connections and never answers: the first refresh still costs 16.02 s, every refresh after it 5.00 s. The per-attempt timeout is deliberately unchanged, so a backend that answers again is restored to the full budget on its first success, within one refresh of its recovery, and a slow-but-working backend cannot be locked out by a timeout it can never meet. A backend with no history always gets the full budget, so the first refresh of a process and the first refresh of a newly added backend are never cut short. When every backend fails in one refresh, the backends probed once get the rest of their budget before the result is stored, so a fleet-wide blip is not published as an empty list and such a refresh costs the same 16 s as before. A cut-short backend is an ordinary transient failure: it lands in failed_backends, counts in model_transient_errors_total, keeps its last-known models through the #1422 retention, and is not marked unhealthy. Unix-socket backends honor the same budget, the Admin discovery entry points keep the full one, and the Slow model fetch from backend line now also carries budget_attempts. There is no new configuration field; the threshold is one failed refresh.

• Tighten the ordering contract between a pool membership change and its cache invalidation, and bound what a superseded model refresh costs (#1552, follow-up to #1548). The store-time membership compare was not serialized against a pool mutation the way the epoch compare is, so a backend added between that compare and the insert would have published the pre-add list as Fresh for a full soft TTL; the store now re-reads the membership generation after the insert and expires the entry in place through the new ModelCache::expire_entry, which neither re-runs the store nor bumps the invalidation epoch. Hot reload invalidates the model cache directly after mutating the pool and before its immediate health-check round trip, so both stamps move before any await and one reload reaches an in-flight fan-out as one supersession instead of two, which saves a full fan-out per reload; the reorder is safe because the fan-out filters on the config enabled set and never on health state, and /v1/models availability is computed per request. The in-place retry of a superseded revalidation releases the aggregation lock between passes and backs off 200 ms doubling per retry with jitter, so each contiguous hold is exactly one fan-out (a POST /v1/models/refresh or a reader past the stale-serve bound queued behind a pass gets the lock next rather than after the whole chain), and it re-checks the cache on re-acquire so a list another task published in the gap is not aggregated again. A chain superseded on all three passes leaves the entry expired and holds the next revalidation for a 2 s re-arm interval, which turns sustained pool churn into bounded bursts of fan-outs rather than an unbroken chain; the interval never applies to a first supersession, whose follow-up still starts at once, and a Fresh publish from any path lifts it. model_background_refreshes_total is incremented per fan-out again, including in-place retries, so dashboards that read it as a fan-out count are right under supersession; model_background_refresh_successes_total and _failures_total stay one per revalidation and both metrics pages say so.

Fixed¶

• Apply fallback.fallback_policy.fallback_timeout_multiplier to every fallback hop's own timeout (#1615). The knob was parsed, range-validated to 1.0-5.0, exposed in the Admin schema, stored on FallbackPolicy, shipped at 1.5 in three configuration templates and config.yaml.example, and documented as scaling an attempt's own timeout, but no request path read it: calculate_timeout had no non-test caller, so every hop ran with the primary attempt's window and an operator who raised the value got nothing. The attempt number the fallback executor already keeps is now threaded to the four drivers (the executor closure gains it as a third argument, IngressAttempt carries it, and the pre-stream connection loop derives it from its own hop counter), and each scales its base through one shared helper: the primary is attempt 1 and unchanged, hop n runs with base * multiplier^(n-1), clamped to the boot-pinned timeouts.limits ceiling for that timeout class so a chain cannot reach past a value the configuration itself could not have set. The base is whatever the attempt would have used without a chain: the per-model timeouts.request.standard.total (or the image-generation total) on the OpenAI-compatible non-streaming path, the per-attempt streaming window on the pre-stream connection phase, the per-model profile total plus the first-SSE-byte deadline on /anthropic/v1/messages, and the flat standard or streaming total on /v1/responses. timeouts.connection, the wait for a fallback dial permit, a within-hop retry, and a mid-stream hop taken after the client already holds the stream are deliberately not scaled. The scaling composes with timeouts.streaming_fallback_budget_multiplier rather than multiplying with it, because the streaming connection phase scales the hop's window first and then caps it at what is left of the shared chain budget. Deployments that left the knob at its 1.5 default now give a first hop half again its previous window, which is the documented behavior they were already configured for.

• Fold the hub key id into the response-cache caller identity so two hub keys never share an exact or prefix cache entry (#1579). Under api_keys.mode: permissive a hub-keyed request passes local auth without an AuthContext, so the local cache identity was the empty string for every hub key on a router and the exact and prefix caches shared one namespace across all of them: key B was served key A's cached deterministic completion, and an entry stored by key B answered key A. The semantic cache added in #1571 already folded the hub key id into its partition; the exact and prefix paths on /v1/chat/completions (streaming and not), /v1/responses, and /anthropic/v1/messages did not. The caller half of a cache namespace is now one type, proxy::cache_identity::ResponseCacheCaller, which every ingress must build from the presented token and the hub decision before it can mint a cache key, so the fold cannot be skipped by a handler added later, and the semantic cache consumes the same identity instead of resolving its own. A hub-keyed request whose token no longer resolves to a hub key is not cached at all rather than falling into the shared namespace. A local key and an anonymous permissive caller keep their identity byte for byte, so single-tenant deployments see the same cache keys as before; a hub key that local auth also recognizes (blocking mode) gets the id folded on top, which costs it one cold cache after the upgrade.

• Refuse to forward Gemini's MALFORMED_FUNCTION_CALL turn as a success, and pin every Gemini finish_reason to the OpenAI set (#1592). Google intermittently answers HTTP 200 with an assistant message carrying neither content nor tool_calls, completion_tokens: 0, and a finish reason of its own, measured on 2026-09-10 as one call in two and one in five on gemini-3.1-pro-preview with a single tool. The client received {"finish_reason": "function_call_filter: MALFORMED_FUNCTION_CALL", "message": {"role": "assistant"}}, which breaks an SDK switching on the five documented values, loops an agent on a tool turn with nothing in it, and violates the Chat Completions shape by omitting content. Both spellings (bare on the native API, behind a function_call_filter: label on the OpenAI-compatible endpoint) are now recognized by one table shared by map_gemini_finish_reason, the two streaming paths, and the non-streaming response transform, and the outcome becomes a retryable upstream failure through CoreError::upstream_status (502, type: upstream_error, a message naming malformed_function_call), so the retry loop and any configured fallback.fallback_chains hop engage; a retry succeeded in every observed case, while answering a clean stop would hide a turn that produced nothing and still cost prompt tokens. On a streaming request the failure is detected after the response is committed, so it arrives as an SSE error event rather than a status. Two shape guarantees now hold on every Gemini route: finish_reason is always one of stop, length, tool_calls, content_filter, and function_call, because the shared table reduces anything unrecognized to stop rather than passing it through, and an assistant message always carries a content key, null only alongside tool_calls and empty otherwise.

• Run the Gemini chat-completions transform on the non-streaming route, not only when stream: true (#1591). The request transform lived on the streaming dispatch alone, so the same /v1/chat/completions request reached Google with extra_body.google.thinking_config.include_thoughts: true and max_completion_tokens: 16384 when it asked for a stream and byte-identical to the client body when it did not: a non-streaming client got no reasoning_content from a thinking model, ran into Gemini's low implicit output cap, and had reasoning_effort: "xhigh" forwarded instead of downgraded to high, while an unsupported effort was a router 400 only on the streaming half. The proxy path now runs transform_request_gemini followed by strip_non_openai_fields, the pair and the order GeminiBackend::transform_request uses, before the context-cache and cache-salt injections whose extra_body output the top-level strip never touches. Enabling the injection alone would have leaked markup, because Google returns the thought summary inline in message.content, wrapped in <thought>...</thought> and marked extra_content.google.thought: true, and only the streaming transformer unwrapped that; the response side now moves that text into message.reasoning_content, leaves content holding the answer alone (empty rather than absent when the model produced nothing but thought), drops the consumed marker while preserving a sibling thought_signature, and shares one tag-stripping helper with the stream transformer instead of a second copy of the literals. Which models receive the injections is now a family table with a reduction step rather than six contains checks: the substring list matched gemini-3.1-pro-preview but not gemini-3.1-pro, so after the alias rewrite in #1583 the canonical name a caller is most likely to send was the one that got nothing. Both transports are covered, and the newer Flash families are deliberately not in the table, because adding one changes what those deployments already receive.

• Fit reasoning_effort: "none" on a fallback hop whose target model always reasons, instead of forwarding it into a provider error the chain then counts as a failed hop. The ingress gate added with rejects_reasoning_effort_none judges the model the client asked for; a hop lands on a different model chosen by the operator's chain, whose metadata can carry the flag when the original did not. Reproduced with a chain from a dead backend onto gemini-3.6-flash: the hop forwarded none, Gemini answered 400 Request contains an invalid argument., the chain treated that as a failed hop, and the client received 500 An internal server error occurred. naming neither the model nor the field. The hop now drops the effort and logs it, and the same request is served. The effort is fitted rather than the hop refused because a hop has already substituted the model, so a client that accepted a different model over a failure is better served by that model reasoning than by an error. The primary attempt is never fitted, since the ingress gate has already answered for it, and no other effort level is touched. One consequence is worth knowing: the substitute reasons, so a small max_tokens that fit a non-reasoning answer can now truncate one.

• Answer the client in the model name it sent when an alias was dispatched under its canonical id (#1588). PR #1583 rewrites an exact metadata alias to the id its backend serves before backend selection, and each ingress overwrote the only copy of the requested name, so two client-facing surfaces started naming a model the client never asked for: the X-Original-Model fallback header reported gemini-3.1-pro-preview for a request that sent gemini-3.1-pro, and a 404 or allow-list 403 raised after the rewrite quoted the canonical id too. The three rewriting ingresses (/v1/chat/completions, /anthropic/v1/messages, /v1/responses) now carry both names as a DispatchModel pair: selection, the fallback chain lookup, the cache key, and the payload keep using the dispatched id, while the fallback attribution and any model-not-found or not-permitted error are built from the requested name, on the streaming arms as well as the non-streaming ones. X-Fallback-Model still names the model that served, and the response model field is still whatever the provider answered. Nothing in the alias resolution rules changed.

• Turn the Gemini reasoning_effort: "none" gate from an allowlist compiled into the router into a deny list carried in model metadata, and apply it on the /v1/chat/completions ingress. The predecessor refused none for any model its table had not heard of, which reliably meant "released after the last router build": Google shipped four Gemini Flash minors between May and September 2026, and each one lost none support until someone measured it and cut a release. The new rejects_reasoning_effort_none metadata key inverts that. An unlisted model is forwarded and the provider decides, and the flag ships set only on the ids measured to refuse: gemini-3.6-flash, gemini-3.5-flash-lite, and the Gemini Pro tiers. Because it is ordinary metadata, a model-metadata.d/ drop-in can set or clear a row the day a provider changes its mind. The check now runs at the chat-completions ingress before backend selection, so it covers streaming and non-streaming and the previously ungated proxy path, and it answers with a 400 naming the model and the field: for the two Flash tiers the upstream body is only Request contains an invalid argument., which names neither. validate_reasoning_effort keeps vocabulary validation and the xhigh and auto normalizations but no longer decides model support, so GeminiBackend carries no model table at all.

• Key the Gemini thinking-disable gate on the exact model family instead of a name substring, and correct the set it holds. supports_thinking_disable decided reasoning_effort: "none" with five contains checks, which was wrong in both directions: gemini-3.5-flash-lite matched the gemini-3.5-flash substring and was forwarded to a provider that refuses it, while gemini-3.7-flash and gemini-3.8-flash matched nothing and were refused locally though the provider accepts them. Each id in the shipped catalog was measured against the OpenAI-compatible endpoint on 2026-09-10 with one reasoning_effort: "none" request: 2.5 Flash, 2.5 Flash-Lite, 3 Flash, 3.1 Flash-Lite, 3.5 Flash, 3.7 Flash, and 3.8 Flash answer 200, while 3.5 Flash-Lite and 3.6 Flash answer 400 INVALID_ARGUMENT, and the 2.0 family answers 404 because it is retired upstream and is carried unverified so a proxy still serving those names behaves as before. Support follows neither the version order nor the Flash to Flash-Lite relationship, so the gate is now a measured table keyed by a family reduction that keeps a -lite segment and drops a version, date, or preview suffix. The router's own 400 for an unsupported none now names the families that accept it. The gate is consulted by the typed Gemini backend paths (execute_chat_completion, execute_streaming_request, and the control-plane probe) and by the Anthropic-ingress payload transform, which drops an invalid effort with a warning rather than answering 400. A request that reaches Gemini's OpenAI-compatible endpoint through the proxy path still carries the effort through untouched and is answered by Gemini itself, which was measured on the same date. The native thinkingConfig.thinkingLevel: "MINIMAL" field is a separate axis with the opposite answer on the two models that differ (3.8 Flash refuses it, 3.6 Flash accepts it), which is recorded next to the table so a documentation statement about minimal is not used to extend it again.

• Assert the offending field from the message rather than the finding path in the routing.engine_load.base_strategy self-reference test (#1581). A loader validation error reaches config validate through loader_validate_config, which files every such error under the generic config path, so the path assertion pinned a bucket name rather than the diagnostic. The test was added against a base that predated the change which routes this error that way, and it has been failing on main since, blocking every pull request that touches src/.

• Match a multi-word classifier keyword when its words appear in order within one sentence with at most two tokens between them, instead of only when they sit adjacent exactly as the table lists them (#1603). count_matches was a plain substring test, so natural phrasing that inserts a modifier in English ("write a short story") or a particle in Korean ("시를 하나 써줘") missed the marker, the request fell through to multilingual or general, and every policy keyed on that domain missed it too. Lengthening the keyword lists cannot fix this, because enumerating every modifier and every particle placement does not terminate, so the rule sits in the matcher and applies to every table at once. The adjacent spelling is still tried first through the same substring test, so no keyword that matched before stops matching, and a single-word keyword keeps exactly its previous behavior. A sentence terminator (a period, an exclamation mark, a question mark, the ellipsis character, a newline, or the fullwidth CJK forms) closes the window, so words on opposite sides of one never form a match however close together they sit. Each word must line up with a whole token, punctuation at the token edges ignored, with one exception: the last word may be followed by more characters in the same token when it ends in a non-Latin letter, which is 써 matching 써줘. Confining that exception to the last word is what keeps the 시 in 시 한 편 from matching 시스템, the collision KO_CREATIVE was written to avoid, and Latin script never takes the exception at all, or how do i would fire on "how do you implement". Because the rule sits in count_matches, the simple, complex, creative, and analysis tables, both built-in languages, operator keywords under smart_routing.classifier.rule.keywords, and any language table added later all inherit it, which makes it operator-visible semantics and it is documented as such. Measured on the labelled dataset, domain accuracy moves from 94.6% to 97.3% overall and from 88.9% to 94.4% on Korean, with complexity unchanged.

• Raise rule-classifier complexity accuracy from 62.2% to 69.5% by correcting the scoring mechanisms behind its misses (#1605). Complexity measured 62.2% while domain sat at 94.6%, and since routing_policies[].when.complexity selects a model tier, that signal was choosing tiers correctly about three times in five. The simple-keyword term was inverted: it contributed (1 - strength) * 0.1 against a weight of 0.2, an effective ratio of 0.15 to 0.30 that sits at or above the trivial boundary, so adding a keyword that means "this is easy" raised the score, and an identical greeting read trivial in English and simple in Korean purely because the Korean one carried a simple keyword. It now has the same ratio * weight shape as every other term, with a ratio that falls as the signal strengthens. The labelled dataset grows from 37 to 59 cases and COMPLEXITY_ACCURACY_GATE rises from 0.55 to 0.65 against the measured 69.5%; per language, English moves from 68.4% to 73.3% and Korean from 55.6% to 65.5%. Domain reaches 100.0% on the larger dataset, which is the combined effect of #1603 and #1604 landing first rather than of this change. The remaining misses all under-read, so the residual bias is one-directional and recorded rather than papered over.

• Dispatch a model alias as the canonical id its backend serves on /v1/chat/completions, /anthropic/v1/messages, and /v1/responses, so a request for gemini-3.1-pro reaches Google as gemini-3.1-pro-preview instead of failing (PR #1583). model-metadata.yaml has declared gemini-3.1-pro as an alias of gemini-3.1-pro-preview since #594 because Google serves only the -preview id, and the sample config.yaml lists both names under the Gemini backend; but backend selection matched the requested name literally against the configured models: list and dispatched that same name, so a client asking for gemini-3.1-pro was selected onto the Gemini backend through the configured-name fast path and forwarded verbatim, which Google answers with 404 models/gemini-3.1-pro is not found for API version v1main and the router surfaced as a 400. Verified against the live API on 2026-09-10: Google's catalog still lists only gemini-3.1-pro-preview and gemini-3.1-pro-preview-customtools for the 3.1 Pro family, and the same request through the router now returns 200. The new proxy::alias_dispatch module applies one rule from one place, and the rule is deliberately narrow: the requested name must be an exact alias of a different canonical id (the fuzzy date-suffix, quantization, prefix, and wildcard phases of the metadata pipeline are not consulted, so a pinned snapshot such as gpt-5-2025-08-07 is never silently served by gpt-5), every enabled backend that lists the requested name in models: must have been enumerated into the aggregated live catalog (a backend whose discovery is cold or failing contributes nothing, and its silence is not evidence), no backend may serve the requested name in that catalog, and a user-routable backend must serve the canonical id in it. The catalog is consulted rather than the configured models: lists because a configured name is an operator allow-list entry, and in the motivating case it is exactly the entry that is wrong. A name any backend serves literally is never rewritten, so a local engine that lists unsloth/Qwen3.6-35B-A3B-GGUF keeps receiving that exact id; a request nothing can serve keeps its original name and fails as it did before. The rewrite runs after each ingress's per-key allowed_models gate and request_params policy snapshot, both of which keep judging the name the client asked for, and the allow-list handed to the dispatch funnel is extended with the canonical id so its re-check cannot turn the admission into a 403; it is skipped for Hub-attributed requests, requests under a Hub exact-model budget guard, and AppProxy ingress-pinned requests, which already carry a model decision. A fallback.fallback_chains entry keyed on the alias no longer fires once the alias is dispatched as its canonical id; the router logs a warning at request time when it meets one, and the fix is to key the chain on the canonical id. The upstream answers with the id it served, so an alias request now shows the canonical id in the response model field, which is how OpenAI itself answers a floating name with the pinned snapshot it resolved to. The embeddings, image, Responses compaction, ACP, and realtime paths are unchanged. ModelAggregationService::catalog_snapshot is the new read-only catalog accessor behind the decision: it answers from cache, serving stale data with a background revalidation once it has aged, and pays a blocking aggregation only in a process that has never aggregated. tests/model_alias_dispatch_test.rs pins each clause on the chat path (non-streaming and streaming), the Anthropic Messages path, and the Responses path by reading the model a mock upstream received, together with a key scoped to the alias, a failing discovery endpoint, and a date-suffixed spelling.

• Report a routing validation error under its field path in continuum-router config validate and the MCP validate tool, so routing.engine_load.base_strategy: EngineLoad is filed as an error on routing.engine_load.base_strategy rather than under the generic config path (PR #1583). The loader validates the section with its Validate impl and fails fast with one routing: <errors> string; the report now renders the identical text with the flattened field paths attached, so the fail-fast gate deduplicates it and the operator sees which field to fix. This also makes the a_self_referential_base_strategy_is_an_error unit test that #1581 added pass, which it did not on main.

• Make Hub-managed kv_routing reach request routing without a restart, and stop KV ingestion from losing or misattributing cache transitions (#1560). The section shipped as immediately reloadable, but the live index, the event consumers, and the KV scorer were each fixed at process startup, so an apply changed the stored configuration and nothing else: a rollout, a source change, a scoring change, and a rollback all needed a restart to take effect. The index and its consumers are now materialized or torn down live, the named scorer is replaced atomically, and startup seeds itself from a restored Hub snapshot. Four ingestion defects go with it: an empty-token offload lost its identity, a prefix held by several chains was evicted by whichever chain ended first rather than refcounted, a reused block hash was not evicted before its reuse, and a source sequence was committed before the records it covered had been processed, so a failure mid-processing skipped them. Event delivery now applies backpressure instead of dropping cache transitions, and a consumer whose listener fanout has fallen behind closes the SSE stream and reconnects with Last-Event-ID so the retained history is replayed rather than lost. A salt_echo identity that is malformed, missing, or issued for another backend is treated as unattributable instead of falling back to detokenization, and event-source endpoints are redacted in runtime error logs.

• Enforce non-empty API key scopes on OpenAI-compatible and Anthropic API routes in api_keys.mode: blocking, so a scopes: [read] key can list/read models and stored outputs but can no longer drive chat, completion, embedding, rerank, image, Responses create/delete, model refresh, batch mutation, or realtime inference routes (#1557). Empty or absent scopes remains unrestricted for backward compatibility, write implies read, admin implies both, files controls only the separate Files API group, and permissive mode still never rejects a request solely because a presented key has narrow scopes. The route-class table is now source-audited against the build_api_routes inventory so newly added /v1 routes must choose a scope class instead of silently inheriting all-or-nothing authentication.

• Publish a backend added by hot reload while a model-list fan-out is already in flight within the remaining fan-out plus one fresh fan-out, instead of hiding it for soft TTL plus one fan-out (#1548, reported from backend.ai-go#4812). ModelFetcher::fetch_all_models snapshots the pool at fan-out start, and a hot reload that landed during the 16 s that one slow backend costs every refresh mutated the pool and expired the cache without touching the running fan-out, so that fan-out finished with its pre-add membership and stored it as a Fresh entry. The healthy-event revalidate_now and every request-path spawn_revalidation in that window coalesced into the in-flight refresh and were dropped, GET /v1/models/{id} answered 404 from the stale list, and nothing re-aggregated until the soft TTL expired: 79.7 s measured against 1.24.0 with the defaults and one 8 s backend, against 16.5 s when the add lands between fan-outs. Each aggregation now records two stamps immediately before it snapshots the pool. BackendPool::membership_generation is bumped inside the write critical section of add_backend, remove_backend, and drain_backend on their success paths only, which covers hot reload, the Admin API, the AppProxy registry, and control-plane backend sync because all of them funnel through those three. ModelCache::invalidation_epoch is bumped by clear_cache() and ModelCache::clear, which also catches a models: allowlist edit, an auth change, or an enabled toggle that invalidates the cache without changing membership. At store time a result whose stamps moved is written through the new ModelCache::set_expired, or refused by ModelCache::set_if_epoch_unchanged, whose epoch compare and insert share one lock with the invalidation so nothing can slip between them; either way the result is published as an already-expired entry, so readers inside the stale-serve bound from #1324 still get the previous list and the next reader revalidates instead of coalescing into a Fresh hit. The follow-up does not wait for that reader: a background revalidation re-aggregates in place, bounded at three passes so a burst of Backend.AI GO model registrations cannot pin the task, and the request path and POST /v1/models/refresh release the aggregation lock and spawn one background revalidation before answering with the list they produced. The stamps are read before the snapshot on purpose, since reading them afterwards would reintroduce the race, and the code says so. An unchanged pool costs nothing new: the stamps differ only when something changed under the fan-out, and the #1324 suites still count exactly one fan-out per refresh. A superseded refresh is logged at info with the reason and both stamp values, on a line separate from the Model refresh: N models in D pair that #1326 made scrapeable, and counted in the new model_superseded_aggregations_total. The regression suite gates the mock backend so the pool mutation provably lands inside the fan-out on every run rather than on the runs where a sleep happened to line up. The fan-out tail itself, where one unreachable backend costs every refresh 16 s, is unchanged and left for a follow-up.

• Emit tool_calls[].index as the 0-based ordinal of the tool call within the response on streaming /v1/chat/completions, instead of the upstream content-block or parts position (#1546). The Anthropic converter keyed its per-call state by the Anthropic content block index and reused that same number on the wire, so any thinking or text block before the first tool_use shifted the numbering: a turn that began with one sentence streamed its two tool calls as index 1 and 2 rather than 0 and 1. OpenAI defines the field as a slot in the tool_calls array and clients treat it literally, so @ai-sdk/provider-utils built a sparse array and every tool-using turn died at end of stream with TypeError: Cannot read properties of undefined (reading 'hasFinished'), which looked intermittent because turns without a tool call were unaffected. The transformer still keys its state by the block index, which is the only identifier content_block_delta and content_block_stop carry, but assigns each tool_use block a sequential ordinal at its start event and emits only that, in the opening chunk and in every input_json_delta chunk, so interleaved argument deltas for parallel calls stay on their own index. The Gemini native converter had the same defect in two forms: it used the position of the part inside one event's parts array, so a leading text part shifted the first call, and it kept no state across events, so two calls delivered in two events both reported 0 and the client merged the second into the first. It now counts tool calls with a per-stream counter cleared by reset(). Both fixes match the rule the Bedrock Converse converter has followed since #909. The Anthropic change reaches the native Anthropic backend, the HTTP and Unix-socket streaming paths, and the Bedrock InvokeModel-style paths (bedrock-mantle and bedrock-runtime), which reuse the same transformer. Non-streaming responses are unchanged, since their tool_calls array carries no index.

• Classify Korean requests by intent instead of collapsing them into domain = multilingual (#1567). The rule classifier's keyword lists were English only, so no intent signal ever fired for Korean text and the non-ASCII ratio check was left as the strongest domain signal on anything without a fenced code block. Length compounded it: the token estimate divided the UTF-8 byte count by four and a Hangul syllable is three bytes, so a short Korean greeting measured over the 10-token trivial boundary while its English translation measured under it. A new language module holds one detector, a script-weighted token estimate, and per-language keyword tables, with English and Korean built in, every table including the English keywords, and an unrecognized language falling back to English. determine_domain now takes the strongest intent signal and consults multilingual only when none fired, and ClassificationResult carries the detected language. Two optional hot-reloading fields under smart_routing.classifier.rule, primary_language and per-language keywords, complete parity for a request that carries no intent signal at all, such as a greeting. Leaving both unset preserves every pre-existing domain assignment, and the admin classify diagnostic and the WebUI playground now report the detected language.

CI¶

• Add a labelled Korean and English classification dataset with a runner that reports rule-classifier accuracy and confusion matrices per mode, gated in CI (#1568). tests/data/smart_routing/dataset.json labels requests across every DomainTag and ComplexityLevel combination, labelled independently from the doc-comment definitions in src/services/smart_routing/types.rs rather than copied from classifier output, so the measured accuracy is a real regression signal rather than a tautology. tests/smart_routing_accuracy_test.rs runs every case through classify_only for the rule_default and rule_primary_ko deployment modes, prints accuracy and a domain and complexity confusion matrix per mode and per detected language, and asserts a gate per mode. LLM and hybrid modes are out of scope, since they need a live or mocked backend round trip rather than a fast deterministic check. docs/en/development.md and its Korean mirror document how to extend the dataset. No CI configuration changed: .github/workflows/ci.yml and scripts/local-ci.sh already run cargo test --tests -- --skip integration_test, which compiles and runs every top-level tests/*.rs target under default features.

• Exempt the realtime metadata lookup from the backend candidate filter audit (#1602). every_backend_candidate_set_is_filtered_or_explicitly_exempt had been failing on main since model_config_for_canonical_id was introduced in src/proxy/realtime.rs. That function is not a dispatch path: it walks enabled backends' model_configs to find the ModelConfig declaring an already-resolved canonical id so the realtime handshake can read that model's audio capability, and it returns Option<&ModelConfig> rather than a backend set, which is structurally identical to the already-exempt alias_dispatch::canonical_catalog_id. No production code changed, and the visibility guarantee the audit protects is still enforced where a backend is actually chosen: the realtime handshake builds its dispatch candidates separately through find_backends_for_model and applies filter_user_routing_candidates, so a model served only by an internal: true or enabled: false backend is still refused.

Documentation¶

• Describe the semantic response cache as implemented (#1616). The Scope notes in docs/en/architecture.md and its Korean mirror still said the semantic cache path was a deliberate no-op with no embeddings client and no tenant-isolated vector-similarity index, and that CacheHitType::Semantic was reserved with no serving path. #1571 built that path, so the text had been wrong since it merged, and the usage-record paragraph repeated the same claim about the semantic value of cache_hit_type. Both paragraphs now describe what ships, drawn from docs/en/architecture/kv-cache.md and config.yaml.example rather than restated from memory: the opt-in third path consulted only after the exact and prefix lookups miss, its three gates, cosine similarity against threshold or the hub policy's semantic_similarity_bps, the X-Cache-Mode: semantic header, and the bounded per-router index with oldest-first eviction and no external vector database. src/services/smart_routing/load_monitor.rs is also split into a directory module (assessment.rs, state.rs, tests.rs) with no behavior change.

Dependencies¶

• Refresh the Cargo dependency graph, moving roughly three dozen crates to their newest semver-compatible versions and pulling in core_detect, multiversion, multiversion-macros, and multiversion_no_op as new transitive dependencies of the language-detection path. The repository diff is limited to Cargo.lock.

• Bump the Cargo minor-and-patch dependency group with five updates (PR #1556): tower-http 0.7.0 to 0.7.1, lru 0.18.3 to 0.18.4, toml 1.1.4 to 1.1.5, aws-smithy-types 1.6.2 to 1.6.3, and rmcp 3.1.4 to 3.2.0. The resulting repository diff is limited to Cargo.lock.

v1.27.0 - 2026-09-03¶

Twenty-three commits since v1.26.0 closed credential-forwarding gaps, added Basic authentication for self-hosted engines, and carried cross-provider reasoning intent and answered upstream status through fallback. They also rejected backend endpoint URL forms deprecated in v1.26.0 and bounded fallback dials per backend.

Added¶

• Convert an Anthropic thinking config into reasoning_effort on a hop toward an OpenAI-wire target instead of only stripping it (#1518, the residual #1517 left behind). Since #1517 every OpenAI-wire send site removes speed, thinking, and output_config keyed on the selected backend's configured backend_type, which is fail-safe but lossy: a request that asked for extended thinking hopped onto a rescue backend running with that target's default reasoning behavior, and the client's intent was discarded without a trace. The same dispatch-side gate now translates immediately before it strips, on exactly the requests that strip, so no send site can strip without converting and no call site changed. output_config.effort maps level for level, with max becoming xhigh where the target's vocabulary has it and high where it does not; thinking: {"type": "enabled", "budget_tokens": N} maps back through the existing band table (<= 4096 low, <= 10240 medium, anything larger high), which is the inverse of ReasoningEffort::to_budget_tokens rather than a second vocabulary; {"type": "enabled"} with no budget maps to medium; adaptive thinking with no effort and {"type": "disabled"} both emit nothing, disabled winning even when an output_config effort is present; and anything malformed degrades to a plain strip, never to a failed hop. A client-supplied reasoning_effort, flat or nested as reasoning.effort, always wins and is never overwritten. Because the conversion runs at the send site, after the reasoning validation that guards a client-supplied value, the emitted level is fitted to what the selected target documents: openai and azure consult the OpenAI model table and receive nothing at all when the model supports no reasoning_effort (gpt-4o, the chat and instant tiers, an Azure deployment name the table does not recognize), and otherwise take the closest allowed level on the low < medium < high < xhigh ladder, preferring the next higher level on a tie, so low toward gpt-5-pro becomes high and low toward gpt-5.2-pro becomes medium; every other OpenAI-wire target, the self-hosted engines and Gemini included, receives low, medium, or high and never xhigh. Local engines are included deliberately: reasoning_effort is the spelling the router already keeps out of every strip because every target either consumes or ignores it, the inbound /v1/messages handler has fabricated it toward these same engines since it shipped, and cloud Gemini's own transform maps it to thinking_level. none, minimal, auto, and max are never emitted. The guard is an end-to-end suite that asserts on the outbound body a mock backend actually received, non-streaming and streaming, and five of its cases were verified to fail with the conversion removed.

• Add auth.type: basic for self-hosted engine backends (#1476). The credential is the username:password pair in api_key, split on the first colon per RFC 7617, and the router sends Authorization: Basic base64(username:password). This is the supported destination for a credential that used to be embedded in backends[].url: the bytes on the wire are identical, because reqwest produced exactly this header by stripping URL userinfo in RequestBuilder::new, so a reverse proxy sees no change. What changes is that the credential moves out of backends[].url, which the configuration maskers deliberately leave readable and the WebUI renders, into api_key, which they mask. Accepted on vllm, sglang, ollama, lmstudio, llamacpp, and mlxcel only, and a load error on every other type including generic, whose factory arm can fall back to a backend that carries no credential at all: accepting it there would drop the credential silently. api_key is required and must contain a non-empty username before the colon, so a malformed credential is refused at load rather than encoded into a header that authenticates as a bare username and fails against the proxy on every request. An unresolved ${VAR} is accepted and re-checked at startup after interpolation.

Changed¶

• Breaking (behavior): Strip client-supplied credential headers on every chat-completions path (#1523, from the security review of #1519). The OpenAI-wire transports forwarded every client header they did not recognize as a connection header, and suppressed the client's authorization only when the selected backend happened to carry a credential of its own, so x-api-key, api-key, cookie, set-cookie, proxy-authorization, x-auth-token, and x-access-token always reached the backend, and authorization reached it whenever the backend was configured without a key. Because the router's own auth middleware accepts x-api-key as a bearer-equivalent credential, the very header a client uses to authenticate to the router was replayed upstream to a third-party backend operator; and because the mid-stream fallback loop resolves the backend secret once per hop precisely so a hop can switch providers, provider A's client credential rode along to provider B by construction. All four forwarding loops now drop credential headers unconditionally through one shared is_credential_header predicate in src/infrastructure/common/secure_header.rs: the streaming builder build_openai_chat_request_core (which serves the initial dispatch, the mid-stream per-hop loop, and auto backend selection), the streaming Unix socket builder, the non-streaming make_http_request, and the non-streaming Unix socket builder. The predicate reads the same list log redaction reads, so stripping and redaction cannot drift apart, and x-goog-api-key joins that list. A backend's credential now comes only from its own api_key, its auth block, or the hub-delivered provider secret, and the strip is unconditional, so nothing is re-evaluated per hop. Migration: a backend configured without any credential now sends no Authorization header at all rather than relaying the caller's, which used to work by accident of the old condition; set api_key on that backend entry instead, and the symptom of an unmigrated one is a 401 from the upstream on every request to it. Non-credential headers such as user-agent, x-request-id, and tracing headers still forward unchanged, so this is a deny list and not a whitelist; the native Anthropic, Gemini, and Bedrock dispatch paths and the Responses and Realtime paths already behaved this way; and the two Responses header filters now take the credential half of their deny lists from the same predicate without losing a single name. The guard is an end-to-end suite that reads the headers a wiremock backend actually received and asserts by value as well as by name, since reqwest appends rather than replaces and a name-only assertion would pass while the client's value rode along as a second entry; six of its seven tests were verified to fail when the predicate is disabled.

• Breaking (behavior): Extend the client-credential-header strip to the Anthropic Messages, count_tokens, image, and multimodal embedding paths (#1527, from the security review of #1524). Issue #1523 fixed the four chat-completions forwarding loops, but five sibling loops kept the pre-#1523 shape: the Anthropic Messages HTTP builder build_request_with_auth and Unix socket builder build_unix_socket_headers in src/http/handlers/anthropic/handler.rs, build_count_tokens_request in src/http/handlers/anthropic/count_tokens.rs, and the two duplicated loops in handle_streaming_image_generation in src/proxy/image_gen.rs. On those paths a client authorization or x-api-key was suppressed only when the selected backend happened to carry a credential of its own, and cookie, set-cookie, api-key, proxy-authorization, x-auth-token, and x-access-token were forwarded whatever the backend's configuration. That mattered most on the Anthropic surface, because the Anthropic SDK authenticates with x-api-key and the router's own auth middleware accepts that header as a router credential, so the very header a caller uses to reach the router was replayed to a backend configured without a key of its own. All five loops now call the same unconditional is_credential_header predicate the chat paths use, and two more join them. The image variations loop: its openai-* / x-* allow list matched x-auth-token and x-access-token by prefix, so both reached the provider, and its narrower local contains("api-key") heuristic is replaced by the shared deny list. The native Gemini multimodal embedding dispatch behind /v1/embeddings (handle_multimodal_embedding in src/proxy/handlers.rs): it named authorization and x-goog-api-key in the same matches! as the connection headers and forwarded every other credential name to Google, including the x-api-key a caller authenticates to the router with. The two image-generation loops are now one forward_client_headers helper shared by the initial dispatch and the OAuth 401 retry builder, so the copies cannot drift. backend_has_config_auth had no caller left after this and is removed. Migration: the same note as #1523, now on these paths too. A backend configured without any credential receives no client credential at all on /anthropic/v1/messages (HTTP and Unix socket), /anthropic/v1/messages/count_tokens, /v1/images/generations, /v1/images/variations, and a multimodal /v1/embeddings request to a Gemini backend; set api_key on that backend entry, and the symptom of an unmigrated one is a 401 from the upstream. user-agent, x-request-id, anthropic-version, anthropic-beta, and every other non-credential header still forward unchanged. The guard is a new end-to-end suite that drives the real handlers against wiremock backends and asserts by value as well as by name, plus a keyless Unix-socket variant; every assertion was verified to fail when the predicate call is removed from its loop.

• Breaking (configuration): Reject backend endpoint URLs that embed credentials, carry a query string or fragment, or name a transport the router cannot dial (#1476, the phase-2 promotion of #1457). These forms warned for one minor release, v1.26.0, and are now refused at configuration load by startup, hot reload, the Admin backend API, and hub config sync, and reported as errors with a non-zero exit by continuum-router config validate. The message names the backend and the offending property and echoes no part of the URL, because the credentials case is exactly the one where echoing would leak a password into every surface that renders a ConfigError. The same promotion applies to the base rule on engine_stats.metrics_url, whose scheme, same-host, and no-downgrade constraints were already hard errors. The transient probe and the hub-managed paths are unchanged: they always rejected these forms, and the one asymmetry the deprecation window left behind is now gone, so a passing candidate probe is a truthful preview of what saving the backend does. Migration for a credentialed URL is auth.type: basic above; for a type: generic backend that relied on one, name the specific engine type instead.

• Unify the budget_tokens to reasoning_effort bands between the /anthropic/v1/messages ingress and the cross-provider fallback hop mapper (#1532, the follow-up #1530 asked for). Three observable changes for clients on this endpoint whose request lands on an OpenAI-compatible backend: a budget_tokens in 2049..=4096 now maps to low where it used to map to medium, one in 8193..=10240 maps to medium where it used to map to high, and a thinking: {"type": "enabled"} with no budget_tokens maps to medium where it used to map to high. The ingress carried its own older band table and the Responses API transform used for responses_only models carried a second copy of it, so the same request could receive a different reasoning level depending on which door it entered through and whether it was rescued onto a fallback backend. Both ingress transforms now resolve a budget through ReasoningEffort::from_budget_tokens, the one band table shared with the hop mapper from #1518 and #1530, which reads its thresholds from ReasoningEffort::to_budget_tokens and is therefore its exact inverse, so the forward and reverse directions cannot drift apart again. The hop mapper's own bands are unchanged, and an output_config.effort still takes precedence over the budget on every path.

• Breaking (API): SecureHeaderValue::bearer is now module-internal (pub(in crate::infrastructure::common)), and SecureHeaderValue::bearer_for_provider_fixed_scheme is the public escape hatch that replaces it (#1529). SecureHeaderValue is re-exported from a public module, so this is a visible change for an embedder. A caller that built the header for a backend credential switches to SecureHeaderValue::for_backend_auth(auth_type, secret) or holds a BackendCredential and calls its authorization_value(), either of which honors auth.type: basic; a caller that carried a credential whose scheme the provider or the protocol fixes (an OAuth access token, a cloud provider key, a guardrail endpoint key) switches to the new name, whose rustdoc states that contract and whose every in-tree call site is listed in tests/authorization_scheme_audit_test.rs with a written reason. Nothing on the wire changes: the new constructor produces the same Bearer <token> value the old one did, and SecureHeaderValue::basic and for_backend_auth stay public.

Fixed¶

• Carry answered upstream HTTP statuses through typed Gemini and Bedrock chat-completion dispatches instead of flattening them into connection failures (#1539). Native streaming now treats non-429 4xx responses as terminal provider answers, lets 429 and 5xx responses enter the same configured fallback.fallback_policy.trigger_conditions.error_codes gate as OpenAI-wire handshakes, preserves parsed provider error detail and answering-backend attribution, and keeps refused connections on the existing connection_error path. Non-streaming Bedrock Runtime and Converse requests use the same classifier, so an answered 400 remains HTTP 400 and does not count as a circuit failure instead of becoming a retryable 502. Generic typed HTTP dispatches now produce the same dependency-free status carrier, while transport-only executor errors remain unchanged.

• Restart an engine-statistics poll task when a hot reload changes the backend's effective authentication scheme (#1521). The task fingerprint now derives the secret and resolved scheme from the fused backend credential, so switching only auth.type replaces the task and the next scrape uses the new header while an absent auth block and explicit auth.type: api_key remain equivalent.

• Normalize configured backend base URLs by removing redundant trailing slashes before health checks, model availability, and routing build their shared identity (#1492). This fixes backends that logged healthy yet remained unavailable and returned 503 after a trailing-slash URL was pasted into the WebUI or supplied through file, hot-reload, Admin API, or programmatic configuration; intentional path prefixes are preserved.

• Route a native-typed primary's pre-stream failure through the streaming fallback chain (#1531, the observation the PR #1525 body recorded). A stream: true request whose selected backend is type: anthropic, type: gemini, or a type: bedrock runtime endpoint and which fails before the provider answers (a refused connect, a transport error, a handshake timeout, or a circuit rejection) answered HTTP 502 service_unavailable even with fallback.enabled: true and a chain pointing at a healthy rescue backend, because stream_chat_completions_inner returned the native dispatch result before the fallback dispatch decision was computed; the identical fixture with a type: generic primary fell back and served a 200 SSE stream. The native arms now report a failure with no provider answer as a value instead of a response, and the pre-stream connection phase dials whatever wire the current backend speaks on every attempt, so a native primary enters the same walk an OpenAI-wire primary enters, in both fallback.mid_stream_enabled modes, under the same trigger_conditions and max_fallback_attempts, with the same X-Fallback-* headers (x-fallback-reason: connection_error and x-fallback-attempts: 2 for a one-entry chain), the same per-backend dial bound, and the same cross-attempt budget: the Anthropic arm takes the capped attempt window on its request timeout, and the Bedrock runtime and Gemini arms, whose trait seam has no timeout parameter, bound the handshake wait with the same window and report an elapsed one as a timeout trigger. The same change ends the terminal native hop #1522 documented: a chain entry resolving to a native backend now advances to the next entry when it fails before answering, so [dead native entry, live entry] serves from the live entry with x-fallback-attempts: 3, and a saturated dial bound on a native hop advances on backend_unhealthy like any other hop. What does not change: with no chain a dead native primary still answers 502 service_unavailable with no SSE shell, a dead primary and a dead rescue exhaust into identical 502 bodies in both modes, an answered non-2xx from an Anthropic-shape native backend never advances the chain (that pipeline reads the upstream status itself and still relays an upstream 5xx as a 200 SSE stream), a native stream still has no mid-stream recovery, a native primary over a Unix socket stays outside the walk, and a ValidationError, ConfigError, or AuthError from a native primary returns unchanged without consuming an attempt. The observability context now survives a failed native attempt, so the stream that eventually serves still records its token usage. The guard is eight end-to-end cases that drive the real handler with stream: true against dead anthropic, gemini, and Bedrock runtime primaries and mock rescues on both wires; every one was verified to fail on the previous main with the 502 service_unavailable signature.

• Give fallback dials a real concurrency bound (#1522). The guard against a thundering herd of fallback dials onto one rescue backend was a hardcoded, router-wide tokio::sync::Semaphore of 50 permits with a 5 second acquire timeout, acquired on exactly one of the four fallback dispatch paths (the mid-stream relay's post-commit hops) and held across the whole rescued stream rather than the dial, so its real meaning was "at most 50 concurrent fallback-served streams router-wide", an availability cliff during the exact outage it was meant to smooth, while the pre-stream connection phase, the pre-stream-only mode, the native-protocol hop, and the non-streaming funnel had no fallback-specific ceiling at all, and the only test that named it constructed a local semaphore and never touched router code. It is replaced by fallback.fallback_policy.max_concurrent_dials_per_backend (default 50, 0 = unlimited, validated range 0..=10000), a per-backend bound owned by the fallback service and shared by all four paths. The permit covers the dial only: it is taken immediately before a hop's outbound request is sent, before the circuit-breaker admission so a saturation timeout never consumes a half-open probe slot, and released as soon as the provider handshake returns, never across the body. Only hops are bounded; the primary attempt and a selection-time walk onto a fallback model are not, because there the rescue backend is effectively the primary. A saturated backend queues the dial FIFO for at most the effective timeouts.connection (and what is left of the streaming chain budget), then fails the hop for that backend as backend_unhealthy so the chain advances or exhausts with the preserved upstream failure; the non-streaming Unix-socket and Bedrock runtime transports hold the permit for the whole call because they expose no handshake seam, and a saturated native-protocol hop is served as that hop's failure because the native dispatch is terminal (since #1531, a saturated native-protocol hop instead advances on backend_unhealthy like any other hop). The limit reloads immediately with a documented transitional window, and saturation timeouts are counted in fallback_dial_bound_saturated_total{backend} and as dial_bound_saturated in streaming_fallback_total. The guard is a concurrency suite that drives the real handler with six concurrent clients against a mock rescue backend that withholds its handshake until released, on every path, plus a two-chain no-starvation test and an unbounded positive control; with the acquire removed the bounded tests observe six dials in flight and fail. The vacuous test_semaphore_limits_concurrent_fallbacks is deleted.

• Honor the backend's configured auth.type at every remaining credential-carrying site, and make the next hardcoded Bearer a build failure (#1529). auth.type: basic (#1476) reached the executor, the health probe, model discovery, the non-streaming HTTP and Unix socket paths (#1505 family), the streaming builders and the Anthropic inbound transform (#1528), and the OAuth strategy on the streaming builders (#1534), and each of those rounds found its sites by reading code rather than by a failing test. Ten sites remained, and a basic-auth engine received Authorization: Bearer user:password on every one of them: the /v1/responses conversion path for OpenAI-wire backends (ResponsesApiStrategy::ConvertToChatCompletions), non-streaming and streaming alike, /anthropic/v1/messages/count_tokens (which wrote format!("Bearer {..}") straight into the header and so bypassed log redaction as well), the ACP session/prompt dispatch, the startup connection pre-warm, the hub-directed synthetic probe, the realtime WebSocket upgrade, and the streaming image generation, image variations, and image edit routes. All ten now build the value from backends[].auth, so a backend with no auth block or with auth.type: api_key sends exactly what it sent before, a backend with auth.type: basic sends RFC 7617 Basic, and the Anthropic x-api-key branches and the OAuth gates are untouched. The image variations and image edit routes are included because their backend filter accepts a backend whose name or URL merely contains openai, so a generic backend named openai-proxy reaches them today and generic accepts auth.type: basic; the filter itself is unchanged. Each of the ten is pinned by a test that reads the Authorization off a real listener with a Bearer control beside it, and each was verified to fail when its wiring is reverted. Two guards replace the manual enumeration: SecureHeaderValue::bearer is module-internal, so a new site outside the seam's own module is a compile error, and tests/authorization_scheme_audit_test.rs fails the build on any remaining spelling, including the format!("Bearer ..") one no visibility change can reach, unless the site is listed with a written reason.

• Decide whether the Anthropic-native fields speed, thinking, and output_config may reach the wire at dispatch, keyed on the selected backend's configured backend_type, rather than at the fallback hop, keyed on the providers inferred from the two model names (#1517). The hop-side table added in #1512 got the answer wrong in both directions. When both model names were operator-chosen aliases, each inferred Provider::Unknown, the table read that as a same-provider hop needing no hygiene, and all three fields rode through to an OpenAI-wire backend, where a strict cloud endpoint answers HTTP 400 and kills the very hop that exists to rescue the request. In the other direction a claude-* primary hopping onto an Anthropic-typed backend behind a custom alias looked like a provider change, so thinking was dropped by the one layer with no way to know the target would have honored it. Model-name inference cannot distinguish an aliased Anthropic backend from an aliased vLLM backend, which is the same class of defect #1512 removed elsewhere; only dispatch knows which backend serves a request. field_filter::strip_anthropic_native_fields_for_backend_type is now the single enforcement point and runs at every OpenAI-wire send site: the non-streaming HTTP and Unix-socket passthrough branches, the streaming request core (covering direct streaming, the mid-stream per-hop loop, and auto backend selection), the streaming Unix-socket path, and the typed Gemini transform. Every OpenAI-wire type strips (generic, openai, azure, gemini, vllm, ollama, llamacpp, mlxcel, lmstudio, sglang), matched exhaustively so a new backend type forces a decision; anthropic and bedrock keep the fields because their transform is what reads them, and speed there stays governed by the unchanged anthropic_fast_mode opt-in; continuumrouter keeps them because a downstream Continuum Router understands the same canonical extension fields and applies this rule against its own backends; and a backend with no configured type keeps full passthrough rather than being guessed at. reasoning_effort and extra_body are never touched, so a client's reasoning intent still crosses every hop. The hop-side PROVIDER_ONLY_PARAMETERS table is removed along with TranslationResult::removed_parameters, leaving the hop with one job, the model-name swap, and TranslationResult::translated now reports true only when that swap actually happened instead of unconditionally. The guard is an end-to-end suite that asserts on the outbound body a mock backend received, streaming and non-streaming, and it was verified to fail on current main with the three fields present on the vLLM-typed target. A self-hosted engine configured as type: generic (the default when type: is omitted) that previously read a top-level thinking, speed, or output_config field stops receiving them after this change; extra_body remains the escape hatch for engine-specific settings.

• Route a streaming fallback hop through the typed per-backend dispatch instead of forcing it onto the OpenAI SSE relay (#1513, field report #1493). ensure_openai_streaming_relay_target was applied as a blanket pre-dispatch guard, so a native Anthropic, Gemini, or Bedrock backend was rejected with backend type '<type>' is unsupported by the OpenAI SSE fallback relay whenever a fallback chain existed for the requested model. Three failures followed from one guard placement: a streaming request whose primary refused the connection could not hop onto a healthy Anthropic backend, a selection-time chain walk that had already resolved onto one was rejected before reaching the native branch that would have served it, and merely configuring a chain for a model whose own primary is native answered HTTP 400 to every streaming request to that model, healthy direct routing included. A hop now re-selects a backend and re-enters the same typed dispatch direct routing uses, keyed on the configured backend_type, so a native target is served through its own pipeline (for Anthropic, the Messages API at /v1/messages with x-api-key and anthropic-version) while the client keeps receiving OpenAI-format SSE; the guard survives inside build_openai_streaming_attempt_payload, the one place the relay wire is actually about to be used. In the default mid-stream mode the SSE response is now committed at the first successful provider handshake rather than before any backend is contacted, which is what lets a pre-stream hop reach any backend type and carry the X-Fallback-* headers per notify_on_fallback. Three consequences ride along there: pre-stream exhaustion returns a proper HTTP error instead of a 200 stream carrying an error event, pre-stream hops are governed by fallback.fallback_policy.trigger_conditions and fallback.fallback_policy.max_fallback_attempts rather than the relay's own retryable-error check, and no SSE keep-alive comments flow while the router is still connecting, matching what the pre-stream-only and no-chain paths always did. That silent connection window is bounded by the same cross-attempt budget the relay enforces (timeouts.request.streaming.total scaled by timeouts.streaming_fallback_budget_multiplier, 20 minutes on stock defaults): each handshake attempt is capped at what is left of it and an exhausted budget answers with the preserved upstream failure, so the phase can never hold the client socket for max_fallback_attempts + 1 full per-attempt timeouts. The relay never discards a handshake the phase obtained, and the stream it takes over is measured from the moment of handover rather than from the start of the connection phase. Once SSE output has begun the relay speaks only the OpenAI wire, so a chain entry resolving to a native-protocol or Unix-socket backend is skipped without consuming a fallback attempt and the next entry is tried, never a request-wide validation error. The guard is an end-to-end suite that drives the real handler with stream: true against a mock Anthropic Messages backend and asserts both sides of the wire; it was verified to fail with the historical unsupported by the OpenAI SSE fallback relay signature when the old guard is restored.

• Keep a cross-provider fallback hop's payload in canonical OpenAI chat-completions form instead of pre-translating it toward the target's wire format (#1512). ParameterTranslator ran on every hop whose model differed from the original, guessed the target provider from the model-name prefix, and converted the payload toward that provider's native shapes; dispatch then performed the real conversion keyed on the backend's configured backend_type, assuming canonical input. Two format deciders in one path produced a hybrid neither understood, and an OpenAI to Anthropic hop was damaged three ways: pre-converted tools lost their type: "function" wrapper, so dispatch's second conversion skipped every entry and sent tools: []; a pre-converted tool_choice became {"type":"auto"}, which dispatch rejects, so any client that sent tool_choice got a failed hop instead of a fallback; and stop renamed to stop_sequences was never read by the Anthropic transform, which reads stop. The hop now swaps the model name and, on a cross-provider hop, drops only provider-only fields the target does not own, currently the Anthropic-only speed, thinking, and output_config fields, which a non-Anthropic dispatch path would otherwise forward verbatim to the wire; none of the three belongs to the OpenAI chat-completions schema or is covered by the cloud field filter, and a strict cloud endpoint answers HTTP 400 for an unrecognized top-level key. Message, tool, tool_choice, and field-name translation, the Gemini-native renames (maxOutputTokens, topP, stopSequences) that were dead keys against the router's OpenAI-compatible Gemini endpoint, the top_k to top_logprobs and topK to top_logprobs mappings that turned a sampling parameter into a logprob-reporting knob, and the redundant max_tokens default are all removed. Removals toward Anthropic that dispatch already made on its own (frequency_penalty, presence_penalty, logprobs, logit_bias, n, seed, response_format) are gone too, because the hop's provider guess is wrong for claude-* models on Bedrock, Gemini behind its OpenAI-compatible endpoint, and gpt-prefixed open models on vLLM, and dropping a parameter there loses it for a backend that would have accepted it. The guard is an end-to-end test that drives the real handler from a failing OpenAI-compatible mock to an Anthropic-typed mock and asserts the outbound /v1/messages body; it was verified to fail on all three defects when the fix is reverted. Streaming relay behavior is unchanged and tracked separately in #1513.

• Apply the configured auth.type on every credential-carrying request path, not only the executor path (#1505). auth.type: basic was added in #1476 and wired into HeaderBuilder, which serves RequestExecutor, but proxied inference, Unix socket dispatch, health checks, model discovery, and the engine-statistics scrape each build the Authorization header inline and hardcoded Bearer. A backend configured with basic therefore sent Bearer username:password everywhere that mattered, and because health_checks.enabled defaults to true it failed its own health probe, was marked Unhealthy, and never received traffic at all. The five paths now resolve the header through one seam, SecureHeaderValue::for_backend_auth, which takes the configured scheme as a required argument so a new call site cannot silently inherit the wrong one; the auth scheme travels beside the credential through BackendHealthCheckInfo, the model-discovery credential map, and the engine-stats target. Backend-type auto-detection is deliberately left on Bearer and carries a comment saying why: it runs only for type: generic and the hot-reload detection round, and basic is a load error on generic, so no basic credential can reach it until #1504 gives that type a credential path. A backend with no auth block or with auth.type: api_key is byte-identical to before on every path. The guard is a behavior test that reads the header off a real listener and was verified to fail when the fix is reverted; the previous unit test pinned the encoder, which was never the broken part.

• Accept auth.type: basic on a transient Admin backend probe candidate (#1503). validate_candidate_auth refused the scheme alongside service_account and sigv4, so POST /admin/backends/probe could not check the configuration #1476 tells operators to migrate to: an operator whose config was refused at load had no way to verify the rewrite short of applying it. The refusal was a conservative placement made while #1476 was in flight rather than a capability gap, because both probe paths already apply the scheme, the health probe through BackendHealthCheckInfo.auth_type and the catalog probe through fetch_models_from_backend, both wired in #1505. service_account and sigv4 stay refused, since each needs lifecycle-managed material a transient candidate cannot load. Candidate validation still runs the whole-configuration validator over a single-backend config, so the backend-type restriction and the username:password shape rule are inherited rather than duplicated, and credential_status already reported basic truthfully because candidate_static_key_was_applied excludes only the three lifecycle schemes.

• Give a type: generic backend a credential path, and accept auth.type: basic on it (#1504). The generic factory arm attempts auto-detection and, when nothing matches, falls back to a backend built from a BackendInfo, which carries name, url, weight, supported_models, metadata, and enabled and no credential of any kind; HttpBackend contained no Authorization construction at all. A configured api_key on a generic backend was therefore accepted and never sent. The type worked against an authenticating endpoint only because reqwest turned backends[].url userinfo into a Basic header inside the HTTP client, and #1476 made that URL form a load error, which left type: generic with no way to authenticate at all. HttpBackend now holds the resolved Authorization value, produced once at construction through the shared SecureHeaderValue::for_backend_auth seam (#1505), and applies it on all four of its request paths ahead of caller-supplied headers so an explicit header still wins. Backend-type auto-detection is routed through the same seam, over both HTTP and Unix socket transports, which closes the one site #1505 deliberately left on Bearer. A generic backend with no credential sends no Authorization header, unchanged.

• Honor the configured auth.type on the streaming OpenAI-wire request builders and on the /anthropic/v1/messages transform path (#1528). The #1505 family moved the health probe, model discovery, the executor chat path, and the non-streaming HTTP and Unix socket paths onto the scheme-aware SecureHeaderValue::for_backend_auth seam, but six credential sites were left behind: four in src/http/streaming/handler.rs and two in src/http/handlers/anthropic/handler.rs, all hardcoding SecureHeaderValue::bearer. A backend configured with auth.type: basic and api_key: user:password therefore received Authorization: Bearer user:password on every streaming chat completion, which is the main path for chat clients, and on every Anthropic-API request routed to an OpenAI-compatible engine, while the health probe fixed in #1505 kept reporting the backend Healthy, so the reverse proxy answered 401 exactly where most traffic goes. All six now build the header through one new pub(crate) helper, crate::proxy::oauth_helper::backend_authorization_value, which is the same call src/proxy/backend.rs makes, so no site in either file names a scheme of its own: the streaming request core (serving the initial dispatch, every mid-stream fallback hop, and auto backend selection), the streaming Unix socket builder, the two Bedrock-mantle streaming branches, and the Anthropic Messages HTTP and Unix socket transform builders. Because the core reads the hop's own backend_config, a mid-stream hop re-evaluates the scheme per hop rather than inheriting the primary's. No signature changed, the hub-secret substitution from #888 is untouched, and a backend with no auth block or with auth.type: api_key is byte-identical on the wire before and after. The Bedrock-mantle branches always resolve to Bearer, since basic is a load error on bedrock, and go through the helper only so the scheme is never named at a call site. The guard is a new end-to-end suite that drives the real handlers and asserts on the bytes a wiremock backend or a Unix socket listener actually received, reading each SSE body to the end so a header assertion cannot pass on a request that never completed; each Basic assertion was verified to fail when its site is reverted to SecureHeaderValue::bearer. Separately verified and not fixed here: these streaming builders apply no registered OAuth strategy, so a backend declaring auth.type: oauth is dispatched with its static key or with nothing at all, unlike the non-streaming path, which is tracked in #1534.

• Apply the registered OAuth strategy on the four streaming request builders, and stop sending the static key when auth.type: oauth is declared (#1534, the gap #1528 verified and left open). build_openai_chat_request_core (serving the initial dispatch, every mid-stream fallback hop, and auto backend selection) and stream_via_unix_socket looked up no strategy at all: a backend declaring auth.type: oauth was dispatched with whatever static api_key it carried as a plain Bearer, or with no Authorization when it carried none, the token was never refreshed, and the strategy's Codex extra headers (originator and the Codex User-Agent, which the ChatGPT-subscription front requires on every request rather than only on the OAuth endpoints) never left the router. All four now hold the contract make_http_request already held: with a strategy registered the token is refreshed, its whole header map is applied after the client-header forwarding loop so the two paths cannot drift on ordering, and the static key is suppressed; with auth.type: oauth declared but no strategy loaded, from a missing or malformed token store, no Authorization is sent at all rather than the static key standing in for the absent token; and otherwise the static path is byte-identical to before, which a Bearer control with an empty registry pins. The core became async and takes the caller-resolved strategy, so its two build_streaming_request call sites gained an .await and nothing else. The mid-stream relay resolves the strategy PER HOP from a registry handle moved into its spawned task, next to the effective-secret resolver and for the same reason: the relay outlives any one token, so a pre-spawn snapshot would pin the whole stream to the credential of the backend selected first and would be the wrong backend's token entirely after a cross-provider hop. The Unix socket transport builds a header Vec rather than a reqwest::RequestBuilder, so oauth_helper gained strategy_headers, the refresh-then-snapshot half that apply_strategy_headers now calls too, which is what keeps the two transports on one refresh rule. The guard is a new end-to-end suite that drives the real handlers and asserts on the bytes a wiremock backend or a Unix socket listener actually received: every OAuth backend carries a decoy static api_key, so asserting the single authorization value equals the seeded token also proves suppression, and originator is asserted alongside the bearer so a bearer-only shortcut cannot pass. Each of the seven wired sites was reverted in turn and the matching assertion observed to fail. Two decisions are recorded rather than deferred silently. stream_with_auto_backend_selection does not gain the native_stream_kind guard here: it is a library-only entry point bound to no route, the only correct guard is delegation to the typed dispatch #1531 is restructuring concurrently, and the credential fix itself does land on that site. And auth.type: service_account on a type other than gemini, or sigv4 on a type other than bedrock, is now a warning in both the startup log and continuum-router config validate rather than a load error, because those combinations load today and send the static key as Bearer, so rejecting them in a fix release would break a running router; this follows the #1457 then #1476 path of warning for one release and rejecting afterwards. No rule for oauth, which is a working configuration on every OpenAI-wire path once this lands. Still open and tracked for a follow-up: the streaming counterpart of the non-streaming 401 force-refresh retry, which would have to live in connect_with_pre_stream_fallback while #1531 is restructuring it.

Documentation¶

• Correct the phase-1 claim that embedded backend URL credentials "are never forwarded" (#1476). Measured against a local listener with the router's own call shape, a backend URL of the form http://svc:pw@host/v1 with no api_key produces authorization: Basic ... on the wire, through both the shared executor and the hot proxy path, because reqwest::RequestBuilder::new calls extract_authority. With an api_key set, the Bearer header replaces it and the URL credential is silently dropped. The v1.26.0 entry below is corrected in place, and the endpoint rule sections of docs/en/configuration/backends.md and its Korean mirror now state the real reason the form is refused, which is that backends[].url is deliberately unmasked in Admin API responses while api_key is masked.

v1.26.0 - 2026-09-01¶

Twenty-three commits since v1.25.0. Six of them close a bundle of credential-disclosure defects that one redaction review opened up: a backend URL embedding inline credentials was written verbatim to the log on the ordinary success path at 67 sites, echoed by unsupported-scheme transport errors and by invalid-URL configuration errors, and returned by the Admin backend probe, while a Redis connection URL's password was returned by GET /admin/config/full and reachable through Debug. Epic #1448 finishes with engine-native statistics collection, engine-load-aware backend selection, and type: sglang as a first-class backend type, and five defects in KV-cache-aware selection are repaired alongside it. Backend endpoint URLs that embed credentials, carry a query string or fragment, or name a transport the router cannot dial are deprecated and will be rejected at configuration load in v1.27.0. Admin configuration mutations that refuse a candidate now answer 400 instead of 200.

Added¶

• Add engine-load-aware backend selection behind a new routing.engine_load section (#1447, the last sub-issue of epic #1448). With routing.engine_load.enabled: true, the KV overlap scorer (#1444) gains an additive term that prefers, among a request's prefix holders, the backend whose engine reports fewer waiting_requests (normalized by total_slots when every fresh candidate states one, otherwise by the request candidate set's maximum waiting_requests, so mixing a bounded and an unbounded basis cannot invert the ranking on a mixed llama.cpp/vLLM fleet) and lower kv_cache_usage, sourced from the engine-stats snapshots collected by #1446. A staleness guard (max_staleness_intervals, default 3 polling intervals, re-applied to an already-held snapshot on every hot reload and every poller task restart, not only on the next successful scrape) drops a candidate to its base score once its snapshot ages out, and matches the base scoring exactly when every candidate is stale (stale_fallback). A two-threshold hysteresis dead band (balance_abs_threshold 64, balance_rel_threshold 1.5, mirroring the SGLang Model Gateway's balance gates) holds the term still unless the fresh waiting_requests spread exceeds both (hysteresis_hold), so two backends whose queues oscillate around each other cannot flap selection. An optional saturation admission hint (routing.engine_load.admission, default off) refuses selection with a retryable 503 that participates in fallback.fallback_chains when every healthy candidate reports a fresh kv_cache_usage above kv_usage_threshold (default 0.98), throttled to at most one WARN log line per 60 seconds per pool so a genuine saturation event stays readable instead of flooding the log. Decisions are counted in the new routing_engine_load_decisions_total{backend,reason} series (engine_load, stale_fallback, hysteresis_hold, admission_reject), with a Grafana panel and an EngineLoadAdmissionRejecting Prometheus alert added to monitoring/. The whole section defaults to off, so selection is byte-identical to a build without it; every field's range is enforced on every path that can change a running router (startup, hot reload, Admin API PUT/PATCH), not only the whole-Config validation chain no production caller invokes, closing a path where an out-of-range admission threshold could have refused traffic fleet-wide; a low admission threshold that would shed ordinary traffic and an enabled term with no data source or no scorer to run in are flagged as config validate advisories. The engine-load term's influence is bounded by engine_load_weight (default 0.3, additive on top of the base weights that sum to 1.0), the prefix-holder-only scope the base scorer already enforces, and the hysteresis dead band; the admission hint can only refuse a request for which every healthy candidate is saturated. Documented in docs/en/load-balancing.md and docs/ko/load-balancing.md, including the trust model for self-reported engine figures.

• Collect engine-native statistics from self-hosted serving backends (#1446). A new always-compiled engine-stats layer polls each engine's own load or metrics endpoint on a jittered per-backend interval: vLLM GET /metrics (accepting both bare and _total counter names and the hidden V0 gpu_cache_usage_perc fallback), SGLang GET /v1/loads?include=core (an object wrapper with one entry per DP rank, whose omit_defaults wire format makes a missing number a truthful zero) with /get_load and /metrics fallbacks, llama.cpp GET /slots and GET /metrics selected from the endpoint_slots/endpoint_metrics capability booleans in /props (501 and 404 both degrade silently, per-slot n_ctx is the context figure, and mlxcel rides the same adapter), Ollama GET /api/ps, and LM Studio GET /api/v0/models. Snapshots live in a per-pool TTL-bounded store next to the in-flight tracker and surface as backend_engine_*{backend} Prometheus series (absent-not-zero, scrape failures emit only backend_engine_stats_scrape_success 0, counters mirrored with reset detection, hot-reload-removed backends vanish on the next scrape), GET /admin/backends/{name}/engine-stats, an engine_stats object in GET /admin/stats/backends, WebUI dashboard badges, and Backend::get_current_load() for vLLM-family and llama.cpp backends while a fresh snapshot exists. Configured by a new hot-reloadable engine_stats section, opt-in with enabled defaulting to false so no poller task runs and no series are emitted until an operator turns it on, plus a per-backend engine_stats block (enabled, source, metrics_url, interval); metrics_url must resolve to the same host as the backend URL unless the section's allow_external_metrics_url (default false) opts in, and an https backend's metrics_url can never downgrade to plaintext http even with that opt-in, because the backend's API key rides on the metrics fetch. Grafana panels and Prometheus alert examples ship under monitoring/.

• Add type: sglang as a first-class backend type for SGLang servers (#1445). The new BackendTypeConfig::Sglang (aliases SGLang, sg-lang, sg_lang) defaults the URL to http://localhost:30000, probes /health with a /v1/models fallback (a 503 while the engine starts or exits is treated as warming up), routes through the vLLM-family OpenAI-compatible path with full passthrough of SGLang request extensions and reasoning_content, converts /v1/responses to chat completions, and tokenizes /v1/messages/count_tokens through the engine at /v1/tokenize. The Admin API, WebUI backend list and dashboard, backend probing, control-plane usage provider (sglang), configuration assistant skill and templates (config generate --template sglang), and documentation all name the type, and the stale Admin config-schema backend-type enum now lists every variant.

Changed¶

• Breaking (API): Answer 400 Bad Request instead of 200 OK when an Admin configuration mutation refuses the submitted candidate (#1466). PUT/PATCH /admin/config/{section}, POST /admin/config/import with apply: true and dry_run: false, and POST /admin/config/apply with a config candidate and hot_reload: true used to answer 200 with the refusal carried only in a "success": false body field, so any caller keying on the status code, which is the default for curl -f and for most HTTP client libraries, read a refused configuration as an applied one and continued while the router kept serving the old configuration. The dividing line is whether the request asked to change running state, not which endpoint received it: POST /admin/config/validate is a verdict endpoint and keeps 200 for every parseable input, an import with dry_run: true or apply: false publishes nothing and keeps 200, and an apply with no candidate, an unchanged candidate, or hot_reload: false keeps 200. Response bodies are unchanged in every case, "success": false and the descriptive error/validation.errors included, so a client that already inspects the body needs no change; the WebUI routes the refusal through the same field-level error rendering it used before. Server-capability refusals ("hot reload not available") and the import size and nesting guards deliberately keep 200, since neither is a client error. Automation that raises on a 4xx now raises where it previously continued silently, which is the intended correction.

• Feed the engine-reported max_model_len from /v1/models entries into the model metadata context window (#1445). This applies family-wide to any backend whose model entries carry the field (vLLM and SGLang report it), so discovered models now expose a real limits.context_window without operator work, with limits.max_output set to the same value: max_model_len is the engine's supremum for generated tokens, while the 0 default is the reserved marker for "not applicable" and would otherwise be published as "max_tokens": 0 on /v1/models/{model}. Explicit model-metadata.yaml / model-metadata.d/ limits always take precedence, including a deliberate context_window: 0; only models with no configured limits pick up the engine value. Because the field is engine-controlled rather than operator config, it is accepted only up to the same 10,000,000-token ceiling the config validator enforces on operator-supplied limits, so a backend cannot publish a number the router would reject from its own config file.

• Publish the minimum engine-reported max_model_len across every backend serving a model id, instead of whichever backend's report survived deduplication (#1456, a Phase 4 unit of epic #1448). Both limits.context_window and limits.max_output now fold every reporting backend's value with min; previously the published number came from the first backend in pool order under the default MergeBackends strategy (the last under LastWins), and pool order moves on hot reload, on Admin API backend mutations, and on AppProxy reconcile, so removing and re-adding a backend could change the advertised window with nothing operator-visible to explain it. The minimum is the only figure the router can promise: selection may send a request to any backend in the model's backends list, and an engine-side context rejection is an HTTP 400, which appears in neither retry.retryable_status_codes nor the default fallback error_codes, so over-reporting produces a replica-dependent failure that neither the retry handler nor fallback.fallback_chains can recover from. A backend that reports nothing contributes no candidate rather than a zero, and a report of zero or above the shared 10,000,000-token ceiling is discarded on its own instead of narrowing or suppressing the others; when no candidate survives, no limits are synthesized, unchanged from before. Explicit model-metadata.yaml / model-metadata.d/ limits still win unconditionally, which is the supported way to advertise a larger window on a heterogeneous fleet. The /v1/models JSON body gains no field, and backend disagreement is logged at debug level with the model id, the chosen minimum, and the full candidate list. The operator-visible consequence: on a fleet where one replica was launched with a larger --max-model-len, the advertised window drops to the smallest replica's value, and that replica's extra capacity is not reachable through the shared model id because routing does not consider the context window, so an operator who wants the larger window advertised must either give that backend a distinct model id or set explicit metadata limits.

Deprecated¶

• Backend endpoint URLs that embed credentials, carry a query string or fragment (a bare trailing ? or # counts), or use a scheme other than http, https, or unix are deprecated in config.yaml, the Admin backend API, and every other registered-backend path, and will be rejected at configuration load starting with release v1.27.0 (#1457; the promotion to an error is scheduled as #1476). These forms have never worked correctly: request URLs are built by concatenating the request path onto the backend URL, so a query or fragment swallows the path and the backend answers 404 on every request; embedded credentials are forwarded, as an Authorization: Basic header reqwest builds from the URL userinfo, but only when the backend sets no api_key, and they sit in a field the Admin API deliberately renders in the clear (corrected in #1476, which also adds auth.type: basic as their supported home); and an unsupported scheme fails per request inside the HTTP client instead of at load with an error naming the field. During the deprecation window such a configuration still loads and serves, with a warning on the startup and hot-reload log and in the continuum-router config validate JSON report (path backends.{name}.url), each naming the backend and the offending property while echoing no part of the URL. Behind the deprecation, the endpoint rule is now a single shared predicate (validate_endpoint_url in the transport layer) used by the Admin transient backend probe, hub-authored candidate tasks, hub backend sync, hub backend-export representability, and registered-backend validation, replacing three byte-equivalent private copies whose duplication is what let the configuration path drift loose in the first place; the probe and hub error contracts are pinned unchanged by table-driven verdict tests. The same deprecation phase applies to the base rule on engine_stats.metrics_url, whose http(s)-scheme, same-host, and no-downgrade constraints remain hard errors as before.

Fixed¶

• Scope engine-load hysteresis, normalization, decision labels, and the scorer's short-lived cache to the exact live candidate set for each request. Previously, any backend tracked by the pool could open the balance gate or set the preferred-backend metric even when model visibility, health, circuit/retry state, or per-key permissions had excluded it, and a cached result for one candidate set could steer another request with the same prefix during the 100 ms cache window. Candidate names are now canonicalized into the scoring context and cache identity, so non-candidates cannot affect selection.

• Report the heartbeat inventory BackendInfo.provider from the operator's typed backend configuration instead of the constructed backend object's runtime type, the same source UsageRecord.provider already resolves from, keeping the runtime type only as a fallback for a pool backend with no live config entry during a hot-reload rename window (#1454, the first defect of epic #1448's Phase 4). This is a hub-facing behavior change for existing deployments running with control_plane.enabled: true: backends configured as ollama, lmstudio, or sglang previously reported provider vllm (all three are served by the shared vLLM backend struct); mlxcel reported llamacpp; bedrock reported anthropic (mantle/runtime endpoints) or bedrock-converse (converse endpoint); continuumrouter reported the hyphenated continuum-router; azure reported openai whenever its endpoint URL did not contain the literal substring azure, which covered every private-endpoint or custom-DNS Azure OpenAI deployment; and a generic backend auto-detected as llama.cpp reported llamacpp. Each now reports its configured type verbatim, so inventory and usage agree for every backend. Migration note for operators of the affected backend types: the wire field is a plain string and PROTOCOL_VERSION does not move, and BackendInfo.stable_id remains the accounting identity, so cost-center attribution cannot shift; but any hub-side grouping, dashboard, or policy keyed on the inventory provider string must accept the widened vocabulary (ollama, lmstudio, sglang, mlxcel, bedrock, continuumrouter, generic) before routers with this fix are deployed against it.

• Type an AppProxy replica from the coordinator's runtime_variant through the shared probe token vocabulary instead of matching the single literal vllm (#1455, the last unit of epic #1448's Phase 4). Reconcile previously accepted only vllm (ASCII case-insensitively) and mapped every other token, sglang included, to generic, so a replica served by any engine other than vLLM lost every behavior the router derives from a backend type. Resolution now runs through parse_probe_backend_type and accepts the six locally served engine types vllm, sglang, llamacpp, mlxcel, ollama, and lmstudio, case-insensitively and including the punctuation spellings sg-lang, sg_lang, llama-cpp, llama_cpp, llama.cpp, mlx-cel, mlx_cel, lm-studio, and lm_studio. An absent variant, the empty string, an unknown token, the explicit generic token, and every cloud-provider token (openai, anthropic, gemini, azure, bedrock, continuumrouter) all still resolve to generic, deliberately: an AppProxy replica is a locally served OpenAI-compatible HTTP server reached over http://kernel_host:kernel_port, and a cloud type would select the wrong auth resolver and endpoint defaults. The same change collapses two byte-identical private copies of the mapping (src/appproxy/common/reconcile.rs and src/appproxy/router/reconcile.rs) into one shared resolver, so the legacy worker path and the ROUTER frontend mode path run the same implementation. Migration note for operators running AppProxy circuits on non-vLLM engines: backend names are unchanged (appproxy-<circuit_id>-r<route_key>), so reconcile stays idempotent and the backend set does not churn on upgrade, only the type field differs. What a newly typed replica gains is the engine's own default health-check contract in place of the generic one (sglang probes /health with a /v1/models fallback, reading a 503 while the engine starts or exits as warming up; llamacpp and mlxcel probe /health with a /v1/models fallback; ollama probes /api/tags with a / fallback; lmstudio probes /v1/models with an /api/v1/models fallback), a backend construction that skips the per-replica /v1/models auto-detection probe the generic factory arm spends and with it the hot-reload type-detection round that could write a detected type back into the live configuration and propagate a second configuration change, /v1/messages/count_tokens answered through the engine's /v1/tokenize on sglang replicas as vLLM-family replicas already were instead of falling back to estimation, and participation in the top-level engine_stats poller (#1446) whenever engine_stats.enabled is true, which the global switch alone governs because AppProxy synthesizes no per-backend engine_stats override block.

• Repair five defects in KV-cache-aware backend selection and record the routing decisions the documentation already described (#1444, PR #1449). The kv_cache_index.scoring block was never wired into the scorer, which ran on hardcoded default weights; the scorer read backend load from a BackendStats field dead since #971 rather than from the pool's in-flight tracker; a backend with no entry in the index scored on load and health alone, so it could win scored selection when the index held data only for ineligible backends; and min_overlap_threshold was compared against a constant that made the check inert, so a StorageWarm-only holder passed the gate. Selection now records kv_aware, KV index fallback, and the prefix_hash/overflow/model-hash-fallback family once per selection through a single seam, dropping non-finite overlap samples so one NaN cannot permanently corrupt the histogram _sum. Two configurations that silently do less than they appear to are flagged at startup and by config validate: selection_strategy: PrefixAwareHash with prefix_routing.enabled: false, which degrades to uncapped model-name consistent hashing, and a kv_cache_index.scoring block whose gpu_tier_weight sits below min_overlap_threshold, which leaves scored selection registered but permanently inert. docs/en/architecture/kv-cache.md and its Korean mirror are rewritten against the post-fix behavior.

• Validate the KV overlap scorer's cached pass on the prefix key rather than on the TTL alone (PR #1474). cached_result_for() checked both, while score() read the cache slot directly and checked only the 100 ms TTL, so within that window a request could be scored against another request's prefix overlap and a backend holding nothing for the requested prefix could win scored selection outright, measured at 0.9 against the shipped 0.3 threshold and thereby overriding the configured selection strategy. CachedResult now stores the raw 32-byte prefix key instead of its hex form and both readers validate on it, which makes the check an allocation-free compare and leaves one predicate where there were two. The split dates to the original prefix overlap scoring implementation in PR #473 and was found during the security review of PR #1462.

• Reject non-finite kv_cache_index.scoring weights at configuration load (#1460, PR #1461). Every range check in the section compared with < or RangeInclusive::contains, and every comparison against NaN is false, so a YAML .nan passed on all five weights and .inf on both tier weights. A NaN weight makes every holder score NaN, which never wins adoption, so KV-aware selection reported itself enabled while falling through to the configured strategy on every request; an infinite tier weight makes the first such holder win permanently and stops load balancing across equally cached holders. The finiteness gate is added to the section validator and, more importantly, to validate_config_with_limits, the path that startup, config validate, hot reload, the config watcher, control-plane config sync, and the Admin config PUT/PATCH all funnel through and that had never touched kv_cache_index at all. Reaching the rejected state requires literally typing .nan or .inf, so no working deployment regresses.

• Serialize the src/metrics/kv_cache.rs tests that assert an exact metric delta (PR #1459). KvCacheMetrics::new registers clones of the module's lazy_static metrics into a test-local Registry, but prometheus metric clones share one Arc core, so a fresh registry never isolates the values and each of two test pairs was its own concurrent writer, measured at 6 failures in 12 runs for one pair. The module's single-counter lock is replaced by one GLOBAL_METRIC_WINDOW_LOCK covering every process-global metric in the module, held across the whole before-record-after window by all seven delta tests, so there is no per-test decision about which lock to take.

Security¶

• Redact inline credentials in backend URLs written to the log on the ordinary success path (#1469, PR #1472). A backend URL of the form http://svc:pw@host/v1 was written verbatim at INFO once per backend at every configuration load, once per backend on every health-status transition with no traffic at all, and again on every proxied request. reqwest lifts that userinfo into an Authorization: Basic header, so this is a working deployment shape, typically a self-hosted engine behind a basic-auth reverse proxy, rather than a misconfiguration, and the shipped default logging.level is info, so the disclosure needed no operator error. All 67 sites across 27 files now wrap the logged value in redact_endpoint; the value handed to reqwest is unchanged at every one. Redaction stays at the call site because at each of them the composed URL is simultaneously the outbound request target and the logged value, so it cannot move into compose_backend_url and its six siblings. A source-scan audit test guards site 68.

• Redact backend URLs echoed by unsupported-scheme transport errors (#1467, PR #1471). A live reproduction with an ftp:// backend URL carrying a password produced nine log lines containing that password from one boot plus one non-streaming and one streaming request, across seven modules; the client-facing body was already a static string, so the disclosure was to the log only. TransportError::UnsupportedScheme now stores the redacted value at construction rather than redacting inside its Display attribute, because the type also derives Debug and a Display-only redaction still prints the password through any {:?}. Nine call sites that interpolated the operator-supplied URL into a log macro are wrapped as well. Separately, the scheme named in the diagnostic was derived with split("://").next(), which yields the whole string when the separator is absent, so a scheme-less value named itself as the scheme; the replacement requires a syntactically valid RFC 3986 scheme and reports unknown otherwise.

• Mask Redis connection URL credentials in both configuration maskers (#1468, PR #1470). A Redis URL carries its password inline and neither masker hid it, so on a default build, where the Admin API is mounted unconditionally, admin.auth defaults to None and is treated as allow-and-warn, and the bind address is 0.0.0.0:8080, GET /admin/config/full returned the password to anyone who could reach the port. Adding url to SENSITIVE_FIELDS would have blinded backends[].url, which two in-tree tests exist to prevent, so each masker instead treats url as a secret only directly under a redis key and the name-only predicates stay byte-identical. The masked value is the existing sentinel rather than a partial redaction, because a partially redacted URL is neither a current sentinel nor a legacy shape, so POST /admin/config/import of an unedited masked export would write that literal string in as the live Redis URL.

• Redact the backend URL echoed by invalid-URL configuration errors (#1463, PR #1464). validate_backends_config interpolated the raw backend.url into its parse-failure message, so a malformed credentialed URL put the password into the startup stderr line, the hot-reload error log, the config validate JSON report, the MCP tool responses, and six Admin API response bodies together with the warn! lines those handlers emit. The message now names the backend, renders the URL through redact_endpoint, and retains the url::ParseError, which is what keeps a value that collapses to *** diagnosable. Control flow and status codes are unchanged.

• Redact the Redis URL password in Debug output (#1465, PR #1499). RedisCacheBackendConfig and RedisConfig derived a plain Debug that printed their credential-bearing url verbatim, unlike five sibling config types in the same file that hand-write Debug to elide secrets, and both are reachable through Config's own derive. No production call site formats either type with Debug today, so this closes a latent hole rather than an active leak: the risk was the next debug!(?config) or a new #[derive(Debug)] wrapper printing a Redis password in full with nothing in the code or CI stopping it.

• Redact backend URL credentials in the Admin backend probe response, and teach the log-redaction audit scan to see inline format captures (#1498, PR #1501). POST /admin/backends/{name}/check echoed the backend URL in checked_url and, through the health-check error string, in error; both are now redacted where each value is composed, which also covers the background monitor's debug! of the same error, and a unix:<path> address is reported in full rather than collapsing to ***. The durable half is the audit scan added by #1469: it blanked string literals before looking for url-suffixed identifiers, so info!("target: {backend_url}"), the idiomatic Rust 2021 spelling, was invisible to it, verified by an unredacted probe site passing with exit 0 under the pre-fix scan and failing with exit 101 under the widened one. The widened scan surfaced one genuine leak that #1469 could not see, the OAuth-backend health-check skip line, which fires on every health round for an OAuth backend; it is redacted rather than allow-listed.

CI¶

• Build the dependency versions Cargo.lock pins instead of running cargo update first (PR #1483). Every CI run resolved whatever was newest at that moment, so it verified a dependency set nobody installs and made a red result ambiguous between a real defect and a dependency that moved overnight. The float is still tested, through make update-deps while preparing a release and through the new weekly dependency-float.yml workflow, and scripts/local-ci.sh never ran the update step, so the change also closes a divergence documented in three places.

• Split the ci job across three self-hosted runners (#1484, PR #1487). Thirteen steps ran serially on one machine in about 40 minutes, of which 83% was rustc recompiling the workspace under a feature set the previous step did not share, because Cargo keys build artifacts on the exact resolved feature set; one step spent 789 seconds to execute 8.1 seconds of tests. Three machines answer the runner label and two sat idle for the whole run. The job becomes ci-default, ci-isolated-graphs, and ci-release-graph, grouped so that steps building the same feature graph stay together, measured at 723s, 942s, and 1037s, so the wall clock is now the longest of the three rather than their sum. Every step's command line is unchanged. A duplicated cargo test --lib run is removed, since --tests already selects the library's unittest target.

• Keep the Cargo build directory outside the checkout (#1494, PR #1496). actions/checkout runs git clean -ffdx before every job and target/ is gitignored, so the build directory was deleted at the start of every job and every CI build was cold; the checkout log names target/ in its removal list. A local composite action now points the build directory at a persistent per-runner path outside the workspace and clears it once it passes 80 GB, so the worst case of a stale cache is one slow run rather than a red one.

Dependencies¶

• Refresh Cargo.lock to the latest semver-compatible versions (PR #1497). Thirteen crates move by a patch or minor release (aws-sdk-sts, chacha20, combine, cpufeatures, deadpool, deadpool-redis, h2, hyper, indexmap, libredox, lru, rand, uuid); nothing is added or removed and the resolved graph stays at 477 packages. No security advisory motivates the change, and cargo deny check passes on both the previous and the updated lock file against an advisory database refreshed to 2026-08-29.

v1.25.0 - 2026-08-26¶

Twenty-four commits since v1.24.1. Four defects reported from a Backend.AI GO deployment running one vLLM backend under load are fixed together: a warmup timeout that never latched, a cancelled streaming request that leaked a half-open circuit probe, a single-backend retry exhaustion that hid the upstream error behind a router-authored 503, and a model that vanished from the catalog while its fallback chain still served it. The root crate and the performance harness move to Rust 2024 and declare rust-version = "1.95". Three additions extend Hub integration and runtime backend management: per-model budget fallback, a durable sidecar for backends created through the Admin API or WebUI, and enrollment from the router itself.

Added¶

• Add Hub-directed per-model budget fallback on /v1/chat/completions and /v1/completions behind the model_budget_fallback_v1 capability (#1410, PR #1430). When a Hub-authored monthly budget for the effective provider and model is exhausted and the tier permits fallback, the router dispatches to the first ordered eligible target instead of returning 429 insufficient_quota. Candidates are intersected with tier allowlists, per-key backend permissions, visibility, provider identity, health, and circuit state, and retries and streaming failover are held to the admitted offering. Usage records keep both the served identity and the pre-budget requested identity, and the fallback is reported on bounded metrics and on streaming and non-streaming response headers. The router side of continuum-hub#909; the capability is advertised only once the durable usage-sequence state and the enforcement path are attached.
• Persist backends created through the Admin API or the WebUI in an opt-in owner-only YAML sidecar (#1423, PR #1436). Runtime backend registrations existed only in memory, so they were lost on restart and on any config-file reload, while runtime API keys had api_keys.persistence_file for exactly this. Local create, update, and delete mutations are written atomically and restored at startup, file-defined and Hub-managed backends stay out of the sidecar, and the Admin API and WebUI report whether a backend is configured, memory-only, or durable. Continuum Hub overlay and authoritative ownership behavior is defined and tested against it.
• Enroll a router into a Continuum Hub from the router itself (#1424, PR #1437). Enrollment previously required editing control_plane.enrollment_token in the YAML and restarting, and the Admin path that looked like it should work could not take effect. A feature-gated Admin endpoint exchanges a write-only token and persists only the Hub-issued owner-only credential, refuses accidental replacement, exposes rejected and pending-restart states, serializes startup and Admin enrollment, and lets a rejected credential recover without editing the state file. The Integrations page gains the matching form. The rest of control_plane stays startup-owned, with an explicit error on a runtime write.

Changed¶

• Activate Rust 2024 for the root crate and the perf/ harness, and declare rust-version = "1.95" (#1417, #1418, PRs #1420, #1421). The source preparation landed first as a separate pass: every machine-applicable rust-2024-compatibility rewrite, explicit matches wherever the compiler flagged a potentially non-trivial destructor in an if let temporary, and an audit of 215 tail-drop warnings across 209 sites, of which four needed explicit bindings to preserve observable ordering. The repository toolchain stays pinned at 1.97.1, rustfmt.toml keeps style_edition = "2021" so the switch is not a repository-wide reformat, and the vendored crates/continuum-protocol stays on edition 2021 and Rust 1.88 to match the upstream Continuum Hub workspace.
• Resolve the health checker's settings from the authoritative health_checks section on both startup and hot reload (#1395, PR #1429). Startup read timeouts.health_check.* while hot reload read health_checks.*, so the same file produced 30s probes at boot and 5s probes after any reload, and --disable-health-checks, --health-check-interval, and --health-check-timeout wrote fields that startup never read. One fallible resolver now serves both paths, health_checks.enabled gains real lifecycle semantics instead of being simulated with a 24-hour interval, and enabled, block_startup, and prewarm_timeout are restart-required while cadence, thresholds, endpoint, and warmup settings stay gradual reloads.
• Replace both control-plane outbox acknowledgment Result<(), ()> returns with a public typed AcknowledgeError that separates a stale result, unavailable state, and a durable persistence failure (#1404, PR #1435). Repeated acknowledgments stay idempotent, and the two agent drain sites now log an accurate reason and mark only durable write failures for retry.

Deprecated¶

• timeouts.health_check is parsed and then ignored (#1395, PR #1429). A value that differs from the health_checks section produces a conflict warning at load naming the field inert. The Admin API, MCP tool surface, documentation, templates, example config, and generated IDE rules are synchronized to the same contract.

Fixed¶

• Latch the backend warmup timeout so a wedged backend settles in Unhealthy (#1396, PR #1415). The transition out of WarmingUp called record_failure(), which cleared warmup_started_at, so the next HTTP 503 probe started a fresh warmup timer: a backend serving 503 indefinitely alternated between WarmingUp and Unhealthy on every probe cycle and max_warmup_duration was effectively unbounded. An explicit not-started, active, and timed-out state machine replaces the optional timestamp, the latch survives ordinary failures until a successful health signal clears it, and it is preserved across same-URL backend renames.
• Probe only the backends that need acceleration (#1397, PR #1419). One WarmingUp, Unknown, or recently-added Unhealthy backend put the whole fleet on the 1s warmup interval instead of the 30s normal interval, multiplying probe traffic to every healthy backend by about 30x. The scheduler now tracks a monotonic last-dispatch reservation per backend and selects accelerated and normal targets independently each cycle, reconciles results by stable URL identity, holds no health-state lock across a network await, and wakes when hot-reloaded cadence settings change.
• Stop validating self-hosted reasoning_effort against Gemini's vocabulary on the generic streaming path (#1393, PR #1416). The payload helper shared by initial and fallback streaming attempts applied Gemini normalization to every backend, so a vLLM request carrying reasoning_effort: "max" or "none" was rewritten. Gemini normalization moves back inside the Gemini backend path, and OpenAI-family normalization and stream_options.include_usage are unchanged.
• Release the half-open circuit probe when a streaming request is cancelled before a backend outcome (#1409, PR #1427). The chat streaming paths called proxy::circuit::admit directly and recorded the outcome by hand after the handshake, so a client disconnect in between ran none of record_success, record_failure, or record_ignored. In HalfOpen that leaked one of half_open_max_requests slots, and because the state machine refuses to leave HalfOpen while any probe is outstanding, three leaks meant every request to that backend was rejected with 503 until a restart or an Admin circuit reset. Every handshake and the non-streaming retry dispatch now go through the existing Admission guard, with first-wins settlement across the standard, fallback, native-provider, and Unix-socket paths.
• Keep configuration hot reload attached across atomic-replace saves (#1398, PR #1428). The watcher handled only EventKind::Modify(_) and watched the config file path itself, so a rename-into-place save (vim, VS Code atomic save, sed -i, provisioning tools, Kubernetes ConfigMap symlink swaps) did not trigger a reload, and the inotify watch stayed attached to the deleted inode so every later change was missed until restart. The watch moves to the logical target's parent directory, and parsing and validation move out of the notify callback into a bounded debounced worker that preserves the last known good revision.
• Return the real upstream failure when retry exhaustion leaves no untried candidate (#1408, PRs #1441, #1442, #1443). With one backend serving a model, a retryable first attempt (429, 502, 503, 504, or a first_byte timeout) left the second attempt with no candidate and the request failed with the router-authored 503 AllBackendsUnhealthy, so a transient vLLM 429 from queue pressure was indistinguishable from a dead backend. The last dispatched backend failure is now preserved with its upstream HTTP status, provider message, Retry-After, and provable backend attribution across the non-streaming, pre-stream, and mid-stream paths; a first-attempt unhealthy selection stays router-authored, and first-byte exhaustion is reported as a backend-specific 504. Two integration contracts that asserted the old generic error were corrected in the same series.
• Keep a model in the catalog while its only serving backend is unhealthy (#1422, PRs #1432, #1438). Discovery dropped the backend's entries, so a model that inference still routed correctly through its configured fallback chain disappeared from GET /admin/models, GET /v1/models, and the WebUI Models page, which made the fallback path unreachable from any client that builds its model picker from /v1/models. Force refresh now fans out synchronously without deleting the prior aggregate first, and only failed-backend memberships are carried forward, bounded by the catalog size limit, intersected with the backend's current configured allowlist, and never resurrected for a successful, removed, disabled, or newly narrowed backend.
• Limit the strict smart-routing classifier schema to its required fields so OpenAI accepts it (#1406, PR #1434). The shared schema declared reasoning without requiring it, which strict structured output rejects, and the field was unused on the routing hot path. Custom complexity and domain enum extensions are unchanged, and tolerant reasoning parsing is retained for non-strict responses.
• Apply the GPT-5 token-field compatibility transform on every OpenAI dispatch path (#1407, PR #1440). The smart-routing classifier built an OpenAI-shaped request with max_tokens and dispatched it through Backend::execute_chat_completion, which forwarded the body without the max_tokens to max_completion_tokens rewrite the proxy path applies, so classification against a GPT-5.x model failed. One idempotent backend-boundary helper now normalizes GPT-5, o1, and o3 token-limit fields across the selected OpenAI and Azure proxy, streaming, typed chat, typed stream, probe, transform, and Anthropic compatibility paths, preserving canonical field precedence, sampling controls, extra_body, and unrelated fields. Gemini and self-hosted providers are untouched.
• Add MLxcel to the WebUI backend type selector (#1414, PR #1431). The router already supported the backend type, but the Add and Edit Backend form did not offer it, so an operator adding an MLxcel backend through the WebUI had to misclassify it.
• Restore the AppProxy registered-but-down circuit fallback tests (#1412, PR #1426). Fallback eligibility moved to the request-time Config snapshot and the fixture attached its FallbackConfig only to the FallbackService, which is not what production reads.

CI¶

• Preserve crash evidence when the perf/ unit tests hit the intermittent SIGILL on native x86_64 (#1403, PR #1433). Kernel, Rust, Cargo, CPU model, and CPU flags are recorded before the suite, and on failure the core, the exact matching test executable, their hashes, the test log, and full debugger output are retained, with bounded compression time and CPU and the raw core kept if compression cannot finish. Twenty formal runs on a GitHub native x86_64 runner forced to Rust 1.98.0, covering 200 perf suites, 10,000 tests, and 20 smoke pipelines, did not reproduce the crash, so the failure-only diagnostic path is what a future occurrence will leave behind.
• Promote the mirrored all-features all-targets build check to cargo clippy --all-features --all-targets --keep-going -- -D warnings in both .github/workflows/ci.yml and scripts/local-ci.sh (PR #1435). The widened gate found one test-only health-config initializer warning, and later one test environment-lock scope finding (PR #1439).
• Run appproxy_ingress_test under the legacy appproxy feature in hosted CI and in the local mirror (PR #1426).

Documentation¶

• Update the Korean Extension Points cache example to the current two-step CacheStore and ResponseCacheStore form, document the shipped RedisCacheStore and its redis-cache feature gate, and drop the deleted service types (#1382, PR #1425).

Dependencies¶

None. Cargo.lock is unchanged since v1.24.1 apart from the version bump.

v1.24.1 - 2026-08-24¶

Four commits since v1.24.0, two of which exist because the last release ran. v1.24.0 published without its two musl archives and without a container image, because the release workflow installed the cross-compilation targets on a different toolchain than the one that built them. macOS notarization moves off one person's Apple ID onto an organization-issued App Store Connect API key. The other two changes are a timeout fix on the Anthropic ingress paths and a direct-dependency major refresh.

Fixed¶

• Read the Anthropic-ingress backend deadlines from configuration on every dispatch path (#18, #1029, #1399, PR #1400). The handlers behind the Anthropic Messages API carried fixed backend deadlines, so timeouts.request and any model_overrides entry were ignored once a request entered that ingress: a vLLM request was cut at 300.027 seconds, and under the configured streaming deadline the same request completed after 952 seconds. Buffered calls now use timeouts.request.standard.total, and true SSE calls use streaming.first_byte, chunk_interval, and total as three independently armed deadlines spanning both response opening and body consumption. Coverage is native HTTP, Unix sockets, the OpenAI and Responses transforms, and Bedrock Runtime, with per-model overrides applied on all five. The response mode is passed through the request builders as a typed value rather than inferred from payload.stream, which is what keeps the Responses bridge correct when it streams upstream but buffers for a non-streaming client, and every phase resolves from the immutable CoreConfig snapshot taken at request entry rather than from the startup cache, which is retained only as a parse-error fallback. A timeout is classified separately from a transport error in the SSE stream, the Unix and typed streaming paths fill in request-failure and circuit-breaker accounting they had been skipping, and standard.total applies as one absolute outer deadline to buffered Unix dispatches, including the request write, which the transport's own response-read timeout could not bound.

CI¶

• Activate Rust edition 2024 for the root router crate and the standalone perf/ harness while declaring rust-version = "1.95" in both manifests (#1418). The explicit MSRV matches the Debian Build-Depends floor and the English/Korean source-install guidance, while the repository toolchain remains pinned to Rust 1.97.1 for reproducible linting. A root rustfmt.toml pins style_edition = "2021" so this language-edition change does not carry the unrelated Rustfmt 2024 mechanical rewrite, and the vendored crates/continuum-protocol crate remains intentionally mixed-edition at edition 2021 / Rust 1.88 until the upstream Continuum Hub workspace moves.
• Install the release build's cross-compilation targets on the pinned toolchain (PR #1405). A rustup target is installed per toolchain, and rustup resolves a directory's rust-toolchain.toml ahead of the rustup default, but .github/workflows/release.yml installed ${{ matrix.target }} through dtolnay/rust-toolchain@stable, whose composite action runs rustup toolchain install stable --target <target>. The musl std therefore landed on 1.98.0 while cargo build --target ran under the pinned 1.97.1 and failed with "can't find crate for core", so v1.24.0 published without its linux-x86_64-musl and linux-aarch64-musl archives. The four host-target jobs passed throughout, because a toolchain always carries std for its own host, and nothing in ci.yml or perf.yml cross-compiles, so no gate exercised the broken path until a release ran. The workflow now reads channel out of rust-toolchain.toml and installs the target on that toolchain, and a check between install and build fails immediately when the target is missing from the active toolchain rather than several minutes into a compile.
• Run the Rust CI jobs when the toolchain pin or the clippy configuration changes (PR #1405). The rust paths filter in .github/workflows/ci.yml listed deny.toml and rustfmt.toml but neither rust-toolchain.toml nor clippy.toml, both of which arrived after that list was written. Raising the pinned channel is the change most likely to break a build or surface new lints, and it would have skipped the entire Rust matrix.
• Notarize the macOS release binaries with an App Store Connect API key (PR #1411). notarytool authenticated with --apple-id and an app-specific password, a credential that belongs to one person's Apple ID: it stops working when they leave the team, rotate their password, or reset 2FA, and the release stops with it. An App Store Connect API key is issued and revoked by the organization instead. The name misleads, so worth stating plainly: the key is not App Store only, because notarization exists for distribution outside the App Store, App Store submissions are not notarized by the developer at all, and notarytool documents --key, --key-id, and --issuer as a first-class credential alongside --apple-id. --issuer is passed only when set, since a Team key requires it and an Individual key rejects it; the .p8 is accepted base64-encoded or raw; and a key that does not parse as PKCS#8 is reported at that point rather than surfacing later as notarytool's opaque invalidAsn1. The packaging environment needs AC_API_KEY_ID, AC_API_ISSUER_ID (omitted for an Individual key), and AC_API_PRIVATE_KEY_P8 holding the base64 of the .p8, or notarization fails with a named error. APPLE_ID, APPLE_TEAM_ID, and APPLE_PASSWORD become unreferenced and can be removed once a release has gone out on this path.

Dependencies¶

• Update five direct dependencies to their current released majors (PR #1413): base64 0.22.1 to 0.23.1, jsonwebtoken 10 to 11, rmcp 2.2 to 3.1, serial_test 3.5 to 4.0, and validator 0.20 to 0.21, with the resolver-compatible transitive set refreshed alongside them. The one API change that reached router code is rmcp 3.1's multiple-response-type handler surface, so the configuration-assistant MCP server converts its existing complete tool and resource results into the new CallToolResponse and ReadResourceResponse enums. generic-array stays at 0.14.7 and matchit at 0.8.4, because crypto-common and axum exact-pin those versions.

v1.24.0 - 2026-08-22¶

Make the timeout subsystem configurable and coherent. The DoS ceilings that cap the whole timeouts: section become operator-tunable and boot-pinned, first_byte starts doing something on the one path where it means something and stops pretending on the paths where it does not, and the remaining hardcoded timeouts get a configuration surface. The toolchain is pinned in the same release, so the lint gate stops changing with the calendar.

Added¶

• Expose the timeout validation ceilings through a new timeouts.limits block, and raise the default max_standard_timeout from 180s to 540s (#1401, PR #1402). TimeoutLimits is the guard that caps every value in the timeouts: section, and it had no configuration surface at all: an operator who needed a longer budget than the compiled-in cap had to patch the source and rebuild, and two of the caps sat at exactly the shipped default value, leaving zero adjustment room. The shipped timeouts.request.standard.total stays 180s, so raising the cap changes what a configuration may ask for rather than what any deployment does; nothing changes until an operator raises the value, and at that point a hung upstream holds a request slot for the larger budget, which connection, health checks plus the circuit breaker, and server.max_concurrent_requests are what bound. The block is resolved once from the boot configuration and pinned for the process lifetime, so hot reload, the Admin API, and control-plane config sync all reject a candidate that changes it rather than letting a runtime or hub-authored path widen a router's own ceiling; continuum-router config validate <file> runs with no pin and therefore resolves from the file under inspection, which is the correct behavior for validating an arbitrary file. Each field also carries a compiled-in absolute ceiling (max_standard_timeout 1080s, max_retry_timeout 120s, max_streaming_timeout 3600s, max_connection_timeout 60s, min_chunk_interval 10s, max_chunk_interval 300s, max_image_generation_timeout 1800s, max_first_byte_timeout 1200s, large_model_bonus 1200s), and a value above its ceiling is a load error rather than a silent clamp. extended_models replaces the hardcoded large-model list, which granted a streaming bonus to gpt-4 and claude-3-opus while matching none of the models config.yaml.example actually configures. The shipped default is now the current frontier tiers (gpt-5.6-sol and its Terra and Luna siblings, gpt-5-pro, gpt-5.5-pro, claude-fable-5, claude-opus-5, claude-opus-4-8, gemini-3.1-pro and gemini-3-pro) rather than the retired one, so this is a deliberate behavior change: a per-model streaming override above max_streaming_timeout now succeeds for a current frontier model and is rejected for a retired one. An operator list replaces the default wholesale, which is the escape hatch in either direction. In the same pass retry.timeout is documented as what the retry loop implements, one budget spanning every attempt that is consulted only between attempts and never interrupts one in flight, and the "total possible retry time" warning was dropped rather than carried forward, because both of its terms assumed per-attempt semantics and a correctly computed value could never reach its own 180s threshold under a 120s absolute cap.
• Give five hardcoded timeouts a configuration surface, each keeping its previous value as the default (#1401, PR #1402). server.http_client exposes the shared outbound client's pool_idle_timeout (300s), pool_max_idle_per_host (64), tcp_keepalive (30s), http2_keep_alive_interval (15s), http2_keep_alive_timeout (5s), and http2_keep_alive_while_idle (true); the client is built once at startup, so like server.connection_pool_size these are restart-required. health_checks.prewarm_timeout (10s) replaces the fixed cap on the startup pre-warm probes, read by both the Anthropic branch and the generic branch. The two Unix-socket streaming paths now take their connect deadline from timeouts.connection instead of a separate hardcoded 10s, so the socket and TCP transports agree. timeouts.streaming_fallback_budget_multiplier (2.0, accepted range 1.0 to 10.0) exposes the factor that turns request.streaming.total into the single wall-clock budget shared by every attempt in a streaming chain; it is deliberately a new knob rather than a reuse of fallback.fallback_policy.fallback_timeout_multiplier, which scales one attempt's own timeout rather than the shared chain budget. An absent block, and every absent field inside one, reproduces the previous client and the previous budgets exactly.

Changed¶

• Enforce timeouts.request.streaming.first_byte and raise its default from 60s to 120s (#1401, PR #1402). The field was parsed, validated, cached, and then read by no request path, so a documented knob silently did nothing; on the mid-stream fallback path it was worse than inert, because that read loop applied chunk_interval from the very first poll, making the effective first-chunk deadline 30s rather than the configured 60s. It is now the deadline for the first chunk on all three streaming paths, with chunk_interval taking over from the first chunk onward, and an expiry before the first chunk is reported as a distinct first_byte timeout so logs and metrics can tell a slow first token from a mid-stream stall. On the mid-stream fallback path this is a relaxation, since 120s replaces an effective 30s; on the two SSE pipeline paths it is a tightening, since nothing bounded the first chunk there at all. The default moves to 120s because reasoning models routinely exceed 60s to their first token, which is exactly why config.yaml.example already prescribed first_byte: 120s for the Gemini thinking models. Two new load-time checks come with it: streaming.first_byte must not exceed timeouts.limits.max_first_byte_timeout (480s by default), and it must not exceed streaming.total, without which a configured deadline could never fire and nothing would say so. A per-model streaming.first_byte override is bounded by the same cap, so a model override is not a bypass.
• Apply timeouts.request.streaming.chunk_interval on the Bedrock and Gemini streaming paths (#1401, PR #1402). The per-chunk deadline was wired only on the mid-stream fallback path: both SSE pipeline constructions left it unset, so a stalled Bedrock or Gemini stream was bounded by nothing except the total budget, and with the per-chunk deadline unset even that was only re-evaluated when an outer layer happened to wake the task, which means at keep-alive granularity rather than promptly. Both paths now pass the per-model resolved chunk_interval, so stall detection is uniform across all three streaming paths instead of correct on one of them.
• Breaking: cap timeouts.request.image_generation.total at 600s (#1401, PR #1402). The image-generation block was validated against nothing at all, so an operator-supplied budget of an hour passed while a 200s standard budget was rejected, and image_generation.first_byte was never compared against its own total either. A deployment that sets image_generation.total above 600s will now be rejected at load. Raise timeouts.limits.max_image_generation_timeout (absolute ceiling 1800s) and restart to keep the larger value; the 600s default was chosen well above the 180s effective default so that realistic existing configurations keep loading unchanged.
• Route the Gemini native multimodal embedding request through the configured standard timeout (#1401, PR #1402). It was pinned to a hardcoded 60s that bypassed the timeouts: section entirely, ignoring request.standard.total, ignoring any model_overrides entry for the embedding model, and unchangeable without a rebuild. It now reads the per-model resolved standard total like every other non-streaming call, which also makes it hot-reloadable. This raises the effective default on that one path from 60s to 180s; a deployment that wants the old ceiling sets timeouts.request.model_overrides.<model>.standard.total: 60s.

Deprecated¶

• timeouts.request.standard.first_byte and timeouts.request.image_generation.first_byte are inert and scheduled for removal in the next major release (#1401, PR #1402). An OpenAI-compatible non-streaming completion is a buffered, Content-Length-delimited body: the upstream generates the entire completion and only then writes headers and body together, so time to first byte on that path IS the full generation time rather than a separable early signal. A deadline there would not be a distinct guard at all, it would be a second and shorter total, and enforcing the shipped 30s default would have turned it into a 30s ceiling on every non-streaming completion. Both fields stay parsed and stay bounded by first_byte <= total for backward compatibility, a non-default value now produces a load-time warning naming the field inert, and continuum-router config validate reports the same thing. The operative budgets are standard.total and image_generation.total, and the advisory for reasoning models moved with them: it now points at standard.total on the non-streaming path and at streaming.first_byte on the streaming path.

CI¶

• Pin the toolchain to Rust 1.97.1 through a new rust-toolchain.toml. The repository pinned nothing, so CI linted against whatever stable resolved to on the day it ran. That is not a theoretical exposure: when Rust 1.98 tightened clippy::result_large_err, cargo clippy -- -D warnings began failing on main itself with no change to the code, which a re-run of main's last green CI run at the unmodified commit 6b0f6e0f confirmed. CI installs stable through dtolnay/rust-toolchain@stable, which sets it as the rustup default, and rustup resolves a directory's rust-toolchain.toml ahead of that default, so this file is what every cargo invocation in the repository actually uses, locally and on a runner alike. components lists rustfmt and clippy, so the pin is self-sufficient rather than dependent on the workflow installing them separately. Note what it masks: on 1.97.1 the wider cargo clippy --all-targets --all-features reports zero warnings where 1.98 reports the clippy::result_unit_err pair tracked in #1404, so that issue is still real but no gate surfaces it at the pinned toolchain. Raising the pin is deliberate work: bump the channel, run make ci-local, and fix whatever the newer lints surface in the same change.

Dependencies¶

• Refresh Cargo.lock against the current semver-compatible set: 36 crates updated, itertools 0.14.0 added to the graph, none removed. The direct dependencies that moved are redis 1.5.0 to 1.6.0, uuid 1.24.0 to 1.24.1, and aws-config 1.10.1 to 1.11.0. The rest are transitive, the largest groups being the icu_* family 2.2.0 to 2.3.x and the AWS Smithy runtime crates.

v1.23.0 - 2026-08-20¶

Ship OpenTelemetry trace export and shared Redis state in official release binaries, and make a forgotten backend visibility filter fail to compile instead of misrouting. The hidden-backend bug class this repository had fixed nine times is closed by a type rather than a tenth patch.

Added¶

• Export spans over OTLP behind a new off-by-default otel Cargo feature (#1248, PR #1391). Export is opt-in twice: the build must carry otel, which official release binaries do, and tracing.otlp.enabled must be true, which defaults to false. The tracing.otlp configuration type is always compiled, so one config.yaml validates identically under any feature set and a default binary warns that export is inactive rather than refusing to start. Four spans and two events make up the whole taxonomy: router.request with the matched route template rather than the raw path, router.select_backend, router.backend_call, and the router.retry and router.circuit_breaker events, each emitted from the seam that already counts the same thing so a trace and /metrics cannot disagree. The taxonomy is enforced twice, at the constructors and again at a span processor on the export boundary that drops any span, attribute, or span event outside it, which is what keeps ordinary log lines and provider error bodies out of the collector. Auth header values resolve ${ENV_VAR} only at exporter construction, so a config dump never holds the secret, and header names must be RFC 7230 tokens with no control characters in values. With export off, every span constructor returns Span::none() after one relaxed atomic load.
• Ship redis-cache in official release binaries and define the shared-state deployment contract (#1249, PR #1390). The shared-state code existed but no official artifact contained it, so the deployment profiles had to present per-replica rate limits as an unavoidable boundary. Compiling the feature activates nothing: rate_limiting.storage: redis and response_cache.backend: redis remain the runtime opt-in. The router drives Redis through one deadpool-redis pool over ConnectionManager, which performs no Sentinel master discovery and no cluster slot routing, so the supported topology is exactly one managed endpoint (standalone, a provider-managed HA endpoint, a proxy fronting a cluster, or a unix socket). redis+sentinel://, redis+cluster://, comma-separated multi-host URLs, and a #master-name fragment are rejected at config load rather than implied and then failing at connect time. An unresolved ${VAR} is accepted and re-checked once resolved, so a rendered manifest validates on a CI runner holding no production Secret. Outage semantics are documented and tested per consumer. The Helm chart and the Kustomize bundle reference an operator-created Secret and never ship a Redis server or invent a credential.
• Add a buffered MaybeReasoning state to the thinking stream transformer (#216, PR #1389). assume_reasoning_first decided what the leading tokens of an unterminated_start model were before any evidence arrived, so a model that skipped its thinking phase and answered directly sent the whole answer on the reasoning channel and clients rendering only content showed an empty response. The new state withholds those tokens and resolves the classification from what arrives: the end marker flushes the prefix as reasoning_content, while a decision timeout, the buffer ceiling, or end of stream flushes it as content. Opt-in per model through buffered, max_buffer_size (50KB default, 8MB cap), and reasoning_timeout (10s default), all defaulting to today's behavior. Measured against a 400ms thinking phase, unbuffered relays the first delta in about 0.14ms and buffered in about 401ms; streams that never enter the decision window arm no timer and hold no extra buffer.
• Automate progressive delivery with an optional Argo Rollouts integration in the Helm chart (#1251, PR #1392). Weighted canary steps, automated analysis against the router's own Prometheus series, automatic rollback, and an operator approval pause before production promotion, all values-gated and off by default. Every existing profile renders byte-identical output to what it rendered before, verified by diffing renders from main against the branch. Both analysis metrics select on rollouts_pod_template_hash so a bad canary cannot hide inside good stable traffic, and both queries carry a minRequestRate guard that makes a thin overnight sample Inconclusive rather than rolling back a good release on one failure out of three requests. The Argo Rollouts CRD schemas are vendored under deploy/schemas/ so kubeconform validates the custom resources offline instead of reporting them as Skipped.
• Add perf/, a standalone load harness and a weekly capacity regression gate (#1250, PR #1388). Reproducible non-streaming and SSE-streaming profiles run against hermetic mock backends, record results as versioned JSON, and are gated against checked-in reviewed thresholds. The crate declares an empty [workspace], so cargo build, cargo check --all-targets, and make ci-local at the repository root never compile it, and the gate is guarded by if: github.event_name != 'pull_request' so it never touches a per-commit build. The driver is a minimal HTTP/1.1 client over TcpStream rather than reqwest, because a driver sharing its pool implementation with the system under test cannot separate client-side from router-side queueing. benches/ stays Criterion-only.
• Report x-served-backend on a failure a named backend actually returned (#1362, PR #1378). The header was suppressed on every non-2xx response, which is a blanket safety measure rather than a fact about the problem: an upstream 429 that a real backend genuinely returned came back unattributed, and that is the case where the answer matters most. Attribution cannot ride in the error value, because RouterError::RateLimited is byte-identical whether it came from an upstream 429 or from the router's own limiter, so provenance is recorded at the four dispatch sites that hold a real HTTP response from a backend and carried out of the retry seam through a per-request slot every attempt overwrites. After a 429 from one backend and a connect failure against the next, the slot holds nothing rather than a stale name. Router-authored failures stay unattributed.

Changed¶

• Make a forgotten backend visibility filter fail to compile (#1359, PRs #1369 and #1373). Nine separate fixes (#868, #1336, #1339, #1348, #1349, #1354, #1355, #1356, #1357) were all the same defect: a path selected a backend for a user request without applying filter_user_routing_candidates, so a backend marked internal: true or enabled: false stayed an ordinary candidate. It recurred for a structural reason rather than a careless one. The filter cannot move into the shared selection seam, because it has to run before the per-key allow-list so a model served only by hidden backends reports model-not-found rather than a misleading forbidden, and the seam never receives that allow-list. So the filter stayed with the caller, every candidate list was a plain Vec<String>, a filtered list and an unfiltered one had the same type, and omitting the filter compiled, passed every test, and misbehaved only on a deployment that actually hid a backend. The new proxy::selection::UserRoutableCandidates newtype wraps a private Vec<String> with a constructor set that carries the proof, so a seam-based path that skips the filter no longer compiles. Alongside it, the two visibility test suites are now driven by the mounted route table read out of the route-composition functions, and a source-scanning audit demands a decision for each candidate-building site, so both stop being hand-written lists that recorded history.
• Rewrite the Dependency Injection section of both architecture documents (#1367, PR #1381). It described a container-resolution design that was scaffolded and abandoned before it was wired. Four names in the samples did not resolve and two never did: setup_services has no commit in the history of src/, and HttpClient::new(&config.http_client)? could not have compiled against any version of the code. The section is rewritten around the pattern the router actually uses, the BackendService trait seam with BackendManager as its only production implementation and the two construction sites in src/server/state.rs.

Fixed¶

• Read vLLM reasoning output on the Anthropic Messages ingress (#1386, PR #1387). The Chat Completions arm of /anthropic/v1/messages read reasoning only from reasoning_content, so current vLLM responses using reasoning silently lost billed thinking output. The Responses arm of the same endpoint already converted that field. Both non-streaming blocks and streaming thinking_delta events now read reasoning_content first and fall back to reasoning, and output guardrails still inspect the converted result, so the alias does not bypass configured reasoning inspection.
• Filter hidden backends out of four control-plane candidate sets (#1355, PR #1361). handle_disaggregated_request, compute_arbitrage_decision, target_provider_backends, and provider_for_model each built candidates from config.backends or find_backends_for_model with no visibility filter. None could dispatch to a hidden backend, because the shared seam applies the filter before the per-key allow-list and a hidden name arriving as an allow-list entry empties the intersection. What the caller got instead was a wrong answer: a 403 blaming a key that was never the problem, a 404 when the hidden backend was the only one serving the model, a 503 on the disaggregated path, or a silently mis-decided substitution rule with no error anywhere. All four were safe only because every current consumer treats allowed_backends as a filter applied after the visibility filter, and nothing in the type system said so.
• Stop the smart-routing LLM classifier sending prompts to a hidden backend (#1356, PR #1363). With classifier.method set to llm or hybrid and no explicit classifier.llm.backend, resolution fell back to config.backends.first() with no visibility check, so every classifiable request's prompt text went to whatever sorted first, which may be an internal: true guard backend or a powered-off enabled: false standby. Unlike the other instances of this class, what leaks is prompt content rather than routing. Resolution happens once per config generation, so the exposure was steady rather than intermittent. The implicit fallback now takes the first user-routable backend and logs at info which one it picked, an internal: true pin stays honored because that is what the flag is for, and an enabled: false pin is refused with a warn while the router continues on rule-based classification.
• Filter hidden backends out of the web_search policy scan (#1357, PR #1364). find_backend_for_model decided the tool-injection policy by scanning config.backends unfiltered. No hidden backend is dispatched to on this path; a hidden backend claiming the model could decide whether web_search was injected into a request a different, visible backend served, which web_search.per_backend override applied, and what backend-type value the metric label recorded.
• Refuse a disabled backend for the guardrail guard model reference (#1371, PR #1377). The backend: reference resolved purely by name, so a backend an operator had switched off with enabled: false kept receiving guard-model prompts on the request hot path. internal: true stays exempt, since reserving a backend ordinary routing will not reach is the usual way to serve a guard model. A refused reference is neither a hard allow nor a hard block: it produces the same ordinary check error an unresolvable guard backend already produced, so the provider's existing on_error policy decides, and the operator's fail_open or fail_closed choice is not overridden in either direction.
• Take one configuration snapshot per image-edit backend resolution (#1372, PR #1376). image_edit::find_openai_backend read state.current_config() three times while resolving one /v1/images/edits request, so a hot reload landing between two reads could decide one resolution against two config generations and select a backend no single generation would have produced. It now takes one snapshot, matching the shape image_gen::find_openai_backend already had.
• Make response_cache.redis.fallback_to_memory: false fail closed (#1249, PR #1390). The store still read and wrote the in-memory tier when the flag was off, so the fail-closed option did not exist. It now reports the outage, which the response cache logs and treats as a miss. In the same pass, rate_limiting.redis turned out never to have been validated on the real load path at all; its endpoint is now checked there, scoped to the URL rather than the whole RateLimitConfig::validate impl, whose numeric ranges have also never run at load and would refuse to start routers that work today.
• Harden three paths after the merge of #1388, #1389, and #1390 (PR #1394). The MaybeReasoning buffer enforced its byte ceiling only after buffering the incoming chunk, so a single large chunk could be buffered whole before the check; the ceiling is now enforced before buffering, marker precedence is preserved exactly at the ceiling, and the full-buffer clone and rescan that made the scan quadratic is gone. Redis fallback state and its exported gauge could drift apart, fallback_to_memory: false could still activate the memory tier through the recovery path, and the health, clear, and statistics operations carried no command timeout. The performance harness read time to first token from the first content delta even when that delta was empty, so a backend whose first SSE frame carried an empty content field reported a wrong TTFT.
• Restore the safe continuum_router::streaming compatibility exports (PR #1385). PR #1380 deleted the module along with the unfiltered selectors it held, which also removed the documented public compatibility namespace on a crate still on the 1.x line. The safe parser, transformer, and streaming helper re-exports are restored; every unfiltered backend selector stays deleted, and both source-audit and compiled-path coverage now prevent one from returning.
• Cover SelectionStrategy::WeightedRoundRobin and retire tests/integration/ (#1366, PR #1374). A shipping selection strategy had no behavioral coverage at all. The gap was hidden by tests/integration/, seven files and 63 test functions that belonged to no Cargo test target: Cargo discovers tests/*.rs and tests/<dir>/main.rs, this directory offered mod.rs, and no top-level target declared it. It had been dead for about twelve months, hand-edited across twenty-one commits by people who reasonably assumed it ran, and produced 57 compile errors when finally built. Four tests now cover proportional distribution, a zero-weight backend, the single-unit-of-weight branch, and the all-zero fallback, and a discovery guard fails the build on any test file placed where Cargo will not compile it.
• Remove two test flakes (#1331, PR #1375 and #1383, PR #1384). The token bucket refill test asserted against a fixed window after a 1.1s sleep, but tokio::time::sleep guarantees only a lower bound, so a loaded runner could legitimately overshoot it; the expected value is now derived from measured elapsed time. A Gemini context cache test used a 3 second wall-clock expiry window that had to cover manager construction, a background create round-trip, and a polling wait, and its windows are now wide enough that scheduler delay alone cannot fail it. Two health-probe and model-timing tests that depended on HTTP connection reuse and on sibling-test execution order were also made self-sufficient (PR #1385).

Removed¶

• Delete the unreachable three-function select_backend chain (#1358, PR #1365). It selected a backend on health alone and bypassed the visibility filter, the per-key allow-list, the circuit breaker, and the configured selection strategy. Nothing had a production caller, but two of its three types were pub use-exported, which made it exactly the ready-made unfiltered selector someone finds when they grep for how to select a backend. ProxyServiceImpl, BackendServiceImpl, BackendManager::select_backend, and the select_backend requirement on the BackendService trait are gone.
• Delete ModelServiceImpl (#1370, PR #1379). get_backends_for_model enumerated the whole pool with no visibility filter and no per-key allow-list. Nothing constructed it outside its own unit tests and its ServiceRegistry registration was commented out.
• Delete BackendPool::get_backend_async (#1322, PR #1380), the unfiltered counter % len selector that applied no model, health, or circuit filter. #1316 already showed the cost of leaving one in reach: the ACP path picked it up and shipped an (N-1)/N mis-routing rate. With it gone, the contract stated on select_admissible_backend, that every dispatch path not already inside the retry loop comes through there, is enforced by the compiler on this axis rather than by convention.

v1.22.0 - 2026-08-16¶

Make x-backend work on every path where the router picks a backend, let a client read back which backend actually answered, and close six paths where a backend hidden with internal: true or enabled: false could still take user traffic. One of those six is an authorization bypass present in v1.21.2 and earlier.

Added¶

• Honor x-backend on every path where the router selects a backend (#1332, PR #1335). v1.21.2 shipped the header on chat completions, streaming, and Responses, and documented the coverage as a table of endpoints. That table was wrong in both directions: it omitted /v1/completions, /v1/embeddings, /v1/rerank, and /v1/sparse_embeddings, which honored the header through the shared retry seam, and it claimed /v1/responses honored it on the streaming path, which never read the header at all. Coverage now extends to the Anthropic Messages API and its token counter, image generation, image edits, image variations, streaming /v1/responses, /v1/responses/compact, the Gemini-native image sub-paths, and the multimodal /v1/embeddings sub-path. The documentation states the mechanism, that the header applies wherever the router selects a backend for a request, and names the three exceptions (/v1/realtime sessions, ACP prompts, and the Batch API) instead of listing endpoints that go stale.
• Report the backend that answered, through a new x-served-backend response header (#1333, PR #1342). A client that composed a backend name from its own state had no way to tell an honored preference from a silently dropped one: a single wrong character produced a system that kept returning 200 while ignoring the choice. The header is returned only on a request that carried x-backend, so a deployment whose clients never send the preference is unaffected, and it names the backend that actually produced the response, including after a retry or a fallback-chain hop. Where the answering backend cannot be proven, the header is omitted rather than guessed: streaming chat under mid-stream fallback commits its response headers before any backend has answered and can switch backends afterwards, so that arm reports nothing rather than naming the first attempt. Non-2xx responses, cache replays, guardrail blocks on non-streaming responses, and the web_search tool loop are omitted for the same reason and are listed in the documentation.
• Advertise backend_preference_header_v1 at GET /admin/capabilities (#1334, PR #1343). A client had to compare against the literal 1.21.2 to decide whether the header was worth sending, which is what that endpoint exists to avoid. One token covers the whole feature rather than a per-surface list, which is honest now that coverage is uniform. A client holding only an inference key cannot read the admin endpoint and does not need to: sending x-backend and checking for x-served-backend proves support, though its absence is inconclusive.

Fixed¶

• Enforce the per-key backend allow-list on /v1/realtime (#1337, PR #1345). The WebSocket upgrade was authenticated, but the handler never extracted the auth context, so allowed_backends was never applied to the session's backend. An API key restricted to one backend could open a realtime session served by any other backend serving the model, which is the restriction that setting exists to impose. Present in v1.21.2 and earlier. An empty intersection now answers 403 with the same permission_error shape the Anthropic and Responses paths use, kept distinct from "no healthy backend".
• Stop image variations from routing to a hidden backend (#1336, PR #1344). find_openai_backend on the variations path enumerated the pool and applied the per-key allow-list and an OpenAI capability filter, but never the visibility filter, so a backend marked internal: true or enabled: false stayed an ordinary round-robin candidate. The asymmetry was backwards: naming that backend explicitly with x-backend was correctly refused, while the implicit no-header pick could still reach it.
• Stop multimodal embeddings from spending a hidden backend's credential (#1339, PR #1347). The Gemini candidate set for a multimodal /v1/embeddings request was built without the visibility filter, and the handler attaches the selected backend's provider credential before dispatching, so a hidden backend's secret was spent on user traffic. With the filter in place, that path now also reports x-served-backend, which #1333 had deliberately withheld there.
• Stop no-model streaming chat from selecting a hidden backend (#1348, PR #1352). A streaming POST /v1/chat/completions carrying no model field resolved its backend through a helper that applied only the allow-list. The non-streaming twin of that exact case has filtered since #868; this is the half that was missed.
• Stop mid-stream fallback from hopping onto a hidden backend (#1349, PR #1353). The relay resolved a fallback target from unfiltered configuration while the pre-stream sibling filtered. A hidden backend claiming the target model could become the hop destination. A target the filter empties is now treated exactly like any unreachable hop, so a mid-stream response gains no new failure mode.
• Stop batch dispatch from selecting a hidden backend (#1354, PR #1360). resolve_batch_backend gated only on the hub credential dispatch gate, which is not a visibility check and admits every name when no gate is installed, so an internal: true OpenAI-style backend listed first took every batch job and spent its credential. The implicit first-match fallback now filters. An explicit control_plane.batch.backend pin still resolves an internal: true backend, which is what that flag is for, but no longer resolves an enabled: false one. Requires the control-plane feature, which official release binaries compile.
• Resolve image backend credentials from live configuration (#1341, PR #1350). Six credential lookups across image generation, variations, and edits read the startup configuration snapshot, so a hot reload that rotated an API key left those paths sending the old key until the process restarted, and the failure looked like an upstream authentication problem. Five backend-type classification reads in the same functions were converted with them, because leaving one read live and another stale inside a single function is its own defect.
• Cap the client-supplied x-backend value before logging it (#1340, PR #1351). Fourteen log sites wrote the header value verbatim with no length bound. There was no injection risk, since header parsing rejects control characters, but nothing capped the size, and because an unusable value falls through rather than failing, the request still returned 200 and nothing discouraged repetition.

Removed¶

• Remove the unused non-attributed deduplication entry point (#1338, PR #1346). It shared one cache with the attributed variant, so a live caller would have let a coalesced follower read an entry with no backend attribution, which flows into request stats, control-plane usage, and now the x-served-backend header. Nothing called it; removing it keeps it that way.

v1.21.2 - 2026-08-15¶

Cut the /v1/models tail latency a polling client sees from about 16 seconds to under 150ms, route ACP prompts and x-backend requests to the backend that actually serves the requested model, and sign macOS release binaries with a certificate Gatekeeper accepts.

Added¶

• Honor the x-backend request header as a routing preference on chat completions, streaming, and the Responses API (#1317, PR #1320). Several backends can serve the same model id and /v1/models already reports all of them in the aggregated entry's backends array, but a client could not address either copy: the router load-balanced across them regardless of which row the caller picked. The header can only narrow a choice the router was already free to make. It is a first-attempt bias, honored when the named backend is visible, allow-listed for the caller's key, among the candidates serving the requested model, and admissible; anything else logs a warning and falls through to normal selection. A request never fails because of this header, and a request without it behaves exactly as before. Fallback-chain hops keep normal selection, and /metrics attribution still refuses to trust the client-supplied value. The semantics are the ones proxy::image_edit has used since #674, now shared rather than duplicated.
• Add a Classifier tab to the smart-routing Web UI page (#1314, PR #1319). The admin API already read and wrote classifier.method, classifier.rule.confidence_threshold, and the classifier.llm.* fields with immediate hot reload, but the page could only display classifier state. The tab exposes the seven core controls with a backend-name datalist populated from /admin/backends, a save confirmation, and a validation panel, and the Status tab now binds classifier_method and has_llm_classifier. Advanced LLM classifier fields (timeout, temperature, token limits, structured output strategy, prompts) stay on the generic Configuration page, linked from the new tab.
• Export model aggregation timing from the production metrics registry (#1326, PR #1329). Diagnosing a slow model list previously meant reading source code. model_backend_fetch_duration_seconds is a new histogram labeled by backend, model_refresh_duration_seconds is now registered in the production registry rather than only in the legacy collector list, and 13 aggregation counters that were written on every refresh and read nowhere are exported (attempts, empty responses, unavailable backends, rate limits, transient and permanent errors, stale-while-revalidate serves, coalesced requests, background refresh outcomes, and singleflight acquisitions). A backend that exceeds its per-attempt request_timeout also produces one WARN line naming the backend, its duration, and its attempt count, so the finding is available without Prometheus. http_request_duration_seconds keeps its 10s ceiling; both new duration histograms bucket to 60s instead, so a 16 second refresh is legible on the metric that names the responsible backend rather than merged into +Inf on a shared histogram.

Changed¶

• Serve the stale model list while revalidating in the background instead of blocking (#1324, PR #1327). An expired or invalidated cache turned the next /v1/models request into a full concurrent fan-out across every pooled backend, about 16 seconds worst case at default settings, with concurrent list requests queued behind the singleflight mutex. The cache layer already tracked an expired-but-present body; the aggregation entry point read it through a getter that threw that body away. A request now blocks only when the cache genuinely has nothing to give. The staleness bound is cache_ttl * 3.0 (180 seconds at the default 60 second TTL) as an internal constant rather than a new YAML field, so the public config schema is untouched and operators tune it through cache_ttl; past the bound the request blocks as before, so a wedged refresher cannot serve arbitrarily old lists. Invalidation retains bodies and restarts the age clock from the invalidation instant, which removes the race in the common "register backends, then immediately list models" sequence. Per-request response filters (per-key backend and model allow-lists, internal-backend stripping) still run against the current configuration, so a briefly stale membership list cannot widen what a caller is allowed to see.

Fixed¶

• Make the background refresher actually refresh stale cache entries (#1323, PR #1330). Its StaleButUsable branch called an aggregation that reads through the cache and treats a stale-but-usable entry as a hit, so it returned the stale body without contacting any backend. The soft TTL window did nothing, entries were refreshed only after reaching hard expiry, and every TTL cycle left a window in which an inbound /v1/models request performed the whole fan-out inline. For a deployment whose main traffic is model-list polling, that was the dominant latency. The branch now performs a real coalesced re-aggregation, so entries refresh between soft and hard expiry and in steady state never reach expiry at all. The healthy-event listener took the same no-op path and additionally hand-incremented counters the shared path already records, which double counted; it now shares the same slot, so at most one aggregation is in flight regardless of whether the request path, the periodic loop, or the listener started it. Measured with a 300ms backend, a 2 second TTL, and a 250ms refresher cadence, request latency stays under 150ms across more than three cycles.
• Stop probing backends that cannot serve a model list (#1325, PR #1328). The refresh probed every pooled backend on every cycle and retried every non-2xx except 401 and 403. Because the fan-out is concurrent, the slowest pointless probe set the latency of the whole aggregation: a Bedrock backend spent three attempts at 5 seconds plus two 500ms waits against a /v1/models that does not exist, roughly 16 seconds, every refresh. Anthropic and Bedrock backends are now served from their configured models: with no HTTP request, matching what the admin discovery path already did for them. Disabled backends are dropped from the fetch set by set membership, so pool-only backends such as AppProxy dynamic replicas are unaffected. 404 and 405 join 401 and 403 as permanent failures that fail fast, while 5xx and connection errors keep the full retry budget. One intentional response change: Bedrock models now appear in the aggregated list, since the fetch previously failed outright and dropped them. Internal backends are deliberately still fetched, because the admin catalog, control-plane inventory, and find_backends_for_model all read past the user-facing internal filter.
• Route ACP session/prompt through model-aware backend selection (#1316, PR #1321). Selection used a bare counter modulo the pool length that ignored the requested model, backend health, and circuit state, so on a router whose backends serve disjoint models it mis-routed (N-1)/N of all prompts into an upstream 404 surfaced as -32603. In the same handler acp.default_model was declared, documented, and displayed but never read, so every model-less prompt sent the literal string "default" upstream and failed. Candidates now come from the model lookup, pass the user-routing visibility filter, and go through the shared admission seam that every other dispatch path uses, which brings health filtering, circuit admission, and the configured selection_strategy; ACP traffic also settles its outcome into the circuit breaker. Model resolution is the request override, then acp.default_model, with no literal below that: a prompt naming no model is rejected with -32602 rather than answered from an arbitrary backend. Errors carry structured data (data.field, data.value, data.source for an unknown model; data.backend and data.status for a backend HTTP error), and a blank acp.default_model is now rejected at config validation. ACP stays a thin dispatch path by design and does not inherit the HTTP retry loop, fallback chains, or the per-key backend allow-list; that decision is recorded in the module docs and the architecture page.
• Sign macOS release binaries with a Developer ID Application certificate and notarize them (PR #1313). Every macOS release up to v1.21.0 was refused by Gatekeeper on download for two independent reasons. The signing leaf was an Apple Distribution certificate, an App Store and TestFlight submission identity whose leaf lacks the 1.2.840.113635.100.6.1.13 extension Gatekeeper requires for a download from outside the App Store, so the signature verified locally and was rejected on every user's machine; codesign --sign "Distribution" matches identity names by substring, which is how it picked that certificate out of the keychain. The release also never called notarytool, and a notarization ticket is a separate requirement, so even a correct certificate would have produced the "Apple could not verify this app is free of malware" refusal on a quarantined download. Signing now runs through two composite actions that assert the Developer ID Application authority, the hardened runtime flag, and a pinned code signature identifier before packaging, then submit to notarytool --wait and gate on status: Accepted. Nothing in the old workflow inspected the resulting authority, which is why this shipped unnoticed for eight months; a wrong certificate now fails the release instead of shipping quietly.
• Enable the Configuration review dialog's "Confirm & save" button (#1315, PR #1318). Its :disabled expression ended in two bare .length operands, so with no blockers the whole chain evaluated to the number 0. Alpine removes a boolean attribute only for null, undefined, or false, so disabled stayed set and the button was permanently unclickable with no console error. Both operands are now compared with > 0. An audit of every :disabled, :checked, :readonly, :required, and :selected binding under src/webui/assets/partials/ confirms this was the only unsafe expression.

v1.21.1 - 2026-08-12¶

Advertise backend_tasks_v1 from the running task executor so Continuum Hub can dispatch backend candidate probes to a released Router, resolve 16 quantization and container model-id forms that previously matched nothing, and make Web UI charts readable in dark mode.

Added¶

• Add four NVIDIA Nemotron 3/3.5 catalog entries (#1306), taking model-metadata.yaml from 230 to 234: nemotron-3.5-lightning-30b-a3b (30B total, 3B active, hybrid Mamba-2 and attention MoE), nemotron-3-ultra-550b-a55b (550B total, 55B active LatentMoE with MTP layers), nemotron-3-nano-omni-30b-a3b-reasoning (31B omni-modal with a CRADIO v4-H vision encoder and a 0.6B Parakeet speech encoder), and nemotron-3.5-content-safety (4B Gemma-3-4B-it base with a LoRA safety adapter). The alias audit classifies every new alias as LOAD-BEARING with zero broken baselines.

Changed¶

• Name hot-reload explicitly in the release build feature list, and keep the CI and local CI release-union steps in sync with it (#1307). This is a pin, not a fix: release.yml builds without --no-default-features, so default = ["full"] already supplied hot-reload and shipped binaries have always had config hot reload compiled in. What was missing is the guarantee, which rested entirely on inheritance.

Fixed¶

• Advertise backend_tasks_v1 in heartbeat inventory whenever the outbound backend task channel is attached (#1305, PR #1310). Public v1.20.0 and v1.21.0 binaries attached the executor but never reported the capability, so Hub refused every candidate-probe dispatch with HTTP 409 and the backend-profile workflow could not be validated against a released Router. Three defects contributed. build_inventory_heartbeat replaced the whole reported policy status with the policy store's view, discarding the capability appended moments earlier, so enrollment advertised it and every heartbeat dropped it; the heartbeat inventory is the one Hub retains, which is why a released binary looked capable at enrollment and unsupported from then on. The capability was also appended only when a policy store existed, so a router running with control_plane.policy disabled never advertised it at all. Attachment was additionally re-derived in inventory.rs from a hot-reloadable configuration snapshot instead of being observed from the executor the agent actually attached, which let the two answers disagree. Attachment is now a runtime latch, set only after the identity-bound BackendTaskStore and its loop are running, cleared when that loop exits, reported independently of policy-store availability, and surfaced with a disabling reason in structured logs and on GET /admin/control-plane/status.
• Resolve 16 model-id forms that previously matched no catalog entry (#1297, PR #1309). layered_format_strip peeled trailing hyphen-delimited tokens right to left and stopped at the first unrecognized one, so every recognized token to its left was unreachable: Trinity-Large-Thinking-FP8-Block stopped on block and never peeled fp8. Now recognized are the compressed-tensors scaling qualifiers -block, -dynamic, and -static, peeled only when the token to their left is itself a quantization token so an ordinary trailing word is never stripped; the weight and activation forms -W4A16, -W8A8, and -W4A8, including the quantized. prefixed spelling RedHatAI publishes; the -ONNX container token; the bare -unsloth re-publisher marker; and -mxfp8. Date-then-flavor names such as Ministral-3-14B-Instruct-2512 resolve through a date-stripped form fed back into the peel loop. Behavior-changing derivatives stay unresolved by design, including -abliterated, -REAP-<params>, and -DFlash, since those denote a different model rather than a different serialization.
• Make uPlot chart axis labels and grid lines legible in dark mode on the Usage and Dashboard pages (#1304, PR #1308). uPlot resolves axis colors at construction time and paints them onto <canvas>, so its built-in #000 labels and rgba(0, 0, 0, 0.07) grid survived every .dark CSS override and left the plotted series as the only readable part of the chart. Axis, grid, tick, and border strokes now come from a shared theme helper, and mounted charts re-color on the Light and Dark toggle and on an OS theme change in Auto mode without a page reload. The drag-select rectangle and the crosshair cursor lines, which are DOM rather than canvas, are corrected in CSS.

v1.21.0 - 2026-08-11¶

Serve /v1/realtime so speech backends whose only interface is a bidirectional WebSocket become routable, gate every upgrade on Origin before anything else happens, and grow the model catalog from 189 to 230 entries from first-party sources.

Added¶

• Add GET /v1/realtime?model=<id> and the /realtime SDK compatibility alias, an OpenAI Realtime-compatible WebSocket proxy (#1299, PR #1301). Text and binary frames are relayed byte-identical in both directions with no protocol adaptation; the router routes, authenticates, bounds, and relays. This makes backends such as the NVIDIA NemotronLabs VoiceChat inference container reachable, since that container serves nothing but this WebSocket. Pre-upgrade handling resolves the model through the same 6-phase metadata matching pipeline as HTTP (so aliases and suffix normalization work), requires the audio capability, computes candidates through find_backends_for_model and filter_user_routing_candidates, and picks a backend through select_admissible_backend, the same selection and circuit-admission seam HTTP dispatch uses. When the caller's raw model string resolves only through an alias, the backend is asked for the canonical catalog id while the client-facing 404 still echoes what the caller sent.
• Add the realtime cargo feature, enabling axum/ws for the inbound upgrade and the already-optional tokio-tungstenite for the outbound dial (#1299). It is part of full and deliberately unreachable from embed, whose negative contract excludes tokio-tungstenite; scripts/assert-embed-feature-graph.sh still passes and cargo check --no-default-features --features realtime --lib proves the feature stands alone.
• Add the always-parsed optional realtime config section (#1299): enabled (default false), max_sessions (256), handshake_timeout (10s), idle_timeout (60s, "0" disables), max_session_duration ("0" unlimited), and backend_path (/v1/realtime), all on the shared parse_duration contract. Validation is wired into both impl Validate for Config and the real load-path funnel, so a malformed section fails startup, hot reload, config validate, and the admin config API alike. Disabling both timers warns rather than failing, because that combination leaves a session with no deadline and lets a half-open client hold a max_sessions permit until the OS TCP keepalive expires. The section is snapshotted when routes are built, so changes need a restart.
• Refuse cross-site WebSocket upgrades before any observable side effect (#1299). A WebSocket handshake bypasses both CORS and the same-origin policy, and unlike fetch it hands the initiating page full bidirectional read access, so the Origin check runs immediately after the endpoint gate and ahead of model parsing, model resolution, candidate computation, backend selection, and the session permit. A refused upgrade dials no backend, records no circuit event, and consumes no session slot. An absent Origin is allowed, since every non-browser SDK sends none and a browser cannot forge it; a present one is allowed when it is same-authority with the request or when server.cors.enabled is set and server.cors.allow_origins matches it through the same matcher the CORS layer uses. Everything else, including the opaque literal null even under a * pattern, gets HTTP 403 with error code origin_not_allowed. Browser origins are declared once in server.cors rather than in a second realtime-specific list.
• Bound realtime sessions on every axis (#1299): a max_sessions semaphore, 4 MiB message and frame ceilings on both legs, the dial under handshake_timeout, an idle timer refreshed by every frame, and an optional absolute session ceiling. HTTP timeouts never apply to a session. Relay runs one pump future per direction in a single select!, each owning the read half of its leg and the write half of the other, so close frames propagate with their originating code and reason; an orderly close on either side closes the other with 1000, and a peer vanishing without a close handshake closes the survivor with 1011. The send await is the only backpressure.
• Add four realtime metrics under metrics plus realtime (#1299): realtime_active_sessions, realtime_sessions_total over a closed six-outcome vocabulary, realtime_frames_relayed_total per direction, and realtime_session_duration_seconds. Call sites are always compiled and go through cfg-gated no-op seams.
• Add Admission::settle_success (#1299). A WebSocket upgrade answers 101, which settle_status would have mis-recorded, while a dial failure settles as a connection failure on the circuit HTTP failover already shares.
• Add 41 model metadata entries and fill in nine families that had none (#1298): Liquid AI LFM2.5 (11 entries), Ai2 Molmo 2, Microsoft Florence-2, Arcee AI Trinity and AFM, Xiaomi MiMo V2.5, the MiniMax back catalog through M3, Jina VLM, NVIDIA LocateAnything 3B, Pokee Isaac, Alibaba qwen3.8-27b, and the two Korean sovereign-AI foundation models a.x-k2 (SK Telecom) and k-exaone-2.0-750b-a37b (LG AI Research). Every value comes from a first-party source: HuggingFace config.json for context windows and expert counts, the vendor model card for parameter splits and licenses, and the vendor rate card for pricing. Where a vendor publishes nothing the field stays at the documented zero and an inline comment records why. Resolution was verified against 60 real HuggingFace repo ids covering quantization and container suffix chains.
• Add nvidia-nemotronlabs-voicechat-11b (#1300), NVIDIA's 11B full-duplex speech-to-speech model under OpenMDW-1.1, with the nemotron-voicechat-11b and nemotronlabs-voicechat-11b aliases registered explicitly because the vendor and lab segments are not peelable suffix tokens. context_window is the documented 0, because the repository ships a NeMo training config rather than a transformers config and the model card states no token limit; the backbone's own 128K window is deliberately not asserted for a checkpoint that never claimed it.
• Add muse-glimmer-30b (#1302), Meta's 30B dense multimodal agentic model under Apache-2.0, with context_window: 131072 taken from max_position_embeddings in the published config.json and the code capability grounded on its SWE-Bench scores rather than assumed. No aliases are needed: the HuggingFace repo form and the official GGUF quant tails all resolve through repo-prefix stripping and the existing peels, confirmed by seven id-form probes.

Changed¶

• Promote qwen3.8-max-preview to qwen3.8-max (#1298). Alibaba shipped it GA on 2026-08-03 with published rates and confirmed image and video input, while the entry still described a credit-only preview with zero pricing and no vision capability. The preview id is retained as an alias so existing callers keep resolving.
• Flag minimax-m2.5 pricing as UNVERIFIED in the catalog (#1298). MiniMax moved the model to its legacy catalog and stopped publishing a rate card, so the inherited rates are unverifiable. The comment warns against relying on them for cost routing rather than substituting an unsourced number.
• Redact Sec-WebSocket-Protocol and proxy-authorization in request logging (#1299). OpenAI's browser Realtime clients cannot set Authorization on a WebSocket, so they smuggle the API key through a subprotocol token that would otherwise land in the trace-level log line.

Fixed¶

• Correct minimax-m2.7-highspeed pricing (#1298). It was recorded at parity with standard M2.7; the highspeed tier bills at 2x.

Removed¶

• Remove mimo-v2-pro and mimo-v2-omni (#1298). Both were hosted-only endpoints with no weight release, and Xiaomi switched them off on 2026-06-30. The MiMo V2.5 entries supersede them. mimo-v2-flash is kept, because its MIT weights remain published and self-hostable.

Security¶

• Never log realtime frame payloads at any level, including error text (#1299). Both WebSocket libraries embed a lossy copy of a malformed frame inside their UTF-8 decode error, so read errors are reported through bounded classifiers instead of interpolating the error, and unit tests prove frame bytes cannot reach a log line.

Documentation¶

• Add the realtime operator pages docs/en/configuration/realtime.md and docs/ko/configuration/realtime.md, both in the Zensical navs, covering the browser-origin table and per-session memory sizing (max_sessions * 2 * 4 MiB worst case), plus session-lifecycle sections in the English and Korean architecture guides and a commented config.yaml.example block (#1299).
• Bump the configuration assistant to schema 1.36.0 with a realtime section entry, regenerate the IDE rule files including the "browser realtime client gets 403" row, and update the drift-test inventory in the same change (#1299). The section count is 36: 34 always-on plus 2 feature-gated.

CI¶

• Add a per-crate cargo-deny license exception for webpki-roots (#1299). The crate is the Mozilla CA bundle under CDLA-Permissive-2.0, a permissive data license, and it reaches the default graph through the rustls-tls-webpki-roots feature that realtime brought into full. Official release binaries already shipped it through control-plane, but cargo-deny resolves default features only, so this is the first build where it saw the crate. The exception is scoped to the one crate rather than added to the global allow list, so a future code crate under that license still fails loudly.
• Bump Azure/setup-helm from v4.3.1 to v5.0.1 (#1295), Azure/setup-kubectl from v4.0.1 to v5.1.0 (#1293), and actions/attest-build-provenance from v3.0.0 to v4.2.2 (#1294).

Dependencies¶

• Bump clap from 4.6.5 to 4.6.6 and lru from 0.18.1 to 0.18.2 (#1296), and refresh Cargo.lock against the current semver-compatible set.

Known Issues¶

• The dorny/paths-filter rust filter in .github/workflows/ci.yml does not list model-metadata.yaml, so a metadata-only pull request skips every Rust job even though several integration tests parse and assert on that file. The three metadata changes in this release were gated locally instead. Adding model-metadata.yaml and model-metadata.d/** to the filter closes it.

v1.20.0 - 2026-08-09¶

Hand backend topology and backend credentials to a Continuum Hub over an outbound-only channel, close the last gaps in TTFT telemetry, serve the image pricing and image limits keys the metadata loader had been dropping, and move the smart-routing LLM classifier onto the same backend implementations that serve production traffic.

Added¶

• Add control_plane.config_sync.backends, the Router-side backend_config_v1 executor for hub-managed backend snapshots (#1264). It carries an opt-in mode (disabled, overlay, authoritative), a durable identity-bound last-known-good snapshot store with an apply-id-keyed result outbox, atomic validation and hot reload through the single existing pool writer, graceful-drain evidence, and metadata-only convergence status on the heartbeat. Hub-supplied stable ids are used verbatim, uninterpretable tokens reject instead of being guessed, and pre-activation health checks run only for enabled backends that are new or materially changed. The coordinated continuum-protocol backend-config contract is vendored with its provenance recorded, and the hub's nine fixtures plus both canonical digest vectors are pinned by tests, so a canonicalization divergence between the peers fails a test rather than silently disagreeing.
• Add backends[].enabled, defaulting to true (#1264). A disabled backend is excluded from routing candidates and from every model listing surface (OpenAI, extended, single-model, Anthropic) while staying constructed in the pool, health-checked, candidate-probeable, and model-discoverable. Exclusion reads as model-not-found, never forbidden. Absent stays off the wire, so existing configurations are unchanged, and flipping the flag participates in hot-reload change detection.
• Add backend-scoped Hub credential bundles indexed by credential_id (#1269), advertised as backend_credentials_v1. This removes the one-credential-per-provider last-entry-wins limitation and closes the resolver seam that had rejected every credentialed snapshot as credential_unavailable. A hub-owned backend with a credential_ref resolves exactly that credential by stable backend id, works with no local api_key, and never falls back to a provider-wide or local secret; a hub-owned backend without a ref sends no Hub credential; local backends keep their existing precedence. The acknowledgment digest covers ids, providers, and versions only, never secret bytes, and the capability is advertised only when the complete executor is live.
• Add the outbound Hub backend task channel (#1270), advertised as backend_tasks_v1. The Router executes hub-authored backend candidate probes and redacted local backend exports over its own outbound connection, so the Hub can reach a candidate inside a customer network and read what a Router has configured without any inbound Hub-facing endpoint. The probe is the existing Admin primitive rather than a second implementation: its domain moved to the always-compiled src/backend_probe/ (because control-plane does not enable admin), src/admin_config/backend_probe_support.rs became a thin axum adapter, and Admin request and response shapes are unchanged. Both callers share one process-wide concurrency and rate ceiling.
• Report ttft_ms on both /v1/responses streaming usage taps (#1268). Previously both pushed usage records with TTFT absent, so a fleet whose traffic runs on the Responses API rendered empty prefill and decode panels. The router performs the upstream dispatch and forwards every streamed event, so both instants existed and were simply never taken.
• Record ttft_ms for non-streaming completions on /v1/chat/completions, /v1/completions, /v1/responses, and /anthropic/v1/messages (#1271). The span from provider dispatch to upstream response headers is a genuine router-observed first byte, not a fabrication. It is threaded through a request-scoped tokio::task_local! installed once for the whole API route group, so there are no signature changes.
• Stamp ttft_ms on local cache replays and pin the provider-batch exemption with a test (#1272). A replay writes its whole body in one shot, so first byte and last byte genuinely coincide and the derivation lives once in the single producer behind all seven replay sites. Replays report completion_tokens: 0, so decode and TPOT derivations are unaffected.
• Add per_image, image_input_tokens, and cached_image_input_tokens as optional PricingInfo fields (#1279). PricingInfo had exactly three fields and no deny_unknown_fields, so the shipped catalog's image prices were silently dropped at deserialization: nine paid image models advertised pricing: {input_tokens: 0, output_tokens: 0} and gpt-image-2 served only its text rates. per_image is an untagged flat-or-by-quality map in USD per generated image, a different unit from the per-1M token rates.
• Add the eight image limits keys ModelMetadata was dropping (#1284): max_prompt_length, supported_sizes, max_n, supported_qualities, supported_output_formats, supports_streaming, and dall-e-3's qualities/styles, normalized to supported_qualities/supported_styles. All are informational for API consumers; image request enforcement is unchanged. Unknown metadata keys now produce one load-time warning listing dotted paths labeled with model ids instead of vanishing, and are still never rejected, because metadata download must let an older binary fetch a newer catalog.

Changed¶

• Dispatch the smart-routing LLM classifier through Backend::execute_chat_completion instead of a private reqwest client with a hardcoded endpoint path and auth header (#1290, PR #1292). URL construction, protocol transformation, and effective authentication are now owned by the same backend implementations that serve production traffic, which removes the transport-divergence class rather than its known instances. The backend is resolved by name from the live pool on every classification call, so a hot reload cannot leave a stale transport alive. A classifier.llm.backend name that resolves in configuration but not in the live pool fails per call with the backend named and falls back to the rule classifier; no working deployment becomes a boot failure. The structured-output clamp to prompt_only now covers bedrock alongside anthropic.
• Pin ModelMetadata.pricing to USD per 1M tokens (#1276). The field was documented as per 1K while every shipped value was per 1M, and three independent facts in the code already read it as per 1M. Nothing in the catalog is rescaled; the doc comments, user-facing docs, catalog header, and WebUI label now agree.
• Reject non-finite and out-of-range smart_routing floats at config load rather than at use (#1286, #1288). Profile cost_per_1k_input_tokens and cost_per_1k_output_tokens, load_management.recovery.hysteresis_factor, load_management.thresholds[].error_rate, and two classifier floats are now validated on the real config path, which covers startup, the config watcher, control-plane config sync, and the admin config API. See Breaking Changes.
• Align every runtime retry duration with the config validator's contract (#1282). parse_duration accepts a positive unitless integer as seconds, so retry.initial_delay: "2" passes validation, but RetryHandler carried a private suffix-only parser and silently fell back to 100ms and 2s, with a third 10s fallback for the Retry-After cap. A validated 2s/8s policy could execute as 100ms/2s. All runtime paths now read the same contract, defensive fallbacks are one consistent constant per field, and each fallback logs a warning naming the field, the configured value, and the substitution.

Fixed¶

• Stop addressing a Gemini classifier backend with a doubled version segment (#1290, PR #1292). The router's own default GEMINI_API_BASE_URL ends in /v1beta/openai, to which the classifier's private client appended /v1/chat/completions.
• Dispatch a Bedrock classifier backend through its real transport (#1290, PR #1292). Every Bedrock classifier configuration was previously non-functional. On a build without bedrock-sigv4 the backend now returns its operator-actionable error naming the backend and its endpoint_type, which the classifier surfaces before falling back, so the failure is no longer anonymous.
• Bound provider error text on the classifier hot path (#1290, PR #1292). Backends in the Anthropic Messages protocol family embed the entire upstream body in the error they return, and that string was logged twice per failed classification, so one misbehaving classifier backend could write an unbounded amount of upstream-controlled text to the log on every request reaching the classifier (reproduced at 12,037 bytes per occurrence). The body is now truncated to MAX_BACKEND_ERROR_BYTES on a character boundary.
• Omit temperature from the classifier payload for Claude models that forbid sampling parameters (#1289, PR #1291). Anthropic rejects the parameter for Claude Opus 4.7 and later by its presence, not its value, so a Claude 4.7+ classifier model returned HTTP 400 on every classification request, including at the shipped default temperature: 0.0. No misconfiguration was required to hit it, and the failure was silent: the router burned a full round trip against timeout_ms and degraded to the rule classifier, so the operator saw smart routing working and never using the LLM they configured.
• Meter a /v1/responses cache replay exactly once (#1273, PR #1283). The replay pushed two control-plane usage events, and the second was marked served_from_local_cache: false, so the hub charged the matched key's TPM windows and monthly delta for a request that consumed no provider tokens. The other three inbound families were already correct.
• Correct auto-inferred smart-routing profile costs, which were 1000x too large (#1276). ModelTierRegistry::infer_from_metadata copied per-1M pricing straight into the per-1K cost_per_1k_* fields without the division its sibling performs. That reached cost-savings estimates that mixed inferred and configured profiles, model selection ranking, and the values GET /admin/smart-routing/model-profiles and the WebUI report. Explicitly configured and glob-pattern profiles are unaffected. See Breaking Changes.
• Label the admin WebUI model catalog price column per 1M instead of per 1K (#1276). The numbers were always correct; only the label was wrong, and it was the same mislabel that trapped a downstream consumer.
• Wire the previously dead PricingInfo and ModelLimits range validation into the real load paths (#1284). Violations warn once per model at runtime load and are a hard error from continuum-router metadata download, so a bad download can never replace a working file. The context_window and max_output bounds are reconciled to [0, 10000000] with 0 meaning not applicable, because the shipped catalog carries context_window: 0 on 17 models and max_output above the old ceiling on 43.
• Name the reason when metadata download cannot diff against a pre-existing local file (#1284), instead of silently treating the local file as if it never existed.

Breaking Changes¶

• A config.yaml carrying a negative, NaN, or infinite smart_routing model-profile cost now fails at startup, and a hot reload keeps the previous configuration (#1286). Previously a cost of exactly -1.0 produced an infinite model score, so that model won every candidate comparison in its tier and silently absorbed traffic intended for its siblings, with neither a config error nor a log line. An admin PUT /admin/smart-routing/model-profiles carrying such a cost now returns 400. The same validation descent also makes a pre-existing 200-character limit on model and model_pattern reachable from config load for the first time.
• smart_routing.load_management.recovery.hysteresis_factor and load_management.thresholds[].error_rate are now bounded to [0.0, 1.0] at config load, along with two classifier floats (#1288). Both were documented as (0.0 - 1.0) and enforced nowhere. A negative hysteresis_factor, an ordinary sign typo, made every stay-elevated comparison unconditionally true, so once the router escalated to Warning or Critical it never left and degradation stayed permanently applied.
• Auto-inferred smart-routing profile costs drop by 1000x to their correct figures (#1276). GET /admin/smart-routing/model-profiles rows and the WebUI smart-routing table change accordingly (an auto-inferred GPT-4o row that read 2.5 now reads 0.0025), and candidate ordering between an inferred and a configured profile can change, in the direction of the two being comparable at all.

Documentation¶

• Document the hub backend-sync contract across the assistant and IDE surfaces (#1264): the configuration assistant at schema 1.35.0 with the config_sync.backends.* and enabled rows, regenerated IDE rule files, the MCP fleet_backend_sync_contract resource and the explain_effective enabled projection, a config.yaml.example disabled-backend section, and new sections in the English and Korean architecture and backend-configuration guides.
• Document the metadata unknown-key warning, the range-violation warning, and the dall-e-3 key normalization together in the English and Korean advanced-configuration guides (#1284).
• Correct every remaining per-1K pricing sample in the English and Korean docs and in the shipped catalog header (#1276), including the /v1/models response example that contradicted the table 450 lines above it on the same page.
• Add bilingual technical reports for the TTFT telemetry family (#1268, #1271, #1272) and for the image pricing keys (#1279).

CI¶

• Scan each release image on its own platform (#1260). Each build step pushes by digest, so its digest names an index holding a single platform; the runner is amd64 and Trivy defaults to linux/amd64, so the arm64 scan aborted with no child with platform linux/amd64 in index before reading a single package. The v1.19.0 release failed on exactly this. TRIVY_PLATFORM is now set on both steps, so neither depends on the runner architecture.
• Pin tracing interest for integration capture harnesses (#1274, PR #1285). audit_decision_action_is_allow_in_monitor_mode failed roughly once in four full-suite runs and never reproduced in isolation, because tracing caches interest per callsite in a process-global table and resolves that cache through the rebuilding thread's own subscriber when at most one dispatcher is registered, so a sibling test with no subscriber latched the callsite off for the whole binary. The guard the crate-internal harness already had is now installed in the integration-test binaries, with a regression test that fails deterministically without it.

v1.19.0 - 2026-08-06¶

Changed¶

• Publish one container image instead of two, built on gcr.io/distroless/static-debian12 (#1256). The image carries the statically linked musl binary, a passwd file, and timezone data, and runs as uid 65532. The -alpine tag family is no longer published: :<version>-alpine, :<major>.<minor>-alpine, and :latest-alpine stop receiving new tags, and the Debian base is gone from :<version>, :<major>.<minor>, and :latest. Deployments that pinned an -alpine tag must move to the unsuffixed tag. The image has no shell and no package manager, so docker exec ... sh and kubectl exec ... -- sh no longer work; the deployment guide documents the ephemeral-container recipes that replace them, and the router's own config subcommands and --health-check run as the image entrypoint without a shell. The Debian base carried 22 CRITICAL and HIGH findings that no rebuild could clear, because every one of them was an unfixed Debian package the router never used.
• Remove Dockerfile.alpine and Dockerfile.alpine.ci. Dockerfile and Dockerfile.ci build the distroless image and consume the musl release archives.

Fixed¶

• Make the container HEALTHCHECK work (#1256). perform_container_health_check parsed the target as a SocketAddr, which only accepts a numeric address, so every hostname was rejected, including the localhost in the --health-check-url default. The declared health check had therefore failed on every published image since it was introduced in #198. The target is now resolved rather than parsed, and each resolved address is tried before the check reports a failure.

CI¶

• Install a published Trivy release in the release image scan steps (#1255). aquasecurity/trivy-action v0.33.1 defaults to Trivy v0.65.0, whose release assets upstream deleted, so the installer resolved the surviving git tag, logged found version: 0.65.0, and then failed on the asset download. The scan step died about 0.3 seconds in and no re-run could pass. The action is now v0.36.0 and all four scan steps pin Trivy v0.73.0 explicitly.
• Run the self-hosted macOS CI jobs on the self-hosted-macos-15-x64 runner (#1259).

v1.18.0 - 2026-08-05¶

Add two admin-authenticated discovery endpoints, one for a backend candidate that has not been registered yet and one for a single configured backend, and ship maintained Kubernetes deployment profiles with a supply-chain gate on release images.

Added¶

• Add POST /admin/backends/probe, a non-mutating probe for a backend candidate before registration (#1252). It accepts the same candidate fields as backend creation plus operations: ["health", "models"], is admin-authenticated on every Admin transport including Unix sockets, and is advertised by GET /admin/capabilities as transient_backend_probe_v1. The candidate stays request-local: nothing is written to active configuration, config history, backend pools, health state, circuit breakers, aggregation caches, environment files, or token stores. The response separates health.credential_status (valid, invalid, unknown, not_required) from catalog.source (live, curated, configured, unsupported), so a healthy unauthenticated endpoint cannot be read as proof of a valid credential. Model-list-capable candidates reuse the URL composition, authentication headers, parser, normalization, response-size cap, and per-backend model-count limit that registered discovery uses. Request bodies, concurrent probes, request rate, upstream time, response bytes, and returned model counts are bounded, credential-bearing URLs are rejected, and validation, health, authentication, upstream, timeout, parse, and response-size errors are sanitized.
• Add POST /admin/backends/{name}/models/discover, which returns one configured backend's live model catalog before the backends[].models allowlist is applied (#1245). It resolves exactly one backend by name and reuses the model fetcher normal aggregation uses, preserving timeouts, the response-size cap, the per-backend model-count limit, normalization, Codex account-plan filtering, and centralized OAuth refresh. It writes nothing: configuration, the aggregated model cache, health state, and circuit-breaker state are unchanged, and normal aggregated model visibility is unaffected. Native Anthropic, Bedrock, and non-Codex OAuth backends answer 501 with model_discovery_unsupported rather than falling back to configured model names, an unknown name answers 404 with backend_not_found, and upstream failures are machine-readable as backend_authentication_failed, backend_discovery_timeout, backend_discovery_network_error, backend_discovery_parse_error, backend_discovery_http_error, or backend_discovery_response_too_large without echoing access tokens, refresh tokens, token-store paths, or auth headers.
• Add maintained Kubernetes deployment profiles under deploy/ (#13). A Kustomize base with a staging overlay and a Helm chart with development, staging, canary, and production values cover authenticated configuration, hardened pods, TLS ingress, autoscaling, disruption budgets, and network isolation. monitoring/prometheus/ carries a standalone Prometheus bundle with a 10 GiB persistent volume claim and 15-day retention for clusters without the Prometheus Operator, and the Helm chart can render a ServiceMonitor where the operator CRD is installed.

Changed¶

• Share one live-discovery path between registered and transient backend snapshots (#1253, PR #1254). Candidate and registered discovery had duplicated provider and authentication branching; a single shared function now serves both from the validated backend snapshot, while token-store access stays registered-only.

Fixed¶

• Stop a transient OAuth candidate from reporting a synthetic healthy state (#1252, PR #1254). Probe health inherited the registered-backend active-check bypass and answered healthy with no network I/O, even though the lifecycle-managed authentication strategy a registered OAuth backend depends on does not exist before registration. Transient OAuth health now returns health.status: "unknown", credential_status: "unknown", and backend_probe_health_unsupported.
• Return the stable authentication error for health-only probes (#1252, PR #1254). A 401 or 403 on a health-only probe was collapsed into a generic health error, so a wrong key was indistinguishable from an unreachable endpoint. It now maps to backend_authentication_failed without exposing the upstream body.
• Bound configured candidate catalogs (#1252, PR #1254). A configured catalog read only models and applied no per-backend ceiling, so a probe response could exceed max_models_per_backend. models is now honored first with model_configs as the enhanced fallback, and every configured and curated catalog is truncated to the per-backend limit. Request-controlled configured Anthropic model ids no longer reach the existing info-level catalog logging.
• Reject unsupported transports and invalid enhanced model configuration before dispatch (#1252, PR #1254). An unsupported URL scheme reached lower fetch code instead of failing validation, and the probe-specific validation path accepted model_configs that normal backend validation rejects. Both now fail with typed, sanitized candidate errors.

Documentation¶

• Document both discovery endpoints in English and Korean, in the Admin API reference and the backend configuration guide, including the error codes, the non-mutation boundary, and when to use each one.
• Expand the metrics guide with the Helm ServiceMonitor object and the standalone Prometheus bundle, including its storage sizing, cluster-wide discovery RBAC, and NetworkPolicy caveats.
• Add deployment documentation for rolling, blue-green, canary, and configuration rollouts alongside the new profiles.

CI¶

• Add a Deployment Asset Validation job that renders every Helm profile and Kustomize overlay, validates the manifests with kubeconform, and validates each rendered router configuration with continuum-router config validate (#13). It runs only when deploy/, monitoring/prometheus/, the validation script, or the affected workflows change, and scripts/local-ci.sh runs the same script locally.
• Gate release container images (#13). Trivy scans the Debian amd64 and arm64 images and fails the release on CRITICAL or HIGH findings, buildx emits an SBOM with provenance: mode=max, and the multi-arch manifest digest is verified and attested. Workflow-level contents: write and packages: write were narrowed to per-job grants.
• Add deploy-staging.yml, a release-to-staging workflow behind a protected environment.

v1.17.0 - 2026-08-03¶

Execute hub-governed guardrail policy for the first time, enforce Hub cost-center V2 monthly limits locally, bound file resolution and Hub envelope decoding, make the per-key and per-client rate-limit dimensions fire on live traffic, and stream Files API transfers instead of buffering them.

Security¶

• Bind Responses file and session authorization to the authenticated API-key principal instead of the client-supplied X-User-ID header (#1244). File resolution, stored-session creation, previous_response_id replay, GET, and DELETE all derive the requester from AuthContext::user_id(), X-User-ID is filtered before upstream dispatch, and the spoofable cache and session fallback namespace is gone. Owned files and sessions on both the Responses and Anthropic paths now fail closed when no authenticated requester is present, and a Responses AccessDenied on file resolution answers 403 instead of forwarding the raw file_id. Deployments that partitioned Responses state by X-User-ID must move to per-key partitioning; the English and Korean guides carry the migration steps.
• Redact backend URLs and other reqwest-derived transport detail from client-visible error bodies (#1235). A shared client_safe_reqwest_error helper consumes reqwest::Error::without_url() at the response boundary, covering the Responses, Messages, count_tokens, image edit, multimodal embedding, and responses-only passthrough builders that previously interpolated the raw display text; the responses-only passthrough mapper now matches the Responses mapper and returns a static backend connection message. HTTP status, error type, error code, circuit-breaker accounting, and the full operator logs are unchanged.
• Escape and cap the untrusted log values #1161 left out of scope (#1179). Client-supplied multipart field names on POST /v1/files, the image edit and variation parser, and the admin Files API upload reached message through format-string interpolation, where EscapeGuard passes \n and \r through and nothing bounded the length. Backend-supplied values in the Anthropic subtree that #1161 did not touch, including the /v1/responses parse-failure body head and the full upstream error body during streaming, get the same escape-and-cap treatment.
• Cap the client-supplied values echoed into the disabled-thinking rejection body (#1183). disabled_thinking_effort_error_message interpolated the raw effort twice and the raw model id once with no length bound, and all three ingresses turn that message into a 400 body. Neither gate bounds its input first: effort_exceeds_high compares a trimmed, lowercased copy while interpolating the original, and claude_family_version inspects only the leading tokens. The cap now lives in the shared builder, so every ingress inherits it.
• Bound Responses file_id resolution per request (#1180, closes #1174), the same defect class #1169 fixed on the Anthropic path. The resolver enforced only the per-file 10 MiB MAX_INJECTION_SIZE, so any number of individually small references resolved with no aggregate ceiling, and each one holds its own base64 copy resident until dispatch. A count pre-pass now rejects above 100 references before the first metadata lookup, an aggregate byte budget is charged per reference from metadata.bytes before the content read (so a repeated reference is charged again), and the whole resolution phase carries the 30 second timeout the other two paths already had. A new src/core/files/limits.rs holds the bounds all three paths agree on, replacing two duplicated declarations and the comment asking that they be kept in step.
• Bound chat-completions local file_id resolution with the same 32 MiB aggregate raw-byte budget used by Anthropic Messages and Responses (#1240). The chat path now charges every reference from metadata.bytes before reading content, including repeated references to the same file, and returns a 400 invalid_request_error when one request exceeds the budget instead of forwarding partially resolved payloads. Chat-completions also now applies the 10 MiB per-file inline ceiling to PDFs as well as images; larger PDFs remain uploadable under files.max_file_size but are no longer expanded into a chat-completions JSON body.
• Bound the Hub policy envelope decode, which previously had no size cap (#1193). sync_policy now decodes through the same bounded reader as config sync under a derived ~152.6 MiB ceiling (MAX_POLICY_SYNC_RESPONSE_BYTES), written down term by term against the Hub's own distribution budgets (a documented 100,000-entry key-table allowance for the one member the Hub does not cap per org, 1,024 tiers with the 4 MiB aggregate request_params budget, the 256 KiB guardrail canonical budget, 5,000 cost-center assignments plus 1,024 capped centers, and explicit allowances for the members the Hub does not bound), sized so a legitimate tenant at every ceiling at once still decodes, which a new ceiling-scale round-trip test locks in. An over-limit refusal is loud and typed rather than a silent parse-class failure: a distinct HubError::ResponseTooLarge carrying the limit and the observed size, an error! log naming both (never body content), and a new control_plane_policy_sync_failure_total{reason="envelope_too_large"|"other"} counter separating "envelope too large" from an ordinary sync failure, because a persistent refusal leaves a restarted router permanently fail-open while an ordinary failure heals on its own. Both reason series are instantiated at zero at registration so a fleet alert can see them before any refusal, and the refusal count with its last-occurrence time is surfaced as policy_sync.oversize_refusals / last_oversize_refusal_ms on GET /admin/control-plane/status, where a nonzero count beside a null last_sync_ms is the cold-start fail-open signature. The policy store preserves the last-known-good snapshot across a refusal and never widens enforcement; a cold-start refusal leaves the store exactly fail-open. The WebSocket policy stream now sets its message and frame ceilings to the same constant, replacing tungstenite's 64 MiB default, which sat below the poll bound and would have refused on one path a legitimate large envelope the other path accepts. The remaining unbounded Hub response decodes (enroll, push_usage, push_probes) each get their own derived bound with the same typed refusal, the usage ack bound derived from the 10,000-record MAX_USAGE_BATCH_MAX_RECORDS clamp rather than the 500-record default, and an unreadable oversize usage ack is terminal for its already-processed batch rather than retried forever; heartbeat, report_config_apply_result, and push_batch_status decode no response body and are unchanged.

Added¶

• Add local enforcement of Hub cost-center monthly limits under the corrected V2 convergence contract (#1162; hub issues #688, #735, #736, epic #646). The vendored continuum-protocol crate syncs the complete cost-center surface from the hub at rev 98bb8ac, byte-for-byte with the hub's committed fixtures pinned by a new wire test, so the V1 member (cost_centers, cost_center_policy_digest) stays parse-compatible while remaining enforcement-inert and unacknowledged, and the V2 member (cost_centers_v2) becomes the enforced contract: a per-router stable allocation statement (Hub-pre-resolved key_id -> cost_center_id ownership, router fallback, provider/model and stable-backend fallbacks, per-center caps) plus the persisted dynamic budget snapshot it references (consumed-to-date counters, Hub-authored per-dimension verdicts, and the per-router included_router_usage_seq watermark). The statement is one atomic apply unit validated for bounds, duplicates, references, the router-scoped digest, the deterministic cursor, snapshot staleness, and watermark possibility, refused whole with a typed code (invalid_cost_center_policy, digest_mismatch, stale_budget_snapshot, invalid_usage_watermark) while the last-known-good unit stays enforced, and acknowledged exactly (stable cursor/digest plus dynamic budget cursor) on both the applied and rejected side of heartbeat status. Enforcement is local and the hub is never in the request path: the admission gates engage only on request paths that can produce a usage record the hub will meter; the key and router tiers are refused and atomically reserved (one request plus a deterministic token allowance: a bounded byte-derived prompt estimate plus the request's stated output maximum clamped to the tier's request_params ceiling and a hard bound) at the admission middleware, with the reservation riding the response body to the last streamed frame and counting toward every other admission so concurrent at-limit requests cannot all pass, while a single request is refused only at the cap and never on nothing but its own estimate; the lower tiers (stable backend, then provider/model, in the hub's own precedence) are refused at admission when every candidate serving backend for the effective post-rewrite target allocates to a cost center that is itself over cap, the condition under which the pre-dispatch decision cannot diverge from the identity that serves and is accounted, since every identity the request could serve under is exhausted (the candidates need not resolve to the same cost center: requiring that was sufficient but not necessary, and it let a client keep serving past a cap by naming a model whose backends spread across two exhausted centers); and accounting for every tier settles exactly once at the usage seam, where the single usage-loop consumer stamps the persisted monotonic router_usage_seq on each record entering the push buffer and ledgers its metered usage under the cost center computed from the final record's own identities, the same inputs the hub allocates from at ingest. Accepting a newer snapshot discards exactly the ledger entries at or below the hub-included watermark (unreflected usage is never reset, included usage is never double-counted); a hub watermark naming sequences a fresh runtime never assigned (a lost sidecar beside a surviving credential) recovers by a durable forward jump of the sequence space instead of wedging into permanent rejection; a populated statement arriving without a budget block is the documented budget-unavailable state (hub #750) and drops the held counters and hub-authored verdicts rather than applying them under new caps; a record permanently dropped from the push buffer, an unsplittable oversize batch, or a hub per-record rejection retires its ledger entry; and the sequence counter, last-known-good unit, applied watermark, and unreflected deltas persist in an owner-only sidecar next to control_plane.state_file (<state_file>.cost-center.json, 0600, atomic rename) so restart resumes the sequence space instead of restarting it. Sell spend is Hub-authored only: the router never prices, decrements, or infers sell spend locally, and the sell_spend_status verdict is the only sell-spend input (exhausted refuses; an unknown verdict from a newer hub never does). cost_center_limits_v2 is advertised only while the complete contract is live (sidecar attached, identity bound, sequencing and enforcement wired); the legacy cost_center_limits_v1 string is never advertised, and the documented overshoot stays bounded rather than claimed away, with four stated terms: the snapshot cadence times the fleet size, the reservation-estimate error, one allowance per concurrently admitted request, and the in-process handoff between a response finishing and its record entering the push buffer. Documented in the English and Korean control-plane architecture references.
• Add hub-managed guardrail policy, so a Continuum Hub can govern content-safety enforcement across a fleet while every router keeps ownership of execution (#1094, closing #1105, #1106, #1107, #1108, #1109, #1110, #1111, #1112, and #1122). v1.17.0 is the first release that executes hub-governed guardrail policy, which is the version the hub compatibility matrix should pin (lablup/continuum-hub#603); an older router ignores the field, never advertises the guardrail_policy_v1 capability, and is derived as unsupported rather than left pending forever. Governance travels in the existing policy envelope under control_plane.policy.enabled and is a closed, typed, integer-only surface: mode, category_thresholds (unsigned microunits, 1000000 is exactly 1.0), stages, inspect_reasoning, and block_behavior, with at most 64 route overrides keyed by resolved model id. Route scope covers mode and category_thresholds; a route-scoped stages, inspect_reasoning, or block_behavior has no slot in the local route schema and is declined with a warning naming the route and the knobs rather than folded into the global setting, because folding would widen a route-scoped decision to every route. Deployment wiring is never sourced from the hub: the providers list itself, provider endpoint / backend: / api_key_env, per-provider and global timeout_ms and on_error, every streaming_* knob, bypass_api_keys, the audit block, per-route enabled and providers subsets, and the global and per-route allow / deny match lists stay local and sovereign, and the policy type has no free-form payload field anywhere, so no prompt, completion, matched span, or redacted value can travel on it in either direction. Composition is strictest-wins per knob (enforce beats monitor from either side, the lower threshold floor wins per category and a stated category with no local counterpart is added, stages union into every provider row, inspect_reasoning ORs, and a stated block_behavior wins as a governance choice rather than a strictness ordering), recomputed whenever either input changes and applied atomically, so a local edit never drops the hub layer, a hub update never freezes local edits out, and a request is always evaluated against one whole policy generation. Delivery is tri-state rather than binary: an absent envelope member keeps the last policy, a stated policy replaces it, and an explicit clear returns the router to local-only configuration and is acknowledged with the canonical digest of the cleared policy rather than reported as absent, so a cleared tenant converges instead of resting at pending or stale. A router that started with guardrails.enabled: false, or with no guardrails: block at all, materializes its guardrail service at runtime when an enforceable policy arrives, through the same factory path startup uses and with local provider wiring only. A policy demanding enforcement this deployment cannot serve, meaning nothing that can gate at all or a demanded stage no runnable provider covers, is rejected whole with the typed code guardrails_unavailable and is never reported as active, so a Fleet view cannot show enforcement that is not happening; a body that fails validation is refused with invalid_guardrail_policy, which keeps "this router is older than the policy" distinguishable from the shared digest_mismatch. A refusal clears without hub involvement: the wire layer keeps the policy applied and re-evaluates on every local configuration change, so adding the missing provider row or deny rule makes the same policy take effect on the next reload. guardrail_policy_v1 is advertised only once a live guardrail-policy reconciler has attached, from the single capability list every status-writing path shares, because the capability is a promise about execution and not a build flag. The guardrail and tier tracks stay independent in both directions. Documented in the English and Korean guardrail guides (governance scope, local sovereignty, the composition table with a worked example, tri-state delivery, refusal codes, and the hub status derivation), the control-plane architecture reference, and the configuration-assistant skill doc (schema 1.30.0 to 1.31.0).
• Report the serving backend's stable configuration id on control-plane usage records and heartbeat inventory, so a backend can be an allocatable cost center (#1153, hub lablup/continuum-hub#686). The new optional backends[].backend_id is entirely operator-owned and nothing derives it: name is a display label an operator may rename at any hot reload, and a (provider, model) pair is not a backend because several backends can serve one pair, so a derived value would silently merge two distinct cost owners that share a pair and silently split one cost owner across a rename, both invisible once the usage has been ingested. A backend without an id reports none and its usage falls through to provider/model attribution exactly as before, which is a normal state rather than an error. Values are 1 to 128 characters from the delimiter-safe ASCII set [A-Za-z0-9._-], the same alphabet fallback.fallback_chains documents, because the value travels as an opaque key the hub matches byte for byte and never normalizes; whitespace, /, :, ,, control characters, and non-ASCII including bidi overrides are refused at the source rather than left to change which cost center a request charges. Uniqueness is required within one router and only within one router, since the hub keys membership on (authenticated_router_id, backend_id), so two routers may reuse one local spelling. The bound runs on the shared ingestion gate in infrastructure::config::validator, not only in the Validate impl, because the real load path validates sections explicitly and never invokes Config::validate(); rejecting the whole candidate there is also what keeps a hot reload atomic, leaving the running router on its last known good rather than a half-applied identity map. The /admin/backends handler family does not reach that gate at all (it publishes straight to the hot-reload watch channel through propagate_config_change), so it enforces the same bound at its own boundary, and POST /admin/config/validate checks it too so the pre-save dry run cannot report a green that the following write rejects. The id is resolved at each serving seam against the config view the selection actually used, and carried on the usage event rather than looked up by backend name when the record is pushed, because a hot reload in that window renames the backend and a later lookup would resolve to nothing and drop the attribution for usage already served: a fallback hop overwrites it exactly as it overwrites the TTFT origin so a failed-over request is charged to the backend that produced the response, the mid-stream relay resolves each hop from the snapshot the request was routed against, a provider batch carries the id captured at submit so a completion hours later still names the backend that executed it, /v1/responses streaming resolves at context construction because it has no dispatch seam, and a local cache hit dispatched no backend and reports nothing. The display name is never substituted when an id is absent. On the wire UsageRecord.backend_id is a verbatim copy of the upstream field and BackendInfo.backend_id is additive ahead of an upstream re-sync, since the hub holds no backend inventory and heartbeat is the only way an operator can discover which ids exist to assign; both are optional with serde defaults, so a router with no ids configured serializes byte-identically to the previous wire and PROTOCOL_VERSION stays 0. stable_backend_identity_v1 is advertised unconditionally because, unlike guardrail policy, there is no separate executor that may or may not attach: config validation, inventory, and the usage taps are compiled and wired as one unit, so the capability describes the implementation rather than the data and a router with no ids configured still advertises it and reports none, which is the state the hub must be able to tell apart from a router that cannot report at all. Documented in the English and Korean configuration and architecture guides, config.yaml.example, and the configuration-assistant skill doc (schema 1.32.0 to 1.33.0).
• Report metadata-only guardrail counters in the heartbeat inventory, so the hub can see fleet-wide guardrail activity without scraping every router (#1120). RouterInventory.guardrails carries checks_total, blocks_total, blocks_by_category, verdicts by (stage, mode, result), stream_buffer_cap_trips by (strategy, outcome), and a counters_since_ms stamped from the same process-start value as SupplySummary, so both counter families reset together. The counters come from an in-process tracker written at the same seams that emit the Prometheus families rather than from the registry itself, because metrics is optional and reporting zeros from a build without it would tell the hub that an actively blocking guardrail is idle. Every label space is closed, so a third-party moderation provider cannot expand the hub's stored key space or carry provider-authored text into it: any category outside the known vocabulary folds into other, and an unrecognized stage, mode, result, strategy, or outcome folds onto an unknown slot instead of being dropped. The block is reported whenever a guardrail service exists, even before the first check, and omitted entirely when the router runs none, so absent means "guardrails are off" while zeros mean "on and quiet". Counts and bounded ids only: no prompt text, no completion text, no matched span, no matched rule, and no redacted value.
• Add an optional in-flight request ceiling, so the router's per-request memory budgets finally imply a per-process bound (#1173). Every memory bound the router enforces is per request: the 10 MiB GLOBAL_BODY_LIMIT, files.max_file_size (512 MiB by default, buffered whole in memory), and the 32 MiB aggregate file-resolution budget added by #1169 and #1174. Worst-case resident memory is the product of those and the number of requests in flight, and nothing bounded the second factor. A rate limit does not supply it: it bounds arrivals per unit time, not residency, and by Little's law the steady-state in-flight count is the arrival rate times the mean duration, so per_client: 10 requests per second against completions averaging 30 seconds implies roughly 300 concurrent from one client entirely within policy, which is about 12.6 GiB at the file-resolution ceiling. The shipped default configures no rate limiting at all, in which case the ceiling is whatever a client can open. server.max_concurrent_requests is unset by default, so no existing deployment starts shedding on upgrade; set it and the router admits that many requests at once and answers the rest with 503, a Retry-After, and a JSON body naming the limit. It sheds rather than queues, because queueing trades an out-of-memory kill for unbounded latency while still holding the memory of everything queued. Implemented as a tokio::sync::Semaphore acquired with try_acquire_owned rather than tower's ConcurrencyLimit / LoadShed, which would have queued by default, needed a Buffer with a background task, given no control over the refusal body or the retry hint, and pulled new tower features into the embed graph that deliberately excludes iOS-hostile dependencies; this adds no dependency at all. The permit rides the response body, so a streaming completion holds it until the last frame is delivered, which means a ceiling tuned for short non-streaming traffic will starve streaming traffic and should be sized against the longest requests. The layer is mounted application-wide but inside both CORS and the rate limiter, so a shed 503 is readable in a browser and a request the limiter would refuse anyway never occupies a permit; it covers the separately-nested /v1/files group, which carries the largest per-request term of all, and exempts /health, /healthz, and the configured metrics.path, because shedding a liveness probe is how a correctly-degrading replica gets restarted mid-overload. SupplySummary::max_concurrency and shed_requests now report the real ceiling and the real shed count instead of promising the hub that no ceiling exists; an unconfigured ceiling still reports as absent rather than 0, since 0 would mean the router admits nothing. Memory-aware admission (charging a request its declared cost rather than counting requests uniformly) was evaluated and deferred: it needs a per-request cost estimate available before the body is read, and the only pre-read signal is Content-Length, which is absent under chunked encoding and does not predict cost anyway, since the #1168 vector was a 57 KiB body carrying 800 file references. The capacity arithmetic, the per-request terms with their config keys, and the interaction with rate limiting are documented for operators in the English and Korean deployment and performance guides, which also raise the Kubernetes example's 512Mi memory limit that a single default-size upload already exceeds.
• Add metadata for five models that shipped without entries, so /v1/models serves display names, pricing, capabilities, and limits for them instead of nothing (#1091, closes #1090): Claude Opus 5 (claude-opus-5, claude-opus-5-latest, which also joins the native Anthropic backend's built-in supported-models list), Kimi K3, Qwen 3.7-Max, Qwen 3.8-Max Preview, and Solar Open 2. Opus 5 needed no capability-gating code, since claude_family_version() already parses the id to (5, 0); new unit tests pin extended thinking, adaptive thinking, sampling-parameter handling, max effort, mid-conversation system prompts, and fast-mode eligibility so a refactor cannot silently regress it the way a substring gate once regressed Sonnet 5 (#854). Qwen 3.8-Max Preview keeps zero rates because preview access is credit-based with no published per-token rate, and Kimi K3 and Solar Open 2 omit knowledge_cutoff because neither vendor published one. solar-pro-2 also gains the tool-use capability and the Japanese language support its own summary already implied.
• Implement the files.retention_days startup sweep, which was configurable and documented but never ran (#1234, closes #1229). A nonzero value now deletes stored files whose metadata sidecar created_at is at least the configured age, and retention_days: 0 still keeps files forever. FileService::sweep_retained_files deletes metadata first, treats a missing content object as benign, restores the metadata if the content delete fails, logs each sweep, and reports file_retention_deletes_total. The read race boundary, the startup-sweep semantics, and the config examples are documented in the English and Korean guides, the Admin configuration schema, and the configuration-assistant skill.
• Complete BackendConfigBuilder coverage for BackendConfig (#1208). Setters were added for backend_id, internal, role, region, endpoint_type, auth, org_id, model_configs, retry_override, health_check, anthropic_auto_cache_control, anthropic_fast_mode, and external_storage; timeout and max_retries now mutate the same retry_override field as the explicit setter; backend_id format is validated in BackendConfigBuilder::build and uniqueness in ConfigBuilder::build. An exhaustive no-rest-pattern field inventory guard fails compilation when a new BackendConfig field is added without a builder decision.

Changed¶

• Reject a disabled thinking config paired with an effort above high on Claude Opus 5, at the router's own boundary (#1103). Anthropic accepts thinking: {"type": "disabled"} on Opus 5 only while output_config.effort stays at high or below and returns HTTP 400 at xhigh or max, so the invalid pair was previously forwarded and the client learned about it after a full round trip. The resolution is fail-fast rather than a rewrite: clamping the effort or dropping the thinking config would both silently rewrite an explicit client instruction, and disabled thinking on Opus 5 has two documented failure modes of its own (tool calls emitted as plain text instead of structured tool_use blocks, and <thinking> tags leaking into the visible response), so neither is a neutral fallback. The 400 states the constraint and names both remedies. The guard runs at every ingress: the shared OpenAI-to-Anthropic transform, both Anthropic-shape streaming entry points (TCP and Unix socket), the typed /anthropic/v1/messages handler after alias resolution and request_params application, and the Responses converter's independently built payload.
• Remove the dead Backend::max_concurrent_requests() trait hook and every production and test-double implementation (#1224). Nothing read it, so reporting the old hardcoded provider constants to the hub would have published an unenforced fiction. Per-backend control-plane inventory stays None, with the comment now pointing at the real router-level server.max_concurrent_requests admission cap added in #1173.
• Stream Files API uploads and downloads instead of materializing whole files in memory, and decide download authorization before any content is read (#1145). FileStorageBackend now has store_stream and retrieve_stream as its primitives, with the slice-based store and retrieve kept as provided wrappers, so peak resident memory per transfer is a 64KB chunk buffer rather than roughly 1.5x the file size. LocalFileStorage streams each upload into a per-process, per-upload temporary file and fsyncs plus renames it into place only after every check has passed, so a reader never observes a partially written .bin, and an upload that is rejected, errors, or is cancelled mid-flight leaves neither a partial file nor a leaked temporary. Content validation became incremental while reaching the same verdict as before: the first 13 bytes cover the image and executable magic-byte checks, and a new non-allocating Utf8Validator carries a partial multi-byte sequence across chunk boundaries, so a character split between two network chunks is judged exactly as it would be in a single buffer and a body ending mid-sequence is still rejected. GET /v1/files/{id}/content and GET /admin/files/{id}/content now resolve authorization from metadata alone before opening the file, closing a path where a caller who knew a file id but did not own it could force a full-size read and allocation and still receive a 403, and both then stream the response body instead of buffering it. The 413 semantics, the exact File too large: exceeds maximum N bytes message, and every other observable Files API behavior are unchanged. Because the temporary now exists for the whole transfer, an abrupt death (SIGKILL, OOM kill, node eviction, power loss) can leave one behind where no in-process cleanup can run, and nothing else in the product could see it: the orphan scanner matches only .bin and .meta.json, while the user-data retention sweep uses metadata sidecar created_at and ignores temporary upload dotfiles. Starting the router therefore reclaims every temporary older than 24 hours, ungated by cleanup_orphans_on_startup because an aged temporary is unambiguously garbage rather than possibly-recoverable data, and FileService::detect_orphans now reports the count so the leak is observable before the next restart. The 24 hour threshold is what keeps the sweep from deleting an upload another process may still be streaming; a temporary's mtime advances with every write, so a live transfer keeps refreshing itself out of range. commit also fsyncs the parent directory, so the rename is durable and not just the bytes it points at.

Alongside this, the upload routes' outer body limits are derived from files.max_file_size, clamped to the documented 1KB to 5GB range, instead of a hardcoded 512MB that silently capped anything larger, and a value outside that range is now reported as a startup warning on the real config-load path (warning rather than a hard error, because max_file_size: 0 is accepted today and a fatal check would stop a working router from starting; the clamp is what protects the routes). Relatedly, a body the transport-level cap truncated now answers 413 file_too_large rather than 400 invalid_request: both handlers classified every multipart failure as malformed input, which was academic while the cap was a fixed 512MB and became reachable once it derived from max_file_size. The previously unwired file_uploads_total, file_upload_size_bytes, file_upload_duration_seconds, file_downloads_total, and file_download_duration_seconds metrics are registered and emitted: an upload records its outcome from the session's drop, which is the only point that also sees an oversize rejection, a missing purpose part, or a client that disconnects mid-body, and a download is counted a success only when its body reaches the end without error rather than by virtue of having started. A startup warning fires when files.max_file_size exceeds a quarter of the cgroup v2 memory limit, staying silent when there is no limit, when it reads max, and on platforms without cgroups.

Fixed¶

• Stop two of the four self-hosted classifier templates from reading unrecognized guard-model output as a clean allow (#1203, follows PR #1182). verdict_llama_guard and verdict_yes_no took the first token of the model's answer and returned GuardrailVerdict::Allow for anything that was not their single positive token (unsafe, yes), so an error string, a refusal to comply, a chat-template artifact, a generation truncated by the 32-token cap, or an answer in an unexpected language was served as safe. It was silent: the parser produced an ordinary Ok(Allow), ProviderOutcome::failure stayed None, and under mode: enforce the Prometheus error family, the heartbeat's errors_total / fail_open_total / fail_closed_total, and the audit record all described an inspected, clean request. The lenient parsers were also the default path: an omitted template, an unrecognized template, granite_guardian, and shieldgemma all resolve to verdict_yes_no, so a single typo pointed a Llama Guard deployment at a parser where unsafe is not yes and every check allowed. Both parsers now recognize both poles explicitly, safe alongside unsafe and no alongside yes, and treat only genuinely unrecognized output as a check error, which is what Qwen3Guard and custom_classifier already did; the four templates now agree on the rule. Behavior change to check before upgrading: a deployment whose guard model emits output the configured template cannot read no longer passes silently. It now fails per the provider's effective on_error, which under the default fail_open means unchanged traffic with new error counts on guardrail_errors_total and the heartbeat's errors_total / fail_open_total, and under fail_closed means refused traffic. With api_format: completion the classified text is the entire prompt with no instruction wrapper, so a completion-served guard model that merely continues that text is the configuration most likely to see this, and it now fails loudly where it used to allow silently.

To avoid narrowing what a working deployment accepts, the verdict token is normalized deliberately rather than by fallthrough: surrounding whitespace, ASCII case, punctuation or markdown emphasis wrapping the token (**unsafe**, Unsafe:, Yes,), and a completed <think>...</think> reasoning block ahead of the verdict are all accepted. Whitespace and case were already accepted; the other two used to read as silent allows even when the model had answered unsafe, so they are now read as the positive verdicts they are. A reasoning block truncated before it closed carries no verdict and fails the check, which is the likely outcome at the 32-token cap these templates use. Nothing past the first token is scanned for a verdict word, because guard models restate both poles inside their prose ("this is not safe") and a scan would read an explanation or a refusal as a verdict, which is the failure this change exists to remove. The two allows that come from a real classification are untouched: an unsafe verdict whose reported hazard codes are all excluded by categories, and a positive verdict suppressed by a category_thresholds entry.

A second behavior change, in configuration validation. An unrecognized options.template on a self_hosted_classifier (or classifier) provider is now refused by Config::validate, where it previously fell back to granite_guardian with only a startup warn!. That fallback is what turned a typo such as llama-guard or shield_gemma into a total silent bypass, and because the failure direction was always permissive it could never produce the false block an operator would notice. A configuration carrying an unrecognized template value therefore stops loading, and continuum-router config validate reports it; fix the value or drop the key, since an omitted template is still valid and still means granite_guardian. The provider-side fallback is kept as defense in depth for construction paths that did not validate. The accepted values and the verdict-reading rules are documented in the English and Korean guardrails guides.

• Stop leaking the router's filesystem layout into client-visible error bodies on the Files API and the two file_id resolution paths (#1191). FileError::Storage(_) carries a message the router writes itself at roughly twenty sites in src/services/files/storage.rs and src/services/files/metadata.rs, every one of them a format! that interpolates path.display(), and map_file_error handed that string straight to the client as the message of a 500. A download of a file whose backing object had become unreadable answered with the absolute path under files.storage_path, the two-level sharding subdirectory, the stored .bin filename, and the underlying std::io::Error; a failed upload answered with the temporary filename, which temp_file_name builds as .{id}.{pid}.{sequence}.tmp and which therefore also disclosed the router's process id (1 under a container, itself a deployment tell) and the per-process count of uploads begun since start. Under the default files.auth.method: api_key the audience is every holder of a valid key with the files scope, including one whose enforce_ownership confines it to its own files, so a tenant that cannot read another tenant's bytes could still read the deployment's storage layout out of an error body; method: none widens it to anyone who can reach the port. The substitution is made once at each response boundary through a new FileError::client_message, not at the construction sites, because the path is exactly what an operator debugging a storage failure needs and the log is where it belongs: Display is untouched, so every error! and warn! site keeps the full text at unchanged fidelity, and the tests assert both halves through capture_logs. The status, the type, and the code are all preserved (500 / server_error / storage_error and io_error), so a client that branches on them sees no change and only the human-readable message becomes the fixed storage error: the file operation could not be completed. The client-caused variants keep their messages verbatim, since those name the caller's own file_id, purpose, or limit. Three client-facing surfaces carried the same string and all three are covered: the public /v1/files* handler, the chat-completions file_id path (where the resolution warnings are joined into FileResolutionError::PartialResolutionFailed and written into a 404 body, so the redaction happens where each warning is built and the full error is logged there instead), and the Anthropic /v1/messages path (where FileResolverError::ReadError reaches a 502 as File service error: ...). The Responses API needed no change and was left alone, since try_resolve_files degrades every non-limit error back to the original request so only LimitExceeded and Timeout ever reach a client. Two adjacent defects found by verifying the per-variant claim rather than assuming it are fixed in the same change. FileError::NotFound raised by the storage backend carried {shard}/{id}.bin rather than the file id, so a download whose backing object had been deleted disclosed the sharding scheme through a 404; FileService now relabels it to the id the caller supplied, which is both safer and a better message. And build_error_response in the admin Files API hardcoded "type": "invalid_request_error" in every body it built, including the 500s that map_file_error routed through it, so the admin API labeled a server-side storage failure as client-caused and disagreed with the public handler, which takes the type from error.error_type(); the type now follows the error, with tests pinning one admin 500 and one admin 4xx so the two classes cannot silently collapse again. The admin surface redacts on the same terms as the public one rather than keeping the detail behind the admin boundary: the operator's answer is in the log either way, and one policy for both surfaces is what stops them drifting apart, which is how the type bug arose. src/admin_config/prompts_api/handlers.rs was surveyed as part of the same sweep and carried a narrower instance: its PathTraversal rejection for a parent directory outside the base rendered the resolved absolute path, while the sibling check a few lines below already reported the caller's relative path, so it now matches; PromptFileError::IoError and InvalidPath gained the same client/log split.
• Stop a failing Azure Content Safety sub-check from discarding the other sub-check's refusal (#1194, follows PR #1182). AzureContentSafetyGuardrail::check_input_text inspects the input stage with two independent endpoints, Prompt Shields (text:shieldPrompt) and Content Safety text analysis (text:analyze), and #1182 combined them with ? when it converted the sub-checks to the reportable GuardrailCheckResult shape. The ? short-circuited: a Prompt Shields failure meant text analysis never ran, the whole check became an error, and under the default on_error: fail_open the service resolved that error to Allow, so content Azure text analysis would have refused was served instead. The symmetric case discarded a jailbreak block when text analysis was the endpoint that failed. Both sub-checks now always run, concurrently rather than back to back so a slow endpoint cannot eat the timeout budget the other one needs, and the outcome is chosen by whether the survivor already refuses: a surviving Block is returned and enforced, while a survivor that does not refuse (Allow, or a non-blocking Flag) still returns the failure so the fail policy decides the disposition and the counters from #1182 (errors_total, fail_open_total / fail_closed_total, and guardrail_errors_total{kind="error"}) still move. The test is "does the survivor already refuse" rather than "did the survivor find anything" because the adapter deliberately cannot see on_error and so has to return something correct under both policies: a Block is exactly as strict as the strictest verdict the fail policy could synthesize, while a Flag maps to allow at the gate, so returning one would serve a request fail_closed was configured to refuse and would leave the endpoint outage uncounted. That keeps both invariants at once rather than a plain "return whichever succeeded": a half-completed inspection that refused nothing never reads as a clean pass, and a genuine refusal is never swallowed because the sibling endpoint happened to be down. fail_closed behavior is unchanged, since an error and a block both refuse the request. The audit re-confirmed this was the only call site with that shape: check_output runs text analysis alone (Prompt Shields is input-only), and the Bedrock provider routes both stages through a single ApplyGuardrail call.
• Report the per-backend inventory identity under the hub's stable_id wire key, repairing a silent v0 wire defect from #1172 that made the entire backend cost-center tier inert (#1162 review, PR #1190). #1172 shipped BackendInfo.backend_id while the hub's independently written counterpart (hub #732) reads stable_id; both sides carry serde defaults with no rename and no deny_unknown_fields, so the hub parsed the router's key as absent, no backend ever became assignable cost-center inventory, and backend_assignments was always empty in practice. Verified against the hub clone at the pinned rev rather than inferred. The vendored field is renamed to the hub's key with the upstream doc comment adopted, the operator-facing backends[].backend_id config key and the separate (correct) UsageRecord.backend_id field are unchanged, and the hub's committed heartbeat_stable_backend_identity.json fixture is copied byte for byte and pinned in both directions with a guard that the identity never serializes under the local config key again. The same re-sync vendored the hub's committed policy_envelope.json as an envelope-level value pin, which immediately caught and fixed a second parse-fidelity gap: KeyEntry.project_id (hub #685) was silently dropped on re-serialization and is now carried verbatim, enforcement untouched pending #1152.
• Stop a dead backend from staying in the per-request rotation under load, and stop non-listed statuses from being recorded as circuit-breaker successes (#1148). Killing a primary backend mid-traffic preserved availability, because streaming fallback rescued every request, but first-token p95 stayed degraded for the whole outage: the dead backend was re-attempted on every request. Three distinct gaps produced that metastable state, and the reporter's proposed mechanism (that fallback's success prevents failures from accruing) was not one of them, since every fallback hop already records the primary's transport failure before advancing the chain. First, CircuitBreaker::record_failure converted a status outside failure_status_codes into record_success, which in the Closed state zeroed failure_count and pushed a success into the sliding window. That is the real defect the issue's title names: a backend interleaving 429s or 404s with genuine 5xx could never reach its threshold, and one answering only 429 looked healthy forever. Those statuses now record a neutral outcome that releases the half-open probe slot and touches nothing else, reported through a new OutcomeRecord return so the Prometheus mirror stops counting them as successes too. 429 stays out of the default failure_status_codes: a transient rate limit means the backend is alive and throttling, opening its circuit would convert throttling into an outage, and the retry path from (#740)/(#742) already honors the upstream Retry-After and fails fast on non-transient quota exhaustion. Second, the breaker is off by default, so every recording seam was a no-op and exclusion fell entirely to health checking, whose 90s default window (interval 30s x unhealthy_threshold 3) contains the reported degradation entirely. The breaker stays opt-in, because switching it on by default would change request routing for every existing deployment; instead the router now emits a startup warning when fallback.fallback_chains is configured without one, naming the failure mode and computing the exposure window from the configured health_checks values, and the three shipped config templates that configure fallback gained a circuit_breaker section. Third, a large set of dispatch paths never touched the breaker at all: image generation and image edit, the Responses-only passthrough bridge, native Gemini multimodal embeddings, the whole Anthropic Messages ingress (native, OpenAI-compatible, Responses-backed, Bedrock Runtime, Unix-socket, and web-search emulation), the Responses ingress (all four conversion strategies plus passthrough and compact), and Anthropic count_tokens. All of them now admit and record through a new proxy::circuit::Admission guard, which pairs admission with exactly one outcome and releases the half-open probe slot on drop, so the multi-exit OAuth-retry paths cannot leak probe capacity. Crucially, admission is never applied on its own: proxy::selection::select_admissible_backend filters candidates through filter_admissible, selects, admits, and re-selects on a lost race, returning the backend together with its guard. Pairing a dispatch-time gate with a selection step that does not know about circuits would have converted requests a healthy peer could serve into router-generated 503s. HalfOpen is the one state where a neutral outcome still advances the circuit toward closing, because that state asks whether to stop rejecting traffic rather than whether the backend is healthy; without it a backend answering only 429 would be pinned at its probe ceiling with no timeout to end it. The unwired second circuit state machine in services::health_service is documented as off the data path rather than mistaken for the real one.
• Converge the heartbeat guardrails block on the hub's shape, so a guardrail-active router delivers inventory at all (#1150). The router's GuardrailSummary and the hub's were written independently and did not match, and because RouterInventory.guardrails is a nested struct rather than a tolerated blob, serde failed the whole struct: a router with guardrails active and control_plane.enabled true delivered no inventory whatsoever, not backends, not health, not load, not supply, not policy_status. Verified against hub 58bb3d0 rather than inferred. GuardrailSummary now carries the six scalars the hub declares required (transforms_total, flags_total, errors_total, fail_open_total, fail_closed_total, stream_buffer_cap_trips_total), and the category breakdown is the hub's by_category object keyed by category id rather than the blocks_by_category array this router used to send, which is the one genuinely breaking wire change and is safe precisely because no hub could parse the old shape. transforms_total, flags_total, and stream_buffer_cap_trips_total are derived in GuardrailSnapshot::into_summary from the breakdowns they summarize rather than counted twice, so the scalar the hub reads and the breakdown a later hub will read cannot drift; a test asserts each against its sum. errors_total, fail_open_total, and fail_closed_total had no source in the control-plane tracker at all and are now recorded by GuardrailTracker::record_error from the same seam in src/services/guardrail/service.rs that records guardrail_errors_total, deliberately not read back from the Prometheus registry: metrics is optional, and reporting zeros from a build without it would tell the hub that a guardrail silently letting traffic through is idle. The two fail-policy counters split by the disposition that actually applied rather than by the configured on_error: monitor mode never gates, so a failed check under monitor is traffic that was served uninspected and counts as fail-open whatever the policy says, and reporting it as fail-closed would zero out the one counter that surfaces an enforcement hole. errors_total is a floor rather than a total, because only a provider timeout reaches that seam: a provider that fails fast resolves its own transport, status, or parse failure into a verdict internally, a gap the Prometheus surface shares and which needs an explicit failure marker on ProviderOutcome to close. The router-only verdicts and stream_buffer_cap_trips breakdowns stay as additive optional fields, ignored by today's hub because it uses no deny_unknown_fields. Category ids stay bounded in count, byte length, and charset by construction, since the only way a key enters by_category is a GuardrailCategoryId literal, and is_bounded() now checks all three against the hub's own ingestion rule. The hub's committed guardrail_summary.json is pinned here byte for byte, with the same "re-copy on any hub change, never regenerate from local types" rule the guardrail policy fixtures carry, because both repositories previously round-tripped only their own types and that is exactly how the shapes diverged while both stayed green.
• Report the guardrails configuration section as an immediate hot reload rather than restart-only (#1127). ConfigSection::Guardrails declared HotReloadCapability::RequiresRestart, which the Admin API surfaced to operators and which had been wrong for the mode flips, threshold edits, route changes, and provider rows that GuardrailService::update_config has always applied live. The enabled toggle no longer needs a field-level exception either: since the guardrail service handle became late-settable, a startup-disabled router materializes a service the first time a reload or a hub policy enables guardrails, and a runtime disable is expressed as enabled: false on the live service rather than tearing it down, so both directions are live. Nothing in the section requires a restart, and the section is now classified accordingly.
• Rewrite response-scoped backend tool-call ids to a reversible, per-response-unique form at the /anthropic/v1/messages boundary (#1143, closes #1142). tool_use.id is conversation-scoped in the Anthropic protocol, but vLLM synthesizes {tool_name}:{index} for Kimi-K3 with the index restarting on every response, so an agent calling the same tool on two turns was handed read_file:0 twice; Claude Code's ensureToolResultPairing then treated the replay as corrupted, substituted a synthetic "tool use interrupted" result, and ended the turn, with the router reporting nothing but 200s throughout. Outbound ids that are not already known conversation-unique (the toolu_, call_, fc_, and chatcmpl-tool- prefixes, and only when the whole id is charset-safe) are now rewritten to crt_<nonce:12><base64url_nopad(raw_id)>; the 12-character UUID v4 nonce is what actually breaks the collision, since base64url alone is deterministic and would reproduce the same client-visible id on every turn. The raw id is restored byte-exact on the way back in rather than charset-cleaned, because a backend has been reported to dead-stop when replayed a mutated form of its own id template. Decoding never fails a request: anything that doesn't match the encoded shape passes through unchanged. New src/http/handlers/anthropic/tool_id.rs; native Anthropic backends stream passthrough SSE and never reach it, and Gemini already synthesizes conversation-unique toolu_ ids of its own.
• Stop POST /admin/config/export with format: "toml" from failing unconditionally (#1159, closes #1138). Most optional Config sections carried #[serde(default)] without skip_serializing_if = "Option::is_none", so an unset section serialized to an explicit null, and the toml crate has no representation for a null; a default export carried 19 such nulls. The 21 remaining top-level optional Config fields now skip when unset, along with the nested optionals that were still leaking a null into a default export (ServerConfig.workers, ServerConfig.connection_pool_size, ResponseCacheConfig.{redis,l1,l2,tiered}, DisaggregatedServingConfig.default_external_storage, S3CacheLayerConfig.ttl_override, and RequestTimeoutConfig.image_generation), so GET /admin/config/full, POST /admin/config/export, and config-history snapshots now omit an unconfigured optional section instead of carrying it as null. get_section_value maps that absence back to Value::Null, so GET /admin/config/{section} on a known-but-unconfigured section still answers 200 with a null body rather than 404, and PATCH still merges onto that null base instead of failing with 500. The WebUI's structural-diff and history-diff helpers (config.js, history.js) now treat an absent key and an explicit null as the same state, so pasting an older export against the new response shape no longer renders a vanished null key as a spurious addition or removal.
• Truncate log strings on UTF-8 character boundaries instead of fixed byte offsets, so a non-ASCII value can no longer abort the whole router process (#1158, closes #1157). Eighteen byte-index &str slice expressions across twelve log-truncation sites in proxy, core::files, http::middleware, http::handlers::anthropic, infrastructure::backends::anthropic, and services::smart_routing panicked whenever the fixed offset landed inside a multi-byte character, and the release profile sets panic = "abort" (Cargo.toml:311), so the panic terminated the process instead of failing the one request. Two sites are reachable by an authenticated client without any unusual deployment: proxy::utils::sanitize_file_id_for_log truncates a raw file id once FileId::from_string has already rejected it, and a byte-for-byte duplicate that used to live in core::files::transformer truncated a validated id, because FileId::from_string's Unicode-aware alphanumeric check accepts non-ASCII values; both panicked on ids such as file-일이삼사오육. infrastructure::backends::anthropic::transform_openai_file_to_anthropic (a malformed multi-byte file_data preview) and http::middleware::admin_audit::mask_username (a multi-byte Basic auth username) are reachable the same way. New src/core/text_utils.rs adds truncate_on_char_boundary and truncate_tail_on_char_boundary, which back off to, or advance to, the nearest valid boundary instead of slicing blindly; every affected call site now routes through them, and the core::files::transformer duplicate was deleted in favor of the shared sanitizer. Old and new output were diffed exhaustively across every ASCII length from 0 to 700 for four alphabets rather than checked by inspection, so existing log formats and doctested output are byte-identical.
• Escape and cap client-supplied values recorded in Anthropic handler log fields, so an authenticated caller can no longer forge additional log records by embedding a newline in a value (#1161, closes #1151). field = %value wraps a value in DisplayValue, and tracing_subscriber's default formatter writes any field other than message as a bare {:?} with no wrapping, so the Display output landed in the log line raw; interpolating into the format string fared no better, because the EscapeGuard applied to message only rewrites ANSI and C1 controls and lets \n and \r through. Roughly 40 client-influenced sites across src/http/handlers/anthropic/ (tool names, tool-call and file ids, effort strings, the request model, the anthropic-version header, and enum discriminants such as AnthropicToolChoice::Simple) now record the value as a named field without the % sigil, so it routes through Visit::record_str and is formatted with Debug for str, which quotes the string and escapes \n, \r, \t, ", \, and non-printables into a single log line. Escaping alone stops forging but not bloat, so every one of those values also passes through a new core::text_utils::cap_client_value_for_log, which truncates at CLIENT_LOG_VALUE_MAX_BYTES (256 bytes) through the existing boundary-safe truncate_on_char_boundary rather than a byte-index slice. Router-controlled values (backend.name, url, status, typed enums, numerics) and two mime_type fields provably constrained to a fixed allow-list keep the % sigil, since escaping bytes that can never be client-chosen would only hide a real regression.
• Preserve live backend credentials when a masked configuration export is re-imported, instead of overwriting them with their own mask placeholder while reporting success (#1160, closes #1139). Exporting a configuration with the default include_sensitive: false and importing it straight back, the flow the WebUI's Configuration page presents, wrote every masked credential through as its own placeholder and answered success: true; the breakage surfaced later as 401s from every backend, with recovery meaning re-entering each secret by hand. The old placeholder shapes (sk...(21 chars), the bare ***MASKED*** for a value of four characters or fewer, ${***VAR***} for an environment reference) were derived from the secret itself, so none of them told a masked value apart from a real credential of the same shape. Every masked value the Admin API returns is now wrapped in a fixed sentinel, ***CONTINUUM-MASKED:<hint>*** (a value of four characters or fewer now reports (N chars) where it previously showed the bare ***MASKED***), and only that sentinel is read as "keep the value that is already live". POST /admin/config/import, POST /admin/config/apply with a config candidate, PUT/PATCH /admin/config/{section}, and the POST /admin/config/validate pre-save dry run all resolve placeholders the same way, so the dry run predicts exactly what a save will do; a write takes the configuration modification lock before splicing so a concurrent change cannot move a secret out from under the resolution. Import reports every preserved path through a new preserved_secret_paths response field, and every write path, including the dry run, reports the same set as MASKED_SECRET_PRESERVED validation warnings. Array entries resolve by id or name rather than position, so reordering backends still restores correctly; a list with neither, such as guardrails.bypass_api_keys or rate_limiting.bypass_keys, has only position to go on, so adding or removing an entry is refused rather than guessed, since two keys from the same provider can share a prefix and length and a shifted index would otherwise splice in the wrong one. A placeholder that cannot be matched to a live value fails the whole write with MASKED_SECRET_UNRESOLVED and the failing path named, rather than borrowing a neighboring secret. The three legacy placeholder shapes are still recognized, but only to refuse the import with LEGACY_MASK_PLACEHOLDER: a document exported by a router built before this fix is refused rather than silently trusted, and the way out is to re-export from the upgraded router or replace the flagged value with the real secret. Both the preserved-path and refused-path lists are capped at 500 entries with a MASKED_SECRET_REPORT_TRUNCATED marker, so a document filled with placeholders cannot inflate the response. Backend PUT/PATCH and the WebUI's Configuration and History pages recognize the new sentinel too. Documented in the English and Korean Admin API and WebUI guides.
• Enforce the per-API-key and per-client (IP) rate-limit dimensions on live traffic, which had never fired in production (#1164, closes #1147). Three wiring defects compounded: the limiter read a bare SocketAddr request extension where axum inserts ConnectInfo<SocketAddr>, so client_ip was permanently None and the per-client dimension, the IP whitelist, and the trusted-proxy X-Forwarded-For path were all dead; the limiter was layered on the whole application outside the API-key authentication that populates AuthContext, so the per-key dimension had no identity to bind to; and rate_limiting.limits.per_api_key was never read at all. Only the global dimension worked, so a single flooding tenant was not shed and collapsed a well-behaved tenant from roughly 22ms to 5600ms first-token latency. The limiter is now split by dimension across two mounts sharing one store: the global dimension is enforced by a single application-wide mount outside authentication and CORS, so every request is charged against limits.global exactly once, including API requests authentication later rejects with 401 (key guessing stays metered), failed admin authentication, CORS preflights, and the 404 fallback; the per-client dimension is enforced on the API routes outside authentication and the remaining identity dimensions (per-key, per-model) inside it (mount placement subsequently refined by #1170). Because the mounts enforce disjoint dimensions, no request is ever counted twice against any bucket. On refusal the refusing dimension's numbers are authoritative, and when the global and an identity dimension would both refuse, the client sees the global refusal because it is checked first. Two behavior changes for existing configurations. rate_limiting.limits.per_api_key is now the enforced default for every recognized key that has no api_keys[].rate_limit of its own, so a configuration that already carried that block will start shedding keys that were previously unlimited; an explicit rate_limit still overrides it and rate_limit: 0 exempts a key. And the 429 body is now a single valid JSON document (error.message, error.type, error.code, and error.details.limit_type), where it was previously a JSON error document re-embedded unescaped inside a second envelope and therefore unparseable; refusals now also carry retry-after and x-ratelimit-*, with Retry-After floored at one second per #1146. Dimensions skipped for missing identity (per-client on a Unix-socket listener, per-key for an anonymous caller in permissive auth mode) are now recorded in the new rate_limit_skipped_total{dimension} counter and a debug log.
• Stop the rate limiter from disclosing caller identity and shared fleet state through its response headers, bind unauthenticated floods on the dimension that exists for them, and stop a full bucket map from refusing traffic (#1170). Enforcement is unchanged throughout: every configured dimension still charges and still refuses exactly as before, and refusal attribution, Retry-After, and the 429 body are untouched. What changed is what the router reports and where two mounts sit. x-ratelimit-* now describes the caller's own quota and nothing else. The previous contract reported whichever dimension had the fewest remaining tokens, which made the numbers a function of who the caller was: with limits.per_api_key burst 7 and limits.global burst 100, a valid key read 7/6 and an unrecognized one read 100/99, the only key-recognition signal the router emits in permissive auth mode. The same headers also handed any unauthenticated caller a live read of the shared global bucket at one token per probe, on 401s, fallback 404s, and unauthenticated 200s such as /version, turning a global-exhaustion flood from blind into precisely timed; and because a bypass_keys holder produced the exemption sentinel and therefore no headers at all while any other token got them, with both responses 401, the limiter alone confirmed bypass-list membership. The rule is now that only quota tied to an authenticated identity is reported, which today means per_api_key: the caller has proved they hold the key the bucket is keyed on, so the numbers tell them nothing they do not own. Everything else is withheld on an admitted response. global, per_model, and per_backend are drawn on by the whole fleet, and per_client is drawn on by everyone behind one source address, so no shared level is the caller's quota; per_client is additionally evaluated before authentication, where the router cannot name whom it would be reporting to, and reporting it would have reopened the very oracle finding 3 describes, since a bypass_keys holder gets the exemption sentinel and no headers while any other token would get real ones on an otherwise identical 401. A response that dimension did not evaluate carries no x-ratelimit-* at all, uniformly, which removes the discriminator from the 401, 404, unauthenticated 200, whitelist, and bypass-key paths together. A refusal still reports the refusing dimension's numbers, including a shared one, because the client needs the retry hint and "the bucket is empty" is already implied by the 429. Two consequences worth checking before upgrading. Admitted responses now carry quota headers only when per_api_key evaluated the request; configure that dimension if your clients read them. And the permissive-mode residual is accepted rather than fixed: a recognized key's response carries per-key headers while an unrecognized token's carries none, so their presence still distinguishes the two. Suppressing them for unrecognized callers would only move the signal to their absence, so the decision is to state plainly that permissive mode is a development and trusted-network convenience rather than an authentication boundary, since it already serves valid and invalid keys identically; set api_keys.mode: blocking where key validity is sensitive. per_client moved outside authentication on the API routes, from inside it. The dimension keys on the peer address, which the connection supplies before authentication runs, so grouping it with the dimensions that genuinely need AuthContext left it unable to constrain the exact pattern it exists for: twelve bogus-key requests from one address in blocking mode all returned 401 and none were shed, leaving the flood metered only by the shared global bucket and shedding every other tenant as it drained. Mounting it on the same route group but outside auth binds the flood on its own bucket without widening its reach to WebUI, admin, or static traffic. The request path on an API route now reads global -> per-client -> authentication -> per-key -> handler, and the three mounts still enforce disjoint dimensions, so nothing is double-charged. The metrics exemption is derived rather than fixed. Matching the literal /metrics had both halves backwards: it exempted a route that did not exist when metrics were disabled, leaving an unmetered 404 sink, and it limited a scrape an operator had moved to a custom metrics.path, shedding exactly the monitoring needed during an incident. The exemption now follows the effective metrics.path and applies only while metrics.enabled is true. Keeping scrapes un-sheddable is deliberate, since a limiter that starves monitoring under load is worse than an unmetered scrape endpoint, so the residual is accepted and stated: a scrape's cost grows with metric cardinality and the limiter will not bound a flood of them, which is why metrics.auth and network policy matter. /health and /healthz stay exempt unconditionally so a saturated router keeps answering its own liveness probe instead of being restarted mid-overload. A full bucket map now fails open instead of closed. per_client, per_api_key, and per_backend each cap at 100,000 entries with a 600 second TTL, and hitting the cap returned a Capacity refusal; the client and backend checks had no already-present guard, so once full they refused callers that already owned a bucket too. Refusing saved no memory, because the refusal path inserted no bucket either, while 100,000 distinct source addresses are easy to source from a single IPv6 /64, so failing closed let an attacker convert a bounded map into a ten-minute outage for every caller on that dimension. A full map now skips that dimension for the request, leaving the others enforcing, and records it on rate_limit_skipped_total{dimension} plus a per-dimension warning rate-limited to one line per minute; the inline expiry sweep is throttled to at most once per second per map, so a flood of unique keys cannot force a 100k-entry scan on the hot path while the background sweep every 60 seconds keeps doing the steady-state pruning. Per-client buckets still key on the full address, so an IPv6 source varying its low 64 bits both evades the dimension and saturates the map; bound that at the edge. A 429 is readable in a browser whichever mount produced it. The global mount sits outside the CORS layer, which is what makes preflights chargeable, so its refusal carried no Access-Control-Allow-Origin at all and a browser turned the one refusal a client most needs to act on into an opaque network error. The mount does not move; its short-circuited refusal now carries an allow-origin echo for an origin listed in cors.allow_origins, Vary: Origin, and the credentials flag when configured. Separately, retry-after and the x-ratelimit-* headers are now always added to the CORS layer's Access-Control-Expose-Headers, because without them cross-origin script could read a refusal's status but not its retry hint, which is the actionable half, and that gap applied to every mount rather than only the outer one. Documented in the English and Korean rate-limiting guides.
• Bound Anthropic file_id resolution per request, so one small request backed by one uploaded file can no longer expand into gigabytes of resident memory (#1169, closes #1168). src/http/handlers/anthropic/file_resolver.rs fanned out over messages, content blocks, and blocks nested inside a tool_result with three unbounded try_join_all calls, and enforced its 10 MiB MAX_INJECTION_SIZE per file rather than per request, so roughly 57 KiB of request body carrying 800 references to a single 10 MiB upload resolved into about 10.7 GiB of base64 held simultaneously. Deduplicating repeated file_id values would not have fixed it: resolve_image_source and resolve_document_source call BASE64_STANDARD.encode once per reference, so every reference keeps its own encoded copy resident whether the bytes were read once or 800 times, which is why the aggregate budget added here counts every reference including duplicates rather than summing unique files. Three bounds now apply, each closing a different term. MAX_TOTAL_RESOLVED_BYTES (32 MiB of raw bytes per request, about 43 MiB once base64 expands it) is charged in load_file from metadata.bytes before any content is read, through a resolver-owned AtomicUsize whose running total is tested on the value fetch_add returns rather than with a check-then-add that would race the other resolutions in flight, so the bound is exact rather than approximate: a read-modify-write always reads the value written immediately before it in the atomic's single modification order, which makes every observed running total a prefix sum, so the references that pass the test are exactly a prefix of that order and their sizes cannot sum past the budget under any interleaving. MAX_FILE_REFERENCES_PER_REQUEST (100) is applied in a single pre-pass over all three traversal levels before the first metadata lookup; it is deliberately five times the chat-completions limit of 20 (src/proxy/files.rs) because the Messages API replays the whole transcript on every turn, so a long multimodal session legitimately accumulates references a single chat-completions call never would, and the count cap is a cheap pre-filter rather than the memory bound. MAX_CONCURRENT_FILE_LOADS (8) is enforced by a tokio::sync::Semaphore on the resolver, because the three fan-out sites nest and a per-level buffer_unordered(8) would multiply to 512; the permit is held across the storage read only and never across the recursive resolution above it, so a tool_result can never wait on a permit held by one of its own ancestors. The resolution phase also gained the 30 second timeout the chat-completions path already had, mirrored rather than imported because proxy::files is private to crate::proxy. Both new limits surface as 400 invalid_request_error and a timeout as 504, through a new FileResolutionResult::into_request_or_response that the streaming and non-streaming handlers now share in place of their two duplicated match blocks, so a resolution outcome cannot be handled on one path and silently missed on the other. The pre-existing per-file FileTooLarge still answers 502 and was left alone. Documented in the English and Korean API guides.
• Keep a classifier verdict that precedes a closing reasoning tag (#1241, follows PR #1214). strip_reasoning_prefix keeps everything past the last </think> on the rule that a reasoning block sits ahead of the verdict, so output where the tag lands after the answer (unsafe\nS1</think>) stripped to an empty head. Since #1203 made unreadable output a failed check, a real refusal became a failed check and, under the default on_error: fail_open, was served. That is the outcome #1203 exists to prevent, relocated rather than removed.
• Report a fast-failing guardrail provider as a failure rather than a clean allow (#1182). A transport error, a non-success HTTP status, or an unparseable body was resolved into a fail-policy verdict inside each adapter, so the telemetry seam, which fired only on outcome.timed_out, never saw it: under fail_open a moderation outage produced a heartbeat byte-identical to a healthy guardrail on clean traffic, and under fail_closed a flood of uncategorized safety blocks. The Guardrail trait now returns Result<GuardrailVerdict, GuardrailCheckError>, so a new adapter physically cannot swallow a failure and the fail policy is applied at the single service seam shared by the buffered and streaming paths, resolved per check from the live config snapshot instead of frozen at construction. The trait change is breaking for out-of-tree adapters. The PII provider keeps a local on_error for its documented degrade path only, where an external-recognizer failure under fail-open completes the check with builtin detections, which is a reduced-fidelity inspection rather than a failed check.
• Enforce the guardrails.allow and guardrails.deny match lists, which were parsed, regex-validated at startup, settable through the Admin API, and documented as enforcement features while no code in the request path ever read them (#1104). A new matchlist module compiles the global and per-route lists once per configuration snapshot, so no regex is compiled on the request path, and the config snapshot became a PolicySnapshot carrying the config and its compiled rules behind one lock, so a check can never read config from one generation and rules from another. Precedence is deny, then allow, then providers: a deny match blocks before any provider runs, an allow match short-circuits with no provider call, and route lists extend the global lists rather than replacing them, so a route override cannot weaken a global deny rule. exact entries are case-insensitive literal substring matches, since a case-only variation must not evade a security control; blank literals and empty patterns are dropped with a warning rather than matching everything. Matches are recorded under the reserved pseudo-provider labels match_list_deny and match_list_allow, so a monitor-mode match is measurable before enforcing, and the block reason never echoes the matched text or the rule. Enforce mode now requires at least one provider or at least one deny rule, where it previously rejected a deny-list-only policy that genuinely gates.
• Correct and complete the built-in OpenAI model catalog, which stopped at GPT-5.2 and left every newer tier to name heuristics (#1177). A discovered gpt-5.3-codex was advertised with a 256K context window, a 4096-token max output, and a fabricated \(10/\)30 per 1M price instead of 400K/128K at \(1.75/\)14. The GPT-5.5, GPT-5.4, and GPT-5.3 families plus gpt-5.2-codex were added to the built-in table, gpt-5.3-codex-spark and the three earlier codex tiers to model-metadata.yaml. gpt-5.1 was wrong in both catalogs in different ways and is now 400K/128K at \(1.25/\)10 with a 2024-09 cutoff and no audio capability it never had; gpt-5.1-codex and gpt-5.1-codex-mini are corrected likewise. Every codex tier is documented as Responses-API-only, so all four now carry responses_only and the router bridges Chat Completions requests for them to /v1/responses. Retirements were decided by cross-checking OpenAI's deprecations page against the Azure Foundry retirement schedule.
• Correct solar-pro-3 pricing, capabilities, and aliases (#1097, closes #1095). The entry advertised zero rates, which is this file's convention for open-weight models with no hosted API cost, so /v1/models reported a paid hosted model as free and any downstream cost estimation under-reported Solar Pro 3 spend to nothing; Upstage charges $0.15 per 1M input and $0.60 per 1M output tokens. The reasoning capability was missing while the entry's own summary claimed improved reasoning accuracy. The solar-pro3 and solar-pro3-260323 names Upstage actually publishes resolved to nothing, since neither the 4-digit nor the 6-digit date suffix is touched by the 8-digit date-suffix normalization, so both are now listed explicitly. solar-pro-2 gains its own solar-pro2 and solar-pro2-251215 aliases for the same reason (#1099, closes #1098).
• Rebuild the runtime state a configuration write drops, so an Admin API change no longer publishes a config with broken backend lookups and model metadata (#1100 closes #1089, #1136 closes #1116). put_config_section and patch_config_section round-trip the running configuration through JSON, which silently resets the #[serde(skip)] runtime state (backends_index, model_metadata_cache, response_defaults_cache) to its type default, and nothing rebuilt it, so a successful section PUT or PATCH broke backend lookup and metadata resolution until the next full config load or file-based hot reload. Both handlers now rebuild the index and reload metadata in the same order reload_config_from_file uses, and the follow-up extends the same treatment to the full-config apply, import, and rollback publication paths and centralizes index rebuilding across the create, update, delete, weight, and models mutations.
• Fail a hot reload when model-metadata assembly fails, instead of warning and publishing a revision with empty caches (#1102). publish_config_revision assigns the whole Config, so model_metadata_cache and response_defaults_cache were replaced wholesale and the router kept serving with pricing, context windows, capabilities, and /v1/models response defaults silently reverted to backend-reported values, with no restart to explain it. Metadata assembly was the one reload step that warned and continued where path validation, the oversize check, parse failure, env overrides, secret processing, and config validation all return Err and leave the previous revision serving intact; it now does the same. Only genuinely fatal conditions reach that path: an unreadable or unparseable base file, a merged document that does not match the metadata schema, and the MAX_METADATA_LAYERS (64) and MAX_TOTAL_METADATA_BYTES (8 MiB) caps. A missing base file and a drop-in that fails to read, parse, or validate are still warned and skipped. The cap breaches are what made this newly reachable, since anyone with write access to a model-metadata.d/ directory can add a 65^th file without touching an existing one.
• Validate every /admin/backends hot-reload candidate through the shared config validator before publishing it to the live watch channel (#1206), so the backend-specific Admin API agrees with the file and config ingestion paths. Validator failures map to 400 while watch-channel send failures keep their server-error behavior, and the backend_id-only Admin API validation helpers were removed because format and uniqueness now flow through the same whole-config gate as the rest of BackendConfig. Regression tests on create, update, delete, weight, and models prove a rejected candidate is neither published nor recorded in config history.
• Allow an empty backend pool at startup in both factory paths, honoring the existing zero-backend deployment contract, while preserving fail-fast behavior for a non-empty configuration whose backends all fail (#1134, resolves #1133).
• Defer half-open circuit admission to the dispatch seam, so selecting a backend for local preflight, cache, or guardrail work no longer consumes a probe slot (#1236, follows #1148). A deferred SelectedBackend token is produced only by circuit-filtered selection and consumed into an Admission immediately before upstream I/O, with a late admission race reselecting over the remaining eligible candidates; Anthropic count_tokens, Anthropic Messages, Responses, compact Responses, and the responses-only Chat-to-Responses bridge all moved to inspecting selected backend metadata before admission. Half-open progress is also split from genuine success evidence, so a probe window carrying only neutral outcomes or timeouts returns the circuit to Open with a fresh cooldown, while a window mixing neutral outcomes with real successes may still close.
• Deduplicate CORS exposed response headers by retaining the first parsed header name, so an operator-configured duplicate of a built-in header no longer appears twice (#1220). The previous Vec::dedup was adjacency-dependent and missed a configured retry-after, whose built-in counterpart is not adjacent in the combined list.
• Preserve per-key in-memory stats attribution when Prometheus metrics are disabled (#1209, closes #1196). API-key attribution and label sanitization moved into an always-compiled core module, and both the local and hosted CI matrices gained metrics-off regression coverage.
• Surface guardrail dry-run provider failures in POST /admin/guardrails/test and the Guardrails WebUI test console, so a fail-open substitute is distinguishable from a genuine allow (#1221). Sanitized failure metadata now travels through GuardrailService::dry_run, and the console renders fail-open and fail-closed result chips with the failure detail. Documented in the English and Korean WebUI guides.
• Surface PII external-recognizer degradation as local telemetry (#1223). The new guardrail_degraded_total{provider,kind} records kind="external_unavailable" when the PII provider falls back to builtin detections under fail-open because its optional external recognizer is unreachable, which is a reduced-fidelity inspection that guardrail_errors_total deliberately does not count. The built-in-only disposition and the fail-closed error behavior are unchanged.
• Restore the four BackendConfig struct literals #1172 broke, and close the CI gap that hid them (#1200). #[serde(default)] covers deserialization and does nothing for a Rust struct literal, so adding the required backend_id field required updating every complete literal in the same change; the two inside a get_health_check_config rustdoc example survived because nothing in CI compiled doctests, and the two Bedrock integration literals survived because bedrock-sigv4 was built by no job and no release binary. .github/workflows/ci.yml and scripts/local-ci.sh each gain two mirrored steps, cargo test --doc and an all-features cargo check --all-targets --keep-going, the second chosen over a narrow --features bedrock-sigv4 because it covers every unpinned feature at once and keeps covering features added later without a matrix edit.
• Add the no-op FileMetrics::record_retention_delete to the metrics-disabled compatibility surface (#1238), restoring compile parity for every no-default feature graph that includes the always-compiled Files service. The gap appeared only once #1234 and #1233 were combined on main.
• Clarify what the guardrail heartbeat verdicts and errors_total fields count (#1227), so they are read against the right units for a multi-provider stage and for fail-closed substitute blocks. The continuum-protocol rustdoc and the English and Korean guardrails guides now describe failed checks explicitly, bound the fail-open reading as an upper bound on uninspected allow traffic, and document where verdicts and blocks_total diverge under fail-closed.

Dependencies¶

• Enable the io feature on tokio-util for the streaming Files API paths (#1189), and make arc-swap a non-optional dependency now that the guardrail runtime service materialization path depends on it unconditionally (#1126). No crate was added to or removed from the dependency graph in this release.

v1.16.0 - 2026-07-28¶

Run the streaming output guardrail gate on every streaming path, make it redact instead of only block, keep it checking past its 4 MiB buffer cap, give monitor mode something to observe, and add metadata download with layered model-metadata drop-in directories.

Added¶

• Add layered model-metadata drop-in directories so a vendor baseline and operator customizations stop competing for the same file (#1069). model_metadata_file had exactly one slot, which forced a choice between never updating the shipped copy and hand-merging every upstream change, and made metadata download (#1067) unsafe next to local edits because it replaces that file wholesale. Metadata is now assembled from a layered search path, lowest precedence first: the base model_metadata_file, then /etc/continuum-router/model-metadata.d/, then <user config dir>/continuum-router/model-metadata.d/, then ./model-metadata.d/, then every directory in a new model_metadata_dirs config key in the order given. That is the inverse of find_config_file's first-match-wins order and deliberately so: expressed as layers, the more specific location wins by being applied last. model_metadata_dirs supplements the conventional locations rather than replacing them and its entries are used verbatim, so a Kubernetes ConfigMap mounted at /etc/router/metadata/ is listed as that path; it defaults to empty, so existing configs deserialize and behave exactly as before. Within a directory only regular .yaml and .yml files are read, in lexicographic filename order so the 10-/50-/90- convention works, with no recursion into subdirectories, dotfiles and *~ and *.swp skipped, symlinks resolved but required to land on a regular file, and a missing directory treated as empty rather than an error. Merging happens on the parsed serde_yaml::Value before deserialization, which is load-bearing rather than incidental: mappings merge recursively key-wise so an override carries only the fields it changes, sequences and scalars are replaced wholesale so an inherited aliases or capabilities entry can be removed, models entries are keyed by id so an existing id merges in place and a new id is appended, and responses_only keeps working because a plain bool with #[serde(default)] is indistinguishable from absent after deserialization, so a typed merge would let an override file that never mentions the key silently clear an inherited true. A new field on ModelMetadata merges correctly with no new merge code for the same reason. The merged document is deserialized and validated once, and backend model_configs still take precedence over it, which still takes precedence over the built-in OpenAI registry. A drop-in that fails to read, parse, or validate is skipped with a loud warning and the rest of the search path still loads, matching the router's existing warn-and-continue posture at startup and on reload so one malformed operator file cannot take down a running router; a malformed base model_metadata_file stays fatal exactly as before. Blame is assigned once, to the layer that was merging when the document stopped matching the schema, so an already-invalid base file is reported as the cause instead of every innocent drop-in layered on top of it. --resolved also stops exiting 1 on an invalid response_defaults, mirroring the router's own warn-and-fall-back behavior there instead of refusing to print the document the router actually serves, and both report renderings surface a config file that was found but could not be loaded, or that could not be found at all, as a # WARNING: header line and a config_warning JSON field rather than silently describing a search path that omits that config's model_metadata_file and model_metadata_dirs. At most 64 files totalling at most 8 MiB are assembled, and exceeding either cap is an error rather than a truncation, because applying a partial override set silently is worse than refusing to assemble. Because skipping is invisible to clients, a new continuum-router metadata show [--resolved] [--json] prints the search path, the applied layers in application order with every model id a later layer took over and the file it came from, and every skipped file with its reason; the default mode prints the raw merge verbatim so a typo is visible, --resolved prints the effective typed document after response_defaults validation and sanitization, and the human header is written as YAML comments so redirecting the output still produces a loadable file. The applied file list and every model-id override are also logged at startup. Reading a layer, including the base file, opens the path once with O_NONBLOCK on unix, checks it is a regular file through fstat on that same descriptor, and reads it through a take bounded by the remaining byte budget, so the type check, the byte cap, and the bytes actually read all describe the same object instead of three separate lookups of the same path; a fifo or device node dropped into an enumerated drop-in directory, or swapped in as the base file, can no longer block open with no timeout, and a file that grows or misreports its length after being stat'd can no longer defeat the size cap. Merging the models list is linear rather than quadratic in model count, indexing the accumulated entries by id once per layer instead of rescanning the whole list per overlay entry. ConfigSection::ModelMetadataDirs is also wired into FromStr, closing a gap where the section was advertised by the section listing and the admin JSON schema but rejected as unknown by the per-section admin GET/PUT handlers. metadata download still writes only the base file and never reads from or writes into a drop-in directory, which is what makes it safe to run repeatedly, and the --model-metadata flag keeps its meaning: it sets the base file, it does not disable drop-ins. Both model_metadata_file and model_metadata_dirs remain requires_restart. libc is now a cfg(unix) dependency for O_NONBLOCK, already present in the lock file, so no new crates are resolved. Documented in the man page, the English and Korean configuration guides, the example config, and configuration-assistant guidance (schema 1.27.0 to 1.28.0).
• Add continuum-router metadata download (visible alias metadata update) to fetch the canonical model-metadata.yaml from the repository and install it at the path the router actually reads (#1067). A release archive, Debian package, or container ships without the file, and an existing copy goes stale silently as metadata-only commits land new frontier models, so there was no supported way to pick those up short of cloning the repo. The destination resolves in a documented order (--output, then the global --model-metadata, then model_metadata_file from the loaded config with the same tilde expansion the loader uses, then model-metadata.yaml next to the discovered config file, then ~/.config/continuum-router/model-metadata.yaml) and is printed on the failure paths as well as the successful ones. --model-metadata sits above the config field because that is exactly what it does for the router itself, so --model-metadata /etc/cr/meta.yaml metadata download installs the file where that instance will actually read it. The download is parsed and run through the loader's response_defaults validation before anything on disk is touched, and unlike the loader (which warns and falls back to defaults) the command treats a validation failure as fatal, so a malformed or schema-invalid response exits 1 and leaves a working deployment byte-for-byte unchanged. Replacement is atomic through a temp file in the destination directory, created exclusively under a per-run unique name so a symlink planted at a predictable sibling path in a directory another user can write to cannot steer the write and two concurrent runs cannot interleave into one buffer file, preserves an existing file's mode (0644 for a new file, not the 0600 of the token store), and keeps the previous file as <file>.bak unless --no-backup. Content is compared by SHA-256 so an unchanged file is skipped without a write unless --force. --check writes nothing and exits 2 when an update is available, so cron and CI can branch on staleness without treating it as an error. Only HTTPS sources are accepted, redirects that leave HTTPS are refused rather than followed by reqwest's default policy, and the streamed body is capped at 8 MiB rather than trusting an advisory Content-Length. While the repository is private a GitHub token is required, because an unauthenticated raw request returns 404 for every ref: the token is read from CONTINUUM_GITHUB_TOKEN, GITHUB_TOKEN, or GH_TOKEN in that order, first non-empty value wins, whitespace trimmed, and there is deliberately no --token flag because a secret in argv leaks into shell history and ps output. The credential is attached only when the request host is exactly raw.githubusercontent.com or api.github.com, compared case-insensitively rather than by suffix, so a --url mirror or a lookalike such as raw.githubusercontent.com.evil.test never receives it, and it is never logged, printed, or placed in the --json report, which names the source variable instead of the value. Basic-auth credentials embedded in a --url are stripped the same way from every rendering of the source, so a private mirror's password does not reach the human report, the --json source_url field, or an error message. A 404 no longer conflates the two causes: without credentials it names the three variables and says the repository may be private, with credentials it says the ref or path may not exist or the token may lack access. --ref pins to a branch or tag, --url overrides the source for mirrors and air-gapped installs, and --json emits the same report as an object. The report covers source URL, resolved ref, destination, model count before and after, added and removed model ids, backup path, and both hashes, plus a reminder that updating the file does not hot-reload a running router. No new crate dependencies. Documented in the man page, the English and Korean configuration guides, and the configuration-assistant CLI quick reference.

Changed¶

• Mirror release container images to cr.backend.ai/product/continuum-router alongside ghcr.io (#1068). Each Debian and Alpine multi-arch tag is now pushed to both registries in one build step, and the final imagetools create sources its manifest from Harbor so a tag is only ever created from a manifest Harbor is known to hold, regardless of ghcr-side state. Downstream deployments get a source that does not depend on ghcr availability or per-user PATs. Nothing changes on the ghcr side.

Fixed¶

• Run the streaming output gate on every streaming path, in each path's own wire format (#1087, closes #1076). StreamingOutputGate had exactly one production construction site, the OpenAI chat streaming handler, so output guardrails silently did not run on any other streaming surface: /anthropic/v1/messages in all five reachable arms, Gemini through the SSE pipeline, mid-stream fallback, all four /v1/responses streaming strategies, the thinking-pattern transform, Bedrock runtime and converse, every Unix-socket streaming constructor, and the responses_only chat bridge. Each is now gated on what the client actually receives. A new Anthropic-wire driver renders a well-formed Anthropic terminal on a block (close the open block, refusal text block, message_delta, message_stop, or a single error event under block_behavior: error); the SSE pipeline gains an output-gate stage between the observability tap and the caching tap, with a shared veto so a blocked or masked stream is never stored; MidStreamFallbackContext carries one gate across every fallback hop so text held through a backend switch is checked as one completion, and a policy block completes the stream rather than triggering a fallback retry; the Responses passthrough arm defers the response.output_text.done and response.completed mirror events past the end-of-stream check and rewrites them from the gate's client-visible text, so a mirror cannot leak what the deltas masked. Gate construction also stopped hardcoding the global guardrails.mode: a shared build_streaming_gate resolves the per-route mode, the per-route enabled, and the bypass_api_keys allowlist, so a route overriding to enforce under a global monitor now gets an enforcing gate instead of a logged check that never blocks, and a disabled route or a bypassed key builds no gate at all instead of one that delays the whole stream only to short-circuit every check to Allow. stream_with_auto_backend_selection is the one out-of-scope entry point (library-only, no HTTP route registers it); its rustdoc records the decision and it warns once per process if an embedder reaches the ungated arm with guardrails configured. Monitor mode costs one output provider call per stream on each newly gated surface, which the guardrail guides now state.
• Apply Transform verdicts on the streaming output path, so output PII masking is no longer silently bypassed on the default traffic shape (#1075, closes #1074). finalize and run_window_check collapsed every verdict to is_block(), so the Transform a mask action produces fell into the allow branch: the original plaintext chunks were released and the masked text discarded, while the same request masked correctly when non-streaming. Held chunks are now released rewritten to carry the redacted text, with per-chunk source-shape detection so one gate serves both OpenAI choices[].delta.content and native-Anthropic content_block_delta payloads, and framing events (role opener, tool-call deltas, finish_reason, usage, message_start, ping, message_stop, [DONE], and anything unparseable) pass through byte-for-byte and in position. Under buffer_full that is a complete redaction, since the whole completion is held. Under chunked the masked window is stripped of its longest common prefix against the already-released context before release and spliced back over the checked region of the accumulator; without that splice the next window's context re-includes the original text, the provider redacts it a second time, and the client sees a duplicated placeholder. The built-in PII provider now overrides check_streaming_chunk instead of inheriting a default that returns Allow for every chunk, which had left it inert under streaming_mode: chunked. streaming_stream_first: true, and enforce with streaming_mode: passthrough, release before checking, so a masking verdict has nowhere to land; the gate warns once per stream and leaves the sent text alone rather than re-sending a masked copy. For OpenAI n > 1 the masked text collapses onto choice 0, which is documented: mangled for multi-choice streams, never leaking.
• Keep checking streamed output past the 4 MiB gate buffer cap (#1082, closes #1078). Reaching the cap failed open: the gate switched to passthrough, flushed everything it was still holding with no check having run, and released the rest of the stream unchecked, so one long completion turned output blocking and PII masking off for that entire response with a single warn! line as the only trace. The cap now degrades to chunked instead. The full-text check runs over everything held and the verdict is applied in full (a block cuts the stream, a Transform redacts the held chunks before release, only a clean verdict releases them), stream_first is forced off for the remainder because buffer_full was configured on the promise that output is checked before release, and the already-checked history is compacted down to the trailing streaming_context_size window so memory stays bounded while live checking continues. chunked streams now compact rather than degrade, so their behavior is otherwise unchanged. A new guardrail_stream_buffer_cap_trips_total{strategy,outcome} counter and one warning fire once per stream. The gate also stopped keeping a third full copy of the stream that existed only to measure the other two.
• Observe streamed output in monitor mode (#1086, closes #1079). Monitor performed zero output-stage evaluation on streaming responses: StreamingOutputGate::new folds every streaming_mode down to Passthrough in monitor (correctly, since monitor must never hold or cut), that arm forwarded without recording, and finalize's passthrough arm is guarded on a non-empty accumulator, so nothing was ever checked. Monitor is how a team decides whether a policy is safe to enforce and streaming is the default for chat UIs and agents, so a team could run monitor for a week, see no output-stage verdicts, and conclude their traffic was clean. The passthrough arm now accumulates when the mode is monitor, keeping only extracted text and never a held chunk, so nothing is held, delayed, cut, or rewritten and the end-of-stream check produces the verdict the rustdoc and the guardrail guides already promised. Enforce with streaming_mode: passthrough stays a true zero-check, zero-provider-call path. Monitor cannot take the cap's chunked fallback, because checking a window before releasing it means holding the stream, so it stops observing at 4 MiB and the truncation is counted as guardrail_stream_buffer_cap_trips_total{strategy="monitor",outcome="truncated"}. is_active() is no longer a safe skip signal and its rustdoc now says so: a monitor gate answers false and still needs every chunk. Monitor results recorded by older releases carry no information.
• Inspect array-shaped streaming delta.content, and bring reasoning text into output-guardrail scope behind a new opt-in (#1083, closes #1080). The streaming extractor skipped an array-shaped delta.content that its non-streaming twin read. A backend survey found no supported backend emits that shape today, so this closes an inspection asymmetry as defense in depth rather than fixing a live leak, but a future backend or proxy that streams content parts is now inspected and masked instead of bypassing the output stage. Reasoning and extended-thinking text was invisible to the output stage by accident rather than by decision, and is now inspectable through guardrails.inspect_reasoning (bool, default false), covering reasoning_content on both the streaming and non-streaming OpenAI-shaped paths and thinking_delta and thinking blocks on the native Anthropic path. It is off by default because reasoning roughly doubles the text volume providers see, which costs latency and, for network-backed providers, money, and it deliberately covers the non-streaming path too, since enabling it for streaming alone would recreate the very asymmetry the fix removes. The scope is an explicit TextScope threaded through every extractor, rewriter, and transform applier on both paths, and a table-driven test asserts extractability and rewritability agree across every chunk shape under both scopes, so the two sides cannot drift apart again. Configuration-assistant schema 1.28.0 to 1.29.0.
• Redact the verdict payload from guardrail debug logs (#1081, closes #1077). GuardrailService::evaluate logged the aggregated verdict with ?aggregated, Debug-formatting GuardrailVerdict and printing Transform::new_content (and, for a block, Block::reason) in full, bypassing the audit module's deliberately text-free design. Both debug! sites now log a redacted() view that prints only the verdict's shape: variant name, category, and score, never the payload. With the built-in PII provider the transformed text is already redacted, so this is log hygiene and defense in depth; the exposure it closes is a custom or self-hosted provider that puts raw content into either field.
• Guard test_legacy_routing_section_is_ignored_on_load with ENV_TEST_MUTEX (#1085, closes #1084). It was the only test in its module that read environment-sensitive config without holding the lock, so it raced the dozen or more siblings that set CONTINUUM_BACKEND_URLS and failed intermittently in CI once the guardrail work reshuffled parallel test scheduling. Test-only, and latent since the test was introduced in #1027.

Dependencies¶

• Bump the minor-and-patch group with 6 updates (#1073): tokio 1.53.0 to 1.53.1, tokio-util 0.7.18 to 0.7.19, clap 4.6.3 to 4.6.4, tokio-stream 0.1.18 to 0.1.19, aws-config 1.9.0 to 1.10.1, and libc 0.2.186 to 0.2.189.
• Bump actions/setup-python from 6 to 7 in the CI workflows (#1072).

v1.15.5 - 2026-07-24¶

Restore stateless multi-turn tool calling on /v1/responses for clients that replay the model's own output items, and stop startup from gating listener binding on unbounded backend probes.

Added¶

• Add health_checks.block_startup (bool, default true) so listeners can bind before backends are probed (#1066, closes #1065). At the default the startup sequence is byte-for-byte identical to before: connection pre-warm and the initial health-check round run inline and gate listener binding. Set to false, the listeners bind first and both steps run in background tokio::spawn tasks that log completion, so /health and /v1/models answer immediately while per-backend health converges through the admin and model endpoints. This matters when an orchestrator polls /health under a readiness cap (Backend.AI GO uses 30s) and a single unreachable or stalling backend pushes the pre-bind window past it. /health is a router-liveness probe that never reported backend health, so it stays meaningful during background warm-up. Documented in the English and Korean health-and-caching, configuration, and deployment guides, the example config, and configuration-assistant guidance (schema 1.26.0 to 1.27.0).

Fixed¶

• Keep replayed Responses input items convertible, so stateless multi-turn tool calling (store: false) works again on conversion backends (#1064, fixes #1063). A v1.15.0 regression (#1051) treated plain id and status as native-only metadata alongside the beta phase/agent/caller fields, so the message, function_call, and function_call_output items that every OpenAI-compatible client replays verbatim (including the model's own item id) became opaque passthrough values, and convert_to_anthropic and convert_to_gemini rejected the whole request with Request field 'native Responses input item' ... would lose it. The preserve-exactly heuristic now triggers only on the genuinely lossy beta fields, and only bounded standard id strings and recognized status values count as droppable replay metadata. Oversized, malformed, or unknown metadata stays byte-preserved, native-gated, and accounted for by request-size validation, so conversion still fails closed.
• Serialize replayed reasoning items byte-faithfully (#1064). encrypted_content, content, and status on a reasoning input item now skip serialization when absent, so a forwarded item no longer gains explicit nulls that OpenAI rejects with Unknown parameter: 'input[N].status'.
• Apply the configured health_checks.timeout as a per-request timeout on the startup health-check path (#1066). HealthChecker::with_shared_client reuses the router's shared reqwest client, whose global timeout is the 600s streaming total, so a backend that accepted a connection and then stalled could hold a health-check round for up to 10 minutes. The timeout now bounds every check on both the shared-client and dedicated-client constructor paths; the Duration is read out of the config lock and the guard dropped before the request, so no lock is held across the await.
• Bound the generic connection pre-warm HEAD request at 10s (#1066), matching the cap the Anthropic branch already used, instead of inheriting the shared client's 600s ceiling.

v1.15.4 - 2026-07-23¶

Accept the Anthropic-native x-api-key header in standard blocking-mode authentication, and key the hub policy hash, guardrail bypass allowlists, and per-key stats off the same presented credential.

Fixed¶

• Accept x-api-key in standard blocking mode (#1062, closes #1061). With api_keys configured in blocking mode, the standard (non-AppProxy) path of the dynamic auth middleware only read Authorization: Bearer, so Anthropic-native clients (the Anthropic SDKs, Claude Code with only ANTHROPIC_API_KEY set) presenting a valid key in x-api-key got 401. A shared extract_presented_api_key helper (Bearer preferred, x-api-key fallback, whitespace trimmed, raw credential never logged) now feeds the middleware in both blocking and permissive mode, the AppProxy ROUTER key extraction delegates to it, and the Anthropic messages, count-tokens, and models handlers derive the authenticated user from the middleware-attached auth context instead of re-requiring x-api-key. Removing the handlers' early return also means an authenticated request with a malformed anthropic-version header now gets 400 instead of silently passing.
• Key every credential consumer off the presented credential the auth middleware validated (#1062). The control-plane hub key hash now trims a whitespace-padded Bearer credential exactly like the auth path, so a padded Authorization header can no longer authenticate while evading hub rate, budget, and tier enforcement and usage attribution. The guardrail bypass allowlists (non-streaming, streaming, and Responses-only paths) and per-key stats attribution read the same presented credential, so x-api-key-only clients get correct bypass and stats, and a stray non-authenticated x-api-key that matches a bypass key cannot trigger a bypass when the request authenticated under a different Bearer key.

v1.15.3 - 2026-07-22¶

Enforce Qwen3Guard verdicts in the self-hosted guardrail classifier, isolate the AppProxy feature graph from the Admin API, fix empty tool-call arguments on the Chat-to-Responses bridge, and refresh five dependencies including the rmcp 2.x migration.

Added¶

• Add a native Qwen3Guard template to the self-hosted classifier so its structured plaintext verdicts are enforced instead of silently allowed by an incompatible parser (#1060, closes #1059). The template parses bounded, line-oriented Safety: and Categories: output for Safe, Unsafe, and Controversial results, maps Qwen's official taxonomy with case-insensitive selection and mapped thresholds (categorizing injection positives as jailbreaks), and adds a configurable controversial_action: flag | block | allow with a non-blocking flag default. Malformed output follows the effective fail-open/fail-closed policy. The classifier options, Qwen parsing, and tests are split into bounded modules under 500 lines each, and the configuration is documented in the English and Korean guardrail guides, the example config, and configuration-assistant guidance.

Changed¶

• Make the AppProxy feature graphs compile in isolation without transitively activating the Admin API (#1058, closes #1053). The process-wide config-mutation lock moves into infrastructure::config and CIDR-aware IP allow-list evaluation into a feature-independent HTTP middleware helper, both with compatibility re-exports, and appproxy-common no longer enables admin. An isolated --no-default-features AppProxy build therefore neither compiles nor mounts the Admin route tree, while official release builds still receive Admin through the default full feature set, so their behavior is unchanged. Isolated appproxy-common, appproxy-router, and control-plane,appproxy-router checks are added to CI.

Fixed¶

• Preserve function-call arguments when a Responses stream provides the complete JSON only on response.output_item.added or done events, so Chat Completions tool calls no longer reach clients with an empty arguments string (#1057, closes #1056). The converter buffers arguments seen on output_item.added per output index, tracks deltas already forwarded to each tool call, and reconciles the terminal events by emitting only the longest prefix-compatible missing suffix.

Dependencies¶

• Bump rmcp from 1.8 to 2.2 and migrate the MCP resource listing to the non-annotated Resource/ResourceTemplate builders, since RawResource, RawResourceTemplate, AnnotateAble, and no_annotation() were removed in the 2.x API. Bump tower-http to 0.7 (cors layer), object_store to 0.14 (s3-cache), aws-smithy-eventstream to 0.61 (bedrock-sigv4), and tokio-tungstenite to 0.30 (control-plane policy stream).

v1.15.2 - 2026-07-21¶

Ship the Windows release binary with the MSVC C runtime statically linked so it launches on a clean Windows install, and refresh the dependency tree.

Fixed¶

• Statically link the MSVC C runtime in the x86_64-pc-windows-msvc release build so the shipped continuum-router.exe no longer requires the Visual C++ Redistributable (#1055). Every Windows release since v0.23.1 linked the CRT dynamically, so on a Windows machine without the VC++ 2015-2022 Redistributable the binary failed to start with a missing VCRUNTIME140.dll error. The release workflow now builds the Windows target with -C target-feature=+crt-static, and because that job builds with an explicit --target, the flag applies only to the final binary and not to host build scripts or proc-macros. macOS and Linux builds are unchanged.

Dependencies¶

• Bump a group of 12 minor and patch dependencies (#1052): tokio 1.52.3 to 1.53.0, redis 1.3.0 to 1.4.1, uuid 1.23.5 to 1.24.0, serde 1.0.228 to 1.0.229, serde_json 1.0.150 to 1.0.151, thiserror 2.0.18 to 2.0.19, async-trait 0.1.89 to 0.1.91, clap 4.6.1 to 4.6.2, futures-util 0.3.32 to 0.3.33, fastrand 2.4.1 to 2.5.0, regex 1.13.0 to 1.13.1, and toml 1.1.2 to 1.1.3.

v1.15.1 - 2026-07-20¶

Align the GPT-5 family across the OpenAI Chat Completions and Responses APIs. This release registers the GPT-5.6 models and GPT-5 Pro, validates reasoning_effort against the exact per-model matrix without silent downgrades, and preserves compatible request fields across the Chat-to-Responses bridge instead of dropping them.

Added¶

• Register the GPT-5.6 Sol, Terra, and Luna models with family aliases, wired into OpenAI model normalization and reasoning-effort handling (#1051). Model metadata covers the new generation, and the family aliases resolve to the concrete models.
• Add GPT-5 Pro as a Responses-only model so /v1/chat/completions transparently bridges to /v1/responses (#1051). GPT-5 Pro is Responses-only upstream, but the router keeps Chat Completions compatible through its existing Chat-to-Responses bridge.
• Expand the Responses API schema for reasoning context and mode, text verbosity, prompt caching, programmatic tool calling, multi-agent items, native replay metadata, safety identifiers, context management, and tool-call limits (#1051).

Changed¶

• Validate OpenAI reasoning_effort against the exact GPT-5 generation and model matrix on both streaming and non-streaming Chat Completions requests (#1051). The original gpt-5 family accepts minimal, GPT-5.1 starts at none/low, GPT-5.2 through GPT-5.5 add xhigh, GPT-5.6 adds max, and gpt-5-pro accepts only high. The router never rewrites auto, xhigh, or max to another effort, and OpenAI models no longer pass through the former Gemini-normalization interference.
• Preserve compatible Chat fields across the Chat-to-Responses bridge, including flat verbosity, metadata, safety identifiers, and prompt-cache controls, rejecting invalid values instead of dropping them (#1051). Native OpenAI persisted-response state and beta headers are kept intact, while local session state is resolved only for conversion strategies and native previous_response_id values are forwarded unchanged. Prompt-cache TTL validation accepts the documented 30m value.

Fixed¶

• Fail closed with unsupported_request_parameter when native-only Responses controls would otherwise be silently dropped by non-native conversion paths (#1051). Native Responses requests carrying GPT-5.6-only controls bypass response caching and are rejected on conversion-only backends rather than degraded silently.

Breaking Changes¶

• OpenAI Chat Completions now rejects unsupported model and effort pairs with 400 invalid_request_error instead of silently rewriting the effort (#1051). A client that previously relied on the router adjusting an out-of-range reasoning_effort receives an explicit error and must send a value the target model supports.

v1.15.0 - 2026-07-20¶

A configuration truth pass. A manual code audit of the v1.14.0 tree found configuration knobs and endpoints that the schema accepted and the manuals documented but that no production code path honored. This release wires each one into the runtime so the documented behavior actually takes effect, adds the environment overrides and interpolation the manuals already described, and finishes with a documentation alignment audit and a post-merge hardening pass.

Added¶

• Interpolate ${VAR} environment references in every string-valued config field, not just the previously hardcoded secret/URL/path set (#1039, closes #1032). The loader walks the parsed YAML/TOML tree and resolves ${VAR}, ${VAR:-default}, and a $$ literal-dollar escape before typed deserialization, with variable-name validation and bounds against oversized values and excessive substitutions. .env files load before parsing so their values feed interpolation, an unset required reference aborts startup and hot reload with a path-aware error, config validate reports unset references as warnings, config show --resolved and config diff interpolate (diff still redacts secret-named fields), and config show plus the MCP surface keep references literal and read no environment values.
• Add a validated model_aggregation config section wired into the runtime (#1042, closes #1022). The section maps to the aggregation service, cache TTLs, fetcher and entry limits, deduplication strategy, force-refresh gate, and background refresh task, so allow_force_refresh and background_refresh are operator-controllable instead of pinned to defaults. Reload class is restart, because the cache and services are built at startup. Schema count moves to 35 with the addition.
• Read the CONTINUUM_SELECTION_STRATEGY environment override in the config loader (#1037, closes #1023). It was documented but never read; the loader now parses all six strategies with the same spelling rules as --selection-strategy, an unrecognized value fails the config load with a clear error instead of being silently ignored, and precedence is --selection-strategy > CONTINUUM_SELECTION_STRATEGY > file value > RoundRobin default.
• Read the four CONTINUUM_FILES_AUTH_* Files API auth environment overrides in the config loader (#1046, closes #1033). CONTINUUM_FILES_AUTH_METHOD, CONTINUUM_FILES_AUTH_SCOPE, CONTINUUM_FILES_ENFORCE_OWNERSHIP, and CONTINUUM_FILES_ADMIN_ACCESS_ALL were documented but inert; they now apply to files.auth, construct a default FilesConfig when the file has no files: section, and the two booleans accept only literal lowercase true/false, rejecting 1, 0, yes, no, and mixed case with a clear error.
• Emit the X-Fallback-Attempts response header the router already advertised (#1035, closes #1024). The fallback path discarded the attempt count before building the response; the header now reports the total model attempts including the primary, appears only when a fallback backend served the response (minimum value 2), and is gated by the per-model notify_on_fallback setting. Streaming responses flush headers before the body, so mid-stream recovery stays observable through the streaming_fallback_* metrics instead.

Changed¶

• Align the English and Korean manuals, CLI --help, generated configuration guidance, Debian packaging text, Docker examples, and the manpage with the implemented router surface (#1034). Historical, aspirational, and nonexistent-feature claims were removed from user-facing docs while real endpoint, feature-gate, authentication, fallback, circuit-breaker, environment-expansion, model-aggregation, and hot-reload boundaries were documented, and Admin and MCP hot-reload metadata and regression tests were aligned with actual wiring. This audit surfaced the follow-up findings that the rest of this release closes.
• Bump the Debian Build-Depends Rust floor from 1.75 to 1.96 to match the lablup PPA, and reconcile the packaging docs (#1021, closes #1020). Corrects the false edition-2024 MSRV claim (the crate is edition 2021), the wrong Noble codename mapping, and the distro guidance to state that stock Ubuntu series do not ship Rust 1.96 and the lablup PPA must be enabled.

Fixed¶

• Drive the circuit-breaker state machine from ordinary proxy traffic (#1044, closes #1028). The breaker was implemented, configurable, hot-reloadable, and Admin-visible, but no LLM proxy request drove or consulted it, so enabling it did not protect the routing path. Selection now excludes open circuits, a pre-dispatch gate admits or rejects, and terminal outcomes are recorded across Chat, Completions, Responses, Embeddings, Images, Rerank, and every fallback hop plus standard streaming. Upstream 5xx and configured failure_status_codes open the circuit, timeouts honor timeout_as_failure, transport errors are unconditional failures, client-side 4xx are not recorded, and admission rejections become a retryable error so selection and fallback advance to another backend before returning the standard 503.
• Apply retry.* and timeouts.* hot-reload revisions to request execution (#1045, closes #1029). Both were classified hot-reloadable, but the request path read only the startup snapshots, so a reload changed the config and reported success while effective behavior stayed at startup values. Each request surface now snapshots the live policy and timeout budget at entry, keeping one coherent policy per in-flight request; the reqwest client connection and overall timeouts remain restart-only because they are baked in at client construction.
• Apply live selection_strategy and prefix_routing.load_factor_epsilon changes to the running BackendPool (#1038, closes #1025). A watcher-driven strategy change was detected and advertised as applied, but HotReloadService never pushed it to the live pool, so routing kept the startup strategy. Both are now shared lock-free cells swapped atomically while preserving backend membership, per-backend stats, in-flight accounting, and hash-ring safety, and the Admin, MCP, and docs surfaces report them as immediate.
• Wire the accepted-but-inert prefix_routing.virtual_nodes and anthropic_cache_control_injection knobs into the runtime (#1041, closes #1026). virtual_nodes now sizes the consistent-hash ring (rebuilding the cache on change), anthropic_cache_control_injection gates automatic Anthropic cache-control injection as a fleet-wide master enable, both are hot-reloadable, and /admin/prefix-routing/stats reports the active pool values.
• Inject the shared API key store into Admin authentication so admin.auth.method: api_key works (#1036, closes #1030). The Admin routes were mounted with AdminAuthState::without_api_key_store(...), so every api_key request hit the empty-store branch and returned 500 Internal Server Error. Admin API-key auth now validates against the same store as ordinary /v1 auth and Admin key CRUD (both Authorization: Bearer and X-API-Key are accepted), enforces required_scope, and honors optional allowed_ips; disabling, rotating, expiring, or deleting a key stops it authenticating immediately with no restart.
• Make POST /admin/config/apply truthful (#1047, closes #1031). The endpoint was a placeholder: it recorded the unchanged current config as a new history version, never published through config_sender, and could report hot_reload_triggered: true without triggering anything. It now takes an optional full config candidate; with no candidate it is an explicit no-op (no history version, hot_reload_triggered: false), and with a candidate it validates, diffs against the running config, and publishes atomically under the modification lock only when it differs. hot_reload_triggered is true only after a successful publish of a changed config, updated_sections/requires_restart reflect the real diff, hot_reload: false previews the diff without applying, and repeated no-candidate calls never create empty history versions.
• Add a metrics::security stub so src/errors.rs builds under --features embed (#1048). The always-compiled error path calls crate::metrics::security::truncate_at_char_boundary, which lives behind the metrics feature; the embed stub module gained the submodule with the real UTF-8-boundary-safe truncation logic. The embed feature is not in the CI matrix, so the break (introduced by #1017) was caught only by running the off-feature build matrix locally.
• Scope the capacity integer-only heartbeat assertion to the capacity fields (#1040). The release feature-set CI job failed deterministically since 1.14.0 because the assertion checked the whole serialized heartbeat for .0 and tripped on the legitimate dot-zero in router_version "1.14.0"; it now verifies only that the three observed_*_capacity values serialize as JSON integers.
• Consolidate post-merge corrections for the configuration and routing changes above (#1050). Made circuit-breaker admission, half-open accounting, and hot reload coherent; snapshotted the retry policy and hardened hash-ring generations; exposed the complete Admin config catalog with truthful reload metadata; and covered every streaming transport while synchronizing configuration documentation.

Removed¶

• Remove the inert legacy routing config section (#1043, closes #1027). The schema accepted it and the docs described it as advanced routing overrides, but no production code read it, so setting routing validated while none of its values affected traffic. Live selection uses top-level selection_strategy, per-backend weight/models, and smart_routing. A config that still carries a top-level routing: key keeps loading (the unknown key is ignored) and the loader and config validate now emit a migration warning pointing at the live settings instead of silently doing nothing. Schema count returns to 34 and schema_version moves to 1.26.0.

v1.14.0 - 2026-07-19¶

Added¶

• Overhaul the embedded WebUI into a full admin console (epic #931). The single-file SPA is vendored (Alpine.js, uPlot, a prebuilt Tailwind stylesheet), drops every CDN dependency so it runs air-gapped under a strict default-src 'self' CSP, and is restructured into a per-page module registry (#948). A login flow with credential validation and session UX gates the console (#949), and a GET /admin/capabilities endpoint drives page-level feature gating so panels for disabled subsystems degrade instead of breaking (#950). New management pages cover guardrails (#951), cache operations (#952), smart routing (#953), usage analytics (#954), prompts (#955), files (#956), models and routing with catalog, aliases, and fallback chains (#957), backend deep-edit forms with an on-demand health probe (#958), schema-driven config forms with diff preview and hot-reload indicators (#960), and integrations status for control-plane, ACP, and AppProxy (#961). Config PATCH saves that would silently drop diff-shown removals are blocked (#962), API keys gain allowlists, annotations, and usage drill-down (#963), the dashboard adds trends, uptime, and circuit detail (#964), and a UX pass adds a command palette, keyboard shortcuts, responsive layout, and accessibility fixes (#965, #966, #1004).
• Add a typed operator request-parameter policy across every LLM API (epic #990). A protocol-independent policy engine (#996) applies operator-defined defaults, overrides, and min/max clamps for the seven canonical parameters (temperature, top_p, max_tokens, presence_penalty, frequency_penalty, top_k, min_p) with typed values and validated ranges, exposed through config (#1006) and enforced on the Chat Completions and Completions proxy path (#1007), the native Anthropic Messages path (#1008), and the Responses API (#1009). Mutations emit deterministic, value-free metadata, and an invalid policy is distinguished from an invalid request field. Runtime paths and the local-bound preservation guarantee were hardened after audit so a Hub tier limit can only tighten, never loosen, a locally configured bound (#1013, #1014, #1015).
• Enforce Continuum Hub tier request-parameter limits on the Router data plane (#1002, Hub #299 / PR #314 and #301 / PR #319). The additive version-0 wire contract carries integer-microunit bounds for seven canonical parameters, exact alias-resolved model scopes, a canonical public tier digest, and typed capability/active/rejection evidence. A full envelope is validated and converted before one atomic last-known-good swap; invalid, stale, or digest-mismatched candidates never partially apply. Each authenticated Hub key freezes the tightest intersection of Hub and deployment-local limits once before cache identity, substitution, arbitrage, backend selection, retry, or fallback across Completions, Chat Completions, Responses, and Anthropic Messages. Authoritative clears, mixed-version Hubs, feature-off/default builds, and Hub outages preserve their documented behavior, and logs, metrics, and heartbeat status remain value-free. The vendored protocol is pinned to Hub PR #319 merge 59387265e755bf02a9dbbb238da369e36d566d49.
• Apply Continuum Hub fleet request-parameter configuration through the outbound-only control-plane agent (#1001, Hub #302 / PR #317). The separate control_plane.config_sync.enabled opt-in advertises a masked, bounded structured request_params capability, fetches revision-bound public scalar content, verifies the canonical Hub-compatible digest and full effective config, layers Hub leaves below explicit local pins, and publishes one atomic hot-reload snapshot. Dry runs, rollback revisions, duplicate delivery, lost acknowledgments, reconnects, and restarts are idempotent through a private last-known-good state/outbox; transient Hub failures never become terminal rejections, in-flight requests keep their captured config, and snapshots, logs, and apply results remain value-free. The vendored protocol stays additive at version 0 and is pinned to Hub PR #317 head 771591f.
• Estimate per-backend observed RPM, input TPM, and output TPM for Continuum Hub inventory (#972). The control-plane agent promotes a rolling 60-second throughput observation only after a confirmed transient upstream 429, requires a full window with at least two successful completions, requires complete token accounting before emitting either token dimension, and retains a 15-minute expiring high-water mark. Cold-start, idle, lightly loaded, token-incomplete, quota-exhausted, and stale backends remain absent rather than reporting 0; observed capacity never reads or merges the operator-declared capacity stored by the hub. The same optional estimates are visible through /admin/stats/backends in control-plane builds.
• Add router-managed Gemini context caching (#929, closes #928). When gemini_context_cache.enabled is set, the router caches large, stable system-prompt prefixes sent to type: gemini backends as Google cachedContents resources and reuses them across requests, transparently to OpenAI-compatible clients, guaranteeing the cached-token discount and cutting prefill latency for workloads that resend a large fixed system prompt. It mirrors the existing Anthropic cache_control injection on both the compat and native Gemini surfaces, keyed by (backend, model, prefix digest) with single-flight creation, negative caching of API-rejected prefixes, LRU eviction, and TTL extension on hit. Default off: explicit caching bills storage per token-hour, and with the feature disabled it is a strict no-op with no map allocation, background task, or added per-request latency. A continuum_gemini_context_cache_* metric family reports requests, creates, failures, entries, evictions, and cached tokens.
• Execute cost-bounded, Hub-synced synthetic provider/model probes (#969, #1019). When the control-plane feature is compiled and control_plane.enabled, control_plane.policy.enabled, and an enabled Hub ProbePolicy with targets all hold, the agent runs one non-streaming probe call per listed (provider, model) target on the policy cadence through the internal Backend::execute_probe_chat_completion seam (normal transformation and effective auth, structured outcomes, no substitution, fallback, cache, or retries, and no UsageEvent), bounded by a per-router monthly budget ceiling persisted in an owner-only sidecar next to control_plane.state_file (0 is unlimited; positive ceilings reserve before dispatch and fail closed without price or state). Results post to POST /api/agent/v1/probes with stable report_id/probe_id redelivery and a non-fatal rate-limited 404 path for older hubs. No config field is added, and every gate off means zero probe behavior, cost, or wire traffic.
• Report per-backend served models (#920), per-backend auth and quota rejection counters (#925), and token-supply and saturation telemetry (#968) to Continuum Hub inventory, and add an opt-in policy.unmatched_key reject strict mode that fails closed on an unrecognized key instead of falling open (#923). All behind the control-plane feature with PROTOCOL_VERSION unchanged.
• Add a GET /admin/capabilities endpoint that reports compile-time Cargo feature flags and runtime-enabled subsystems in one read-only response, gated by the existing admin auth and audit middleware (#950).

Changed¶

• Bump 12 minor and patch dependencies in one group (#976), including regex 1.12.4 to 1.13.0, aws-config 1.8.18 to 1.9.0, http-body 1.0.1 to 1.1.0, rust-embed 8.11.0 to 8.12.0, plus uuid, bytes, lru, and socket2.

Fixed¶

• Wire live Prometheus metric sources and reconcile the legacy collectors for the router observability epic (#973; #982, #986, #993). Metric families that were registered but never written now have production writers, legacy lazy_static counters are synced from the modern metric sources, and the English and Korean metrics references document the exported metric and label names.
• Report observed per-backend capacity through the control-plane inventory (#972, #983), reading the single in-flight gauge rather than a parallel counter.
• Hide internal backends across every user-facing API surface so a backend:-referenced internal model never leaks into model listings or the routing candidate set (#985).
• Harden the epic #990 response cache identity so cache keys cannot collide across distinct effective requests (#1018).
• Surface upstream error bodies on the Unix-socket streaming path (#1017, closes #1016). Non-2xx backend responses on the plain, thinking, and Anthropic Unix-socket streams now drain the error body under an 8 KiB cap and a two-second budget bounded by the remaining request timeout, reaching error-body parity with the TCP streaming path instead of discarding the backend's explanation.
• Account for Unix-socket usage in streaming responses so control-plane usage records are not dropped on that transport (#927).
• Harden control-plane observability and strict auth (#926), reconcile a renamed backend name during health sync (#921), stamp an unattributed sentinel on fail-open usage attribution (#922), and resolve the usage provider from the live config per event rather than a stale snapshot (#919).
• Harden epic #973 observability state after the merged implementation audit. Capacity samples are now kept in timestamp order so concurrent, out-of-order completions cannot retain stale throughput or surface future observations; backend removal and same-name endpoint replacement unconditionally reconcile name-keyed capacity and credential-health state even when health checks are disabled; and the registered aggregate model-usage, token, and final-timeout counters now have production writers. The English and Korean metrics references now document the exported metric and label names.
• Dispatch live HTTP traffic through the configured selection_strategy (#975). No HTTP request path ever called the pool selector: chat completions, streaming chat, the Responses API (non-streaming, streaming, and compact), Anthropic Messages and count_tokens, and image generation each picked the FIRST healthy candidate in model-lookup order, which left RoundRobin, WeightedRoundRobin, Random, LeastLatency, ConsistentHash, and PrefixAwareHash all inert for live traffic. All of these paths now share one selection seam: each keeps its own pre-filters and error semantics (model lookup, internal-backend filter, per-key allow-list, retry-state exclusion, and the model-not-found / 403 / all-unhealthy distinctions are unchanged), the seam health-filters the surviving candidates so an unhealthy backend can never be selected, and the final pick goes through the new candidate-restricted pool entry point BackendPool::select_from_candidates, which shares the strategy core with select_backend_with_context and keys the consistent-hash ring cache by the participating backend names so per-model candidate subsets each hash against a correct ring. Prefix-aware routing is live end to end: with prefix_routing.enabled, the prefix key is extracted from the request body (OpenAI messages[] and Anthropic top-level system shapes) and drives PrefixAwareHash CHWBL placement over the live in-flight gauge from #971, and the KvOverlapScorer is registered at startup when a KV cache index is also configured. Traffic-distribution behavior change: deployments that configured a non-default selection_strategy now actually get it enforced, and default deployments change from "always the first candidate backend" to real round-robin across the healthy candidates for each model.
• Wire per-backend in-flight tracking into the production request path so the CHWBL load cap is no longer degenerate (#971). BackendStats.in_flight_requests previously had no production writer, so the bounded-load half of the PrefixAwareHash selector computed its cap over a permanent zero and could never bind. The backend pool now owns an always-compiled RAII in-flight tracker written at every dispatch seam (chat proxy, streaming chat, Responses API, Anthropic Messages and count_tokens, image generation, image edit), with streaming guards riding the response body until the last frame or a client disconnect. The cap formula now counts the request being placed, ceil((total_in_flight + 1) * (1 + epsilon) / backend_count), and prefix_routing.load_factor_epsilon (default 0.25) finally reaches the pool, so the documented knob takes effect. The gauge is the single source of truth: the admin prefix-routing/stats endpoint and the control-plane inventory's per-backend active_requests now read it, replacing the parallel control-plane gauge from #968, and both report real load instead of zeros.
• Note on scope: this fixes the gauge the CHWBL selector reads, not which code paths invoke that selector. The router's HTTP request paths still dispatch to the first healthy backend serving the model, so selection_strategy (CHWBL included) does not yet influence live traffic distribution and no request will be routed differently after this change. Wiring the pool selector into the request funnel landed in #975 (see the entry above).
• Prevent the Unix socket integration tests from hanging and restore them in CI (#984, #992, #995).

v1.13.1 - 2026-07-10¶

Added¶

• Amazon Bedrock multi-provider support via the Converse API (#615). A new endpoint_type: converse on type: bedrock backends targets POST /model/{modelId}/converse[-stream] and reaches any chat model on Bedrock (Claude, Nova, Llama, Mistral, Cohere, AI21 Jamba, DeepSeek) through one unified wire format: OpenAI-shape requests are transformed into Converse content blocks (system-prompt extraction, toolConfig/toolUse/toolResult, inferenceConfig sampling parameters), signed with SigV4 through the shared #614 plumbing, and the binary ConverseStream event stream is decoded back into OpenAI SSE with tool-call and reasoning deltas. A hard-coded per-model capability matrix (chat, streaming, tools, vision, documents, guardrails, prompt caching, reasoning) rejects unsupported-capability requests with a 4xx naming the capability before signing, vendor-specific options pass through verbatim via extra_body / additionalModelRequestFields, remote image URLs are fetched (SSRF-validated, size-capped) and inlined as base64, and Converse-native bodies are detected and forwarded without double transformation. Gated behind the existing bedrock-sigv4 Cargo feature.
• Backend-level internal visibility flag (#908, #911). Setting internal: true on a backend keeps its infrastructure-only models (a guardrail classifier, an internal summarizer, an embeddings model) out of user-facing surfaces: internal backend names are stripped from the OpenAI /v1/models, /v1/models/extended, single-model, and Anthropic /anthropic/v1/models listings, and removed from the routing candidate set before the per-key allow-list so a user request for an internal-only model returns model-not-found (404) rather than a misleading Forbidden. Router-internal backend: references and admin surfaces still resolve internal backends. The field defaults to false and is omitted on serialize, so existing configs are unchanged.
• Accept the retiring hub key hash during a control-plane key-rotation overlap (#906). KeyEntry gains additive previous_key_hash and previous_retires_at_ms fields so a router admits both the current and the previous hash during the overlap window, retiring the previous hash at the local timestamp or when the hub omits it on the next sync, whichever comes first; a current hash never shadows the previous one and revocation still rejects both. Behind the control-plane feature; older envelopes parse unchanged and PROTOCOL_VERSION stays 0.
• Stamp streaming time-to-first-token on control-plane usage records and keep heartbeat inventory fresh under load (#912). UsageRecord gains an additive ttft_ms, measured from the same request-start instant used for latency_ms up to the first streamed byte across the chat, thinking, Anthropic, and Gemini SSE loops, clamped to latency_ms, and omitted rather than fabricated for non-streaming, local-cache-replay, and batch responses. The heartbeat loop subtracts each beat's build and send time from the interval so a slow beat under load does not drift the cadence past the hub's load-sample interval, and every beat still carries a full router inventory. Behind the control-plane feature; PROTOCOL_VERSION stays 0.

Changed¶

• Official release binaries now ship with the control-plane and appproxy-router features compiled in (#905). The Release workflow builds all six targets with --features control-plane,appproxy-router, and the Debian packages and Docker images repackage those binaries, so every distribution channel carries them. Both features stay out of the crate default/full set, so a plain cargo build from source is byte-for-byte unaffected; for packaged binaries the opt-in moves to runtime configuration (control_plane.enabled, default false, and the appproxy_router section, absent by default), so existing deployments see no behavior change. The legacy appproxy/appproxy-legacy worker remains a source-build opt-in.

Fixed¶

• Resolve clippy lint drift that red-failed the default-feature cargo clippy -- -D warnings CI step on every branch (#907). Two mechanical, clippy-suggested rewrites in the Anthropic Gemini transform and the Bedrock reasoning transform, with no behavior change.

v1.13.0 - 2026-07-09¶

Added¶

• Continuum Hub control-plane enforcement and optimization, promoting the opt-in agent from outbound telemetry (v1.12.0) to a full policy layer. Everything is gated behind the control-plane feature and fails open when no policy is loaded, so a default build has zero added dependencies, zero runtime cost, and is byte-for-byte unchanged.
• Enforce a hub-synced key table, tier rate limits (rpm plus cache-aware input/output TPM), monthly budgets, and model allowlists through a post-auth middleware that emits x-continuum-ratelimit-* headers, fed by an additive KeyUsageSnapshot protocol type and poll/WebSocket policy sync into an ArcSwap store (#877).
• Gate the exact response cache on tier cache_enabled plus the org optimization policy, enforce the x-continuum-cache and x-continuum-batch per-request override headers within tier bounds (off always wins, on never exceeds the tier), emit x-continuum-cache hit/miss/bypass outcome headers, and meter local cache hits as metadata-only events that charge no token budget (#878).
• Stamp a CacheHitType (exact/prefix/semantic, plus an open-ended unknown) on usage records alongside the already-metered cache_hit, cached_input_tokens, batched, and provider_batch_id fields; additive with a serde default and PROTOCOL_VERSION unchanged (#879).
• Dispatch batch-eligible traffic through OpenAI-style and Anthropic Message Batches endpoints on a separate batch rate pool (batch_rpm plus an in-flight batch_queue_depth gauge) that never consumes the synchronous rpm/tpm limits, report each provider job's lifecycle to the hub via at-least-once, 404-tolerant BatchStatusUpdate messages, and expose a full key-scoped /v1/batches surface (create, retrieve, list, cancel, results) behind a per-key ownership gate with an age-based reclaim backstop; inert unless control_plane.batch is enabled (#880, #895).
• Execute a prompt-prefix response cache on a keyspace disjoint from the exact-match cache for OpenAI chat and Anthropic Messages non-streaming, keeping the prompt-determining fields and dropping the sampling-bound tail, guarded to store only naturally-completed responses and serve only when the stored entry fits the new request's max_tokens (#895).
• Select the cheapest equivalent provider via price arbitrage (x-continuum-arbitrage, tier arbitrage_enabled, tri-state equivalence_classes) only after health, circuit-breaker, latency, tier, and per-request header gates pass, constraining dispatch to the selected backend so a later selection pass cannot override the policy decision (#896, closes #891).
• Compress request context router-side for managed requests (x-continuum-compression, TierLimits.compression_enabled, CompressionPolicy), deterministically shrinking chat history before responses-only routing, streaming, cache lookup/storage, and dispatch while preserving leading system/developer and recent turns, and reporting compressed/uncompressed_input_tokens without exposing content to the hub (#898, closes #892).
• Defer eligible /v1/batches submissions until the next matching UTC routing window (PolicyEnvelope.routing_windows tri-state, weekday masks, tier scoping, end-exclusive and wrap-around-across-midnight boundaries) with batch admission checked at dispatch time so no pool slot is held while a request sleeps, and add integer-only LoadSummary request/error-rate/p50/p99 counters to router inventory heartbeats plus advisory OptimizationPolicy.load thresholds (#900, closes #893).
• Substitute the requested model per hub policy on the control-plane path, keeping substitution and arbitrage metadata mutually exclusive (#894).
• Consume hub-delivered, org-scoped provider credentials (PolicyEnvelope.provider_credentials, hub M3 vault) in place of the locally configured api_key, applying rotations without a restart or dropping in-flight requests; secrets stay in memory only, are redacted from every Debug rendering, and are resolved value-only at every provider-dispatch send site (proxy, streaming, images, Responses, batch, native Anthropic, count_tokens, embeddings) so auth-header presence and OAuth/SigV4/Bedrock gating are unchanged (#901).
• Extend hub-policy enforcement and usage attribution to the native Anthropic Messages path (streaming and non-streaming, across the native, OpenAI-bridge, Responses-bridge, and Bedrock-runtime sub-paths) and streaming /v1/responses, accepting the Anthropic-native x-api-key credential and charging cache-aware token usage under the hub key_id instead of anonymous (#882, #889).

Fixed¶

• Harden tiered KV cache event routing: wire configured KV event sources into the index at startup, preserve existing affinity scores when an event only changes the storage tier, expose the full offload/reload/purge event set through metrics and admin status, constrain disaggregated fast-decode execution to the decode backend selected from the KV index, honor per-client backend allow-lists during orchestration, and return a service-unavailable error instead of a false successful prefill/decode header when the two-phase worker protocol is unavailable (#903).
• Harden the epic 864 guardrail hot-reload behavior across the config validator and the guardrail service (#864).
• Thread the model-substitution argument through the prefix cache-hit metering call sites so the tree compiles under --features control-plane after #894 and #895 merged cleanly but on divergent bases (#897).

Documentation¶

• Redesign the AppProxy worker-mode reference (English and Korean) around the new ROUTER frontend: a single binding address per worker, cluster-level model abstraction, two-level hierarchical routing, event+pull propagation of mappings and hash-only key custody, and a BEP-1053 cross-reference (#748).
• Backfill v1.11.0 and v1.12.0 into the docs/en and docs/ko changelog mirrors, which had lagged two releases.
• Correct the config-assistant sync note for the relaxed drift guard and note the CHANGELOG en/ko mirror requirement.

Dependencies¶

• Bump the minor-and-patch group with 2 updates (#883): rand 0.10.1 to 0.10.2 and arc-swap 1.9.1 to 1.9.2.

CI¶

• Relax the skill-doc version drift guard from an exact CARGO_PKG_VERSION match to a compatibility-window check, so a routine in-range version bump no longer needs a hand edit while a window-crossing major bump still fails the build for re-review.

v1.12.0 - 2026-07-05¶

Added¶

• Configuration assistant for authoring and validating config.yaml (epic #530).
• A config CLI subcommand with validate, generate, diff, and show --resolved verbs, plus the configuration-assistant skill and config templates (#829).
• An mcp-serve stdio MCP server mode (behind the mcp feature, included in default/full) that answers claude mcp add continuum-router-config -- continuum-router mcp-serve out of the box (#830).
• An IDE rule generator (scripts/generate-ide-rules.sh) and documentation for the assistant (#831), and a @lablup/continuum-router-mcp npm wrapper package (#832).
• Verification and CI drift guards that fail the build when a schema change skips the assistant doc/template update (#833), plus follow-up hardening (#834).
• Continuum Hub control-plane agent behind the opt-in control-plane feature (#875, closes #874). An outbound-only task that enrolls a router against a hub, heartbeats liveness plus a router inventory, and pushes batched, metadata-only usage records. Not in default/full, so a default build has zero added dependencies and zero runtime cost; the sole new dependency is the vendored serde-only continuum-protocol crate, compiled only under the feature.
• Backend.AI AppProxy ROUTER worker mode (epic). ROUTER wire types, events, and worker config (#815); a ROUTER event overlay with per-node applied-ack (#816); reconciliation of circuits and mappings into backends (#817); a coordinator client with router-config pull (#818); a model-level API-key gate via allowed_models (#819); and the ROUTER worker lifecycle, config wiring, and feature flags (#820).
• Claude Sonnet 5 support (#857).
• Native streaming Gemini Responses conversion via streamGenerateContent (#856).
• Two guardrail providers: reference a served backend as the guardrail model with a backend: prefix (#871), and a custom_classifier provider that returns a prompt-defined JSON verdict (#870).
• Configurable web-search provider base URL for internal mirrors (#836).
• A cli-less library build behind the embed feature for in-process iOS embedding (#850), exposing continuum_router::serve_embedded(config, addr).

Changed¶

• Consolidate the two backend connection pools into a single trait-based pool (#851, refs #812) and rename the legacy Backend struct to PooledBackend (#843, refs #511).
• Unify the dual streaming paths: extract SSE pipeline concerns into composable stream middleware (#839), route Gemini streaming through the Backend trait (#840), collapse the duplicated Anthropic streaming transform (#841), and apply the OpenAI reasoning normalization on the streaming path (#842).
• Extract a shared LLM-backend transport core for guardrails (#869), and add a native-Anthropic system gate with an alloc-free family lookup (#861).
• Relocate the current AppProxy worker to legacy (appproxy v3) and extract a shared common core (#813).

Fixed¶

• Preserve usage attribution metadata on the control-plane usage push (#876).
• Validate /v1/responses streaming requests before file_id resolution so a malformed request is rejected with the right error instead of failing during resolution (#873).
• Honor the per-key allow-list on the /v1/responses streaming paths (#863).
• Normalize Bedrock model prefixes for Anthropic capability gating (#859).
• Reject a Brave web_search base_url that carries a query string (#852).
• Harden the native Gemini Responses audio and image conversion (#853, follow-up to #780) and wire the native Gemini Responses conversion for input_audio (#835).
• Harden the epic 505 streaming prompt parity (#846) and the epic 530 configuration assistant (#834).
• Harden AppProxy ROUTER request gating (#823).
• Anchor the daily-series prune and read tests to the wall clock so they no longer flake (#814).

Documentation¶

• Document the SSE streaming pipeline overhead with a reproducible benchmark (#848) and audit the divergences between the two streaming paths (#837).
• Add backend transform consistency integration tests (#838).

Dependencies¶

• Bump the minor-and-patch group across 1 directory with 3 updates (#845) and with 2 updates (#802).
• Bump actions/cache from 5 to 6 (#844) and actions/checkout from 6 to 7 (#801).

v1.11.0 - 2026-06-21¶

Added¶

• Content-safety guardrails subsystem (epic #644), config-gated through an Option<GuardrailsConfig> that defaults to off, so an absent guardrails key leaves the feature inert and existing configs are unaffected.
• Foundation: the object-safe, async Guardrail trait with input/output/streaming check stages, a GuardrailVerdict (Allow/Block/Transform/Flag) with most-severe-wins severity ordering, a borrowing GuardrailContext, and a GuardrailsConfig schema (global monitor/enforce mode, providers with credentials by env, per-route overrides, per-API-key bypass, per-guardrail timeout_ms and on_error fail-open/fail-closed, block behavior, streaming mode, and exact and regex allow/deny lists) with a validator that rejects invalid combinations such as enforce with no providers, thresholds outside [0,1], non-compiling regex, and zero timeouts (#779, closes #644). The validator is wired into the real config load path so an invalid guardrails block is rejected at load time (#781).
• Service: a GuardrailService pipeline that runs the configured guardrails with per-guardrail timeouts and fail-open/fail-closed handling, hot-reload, and a block-response builder (#782).
• Providers behind a factory: OpenAI Moderation (#783), a self-hosted classifier (#785), PII detection and redaction (#786), and AWS Bedrock and Azure cloud guardrails (#787).
• Gating: input gating wired into the chat, Anthropic, and responses dispatch paths (#788), output gating on the non-streaming response paths (#789), streaming output gating with buffer/chunked/passthru modes (#790), and per-route policy threaded through the dispatch handlers so a guardrails.routes[<model>] override (mode, enable/disable, provider subset, thresholds) takes effect end to end, keyed by the request model id (#796).
• Controls and observability: admin runtime controls for guardrail policy (#784), Prometheus metrics and per-decision audit logging (#792), an end-to-end integration suite for input/output gating (#791), and a guardrails guide with architecture and metrics coverage plus site nav (#793).
• Per-model and daily usage-series dimensions on the admin stats API (#799, closes #798). GET /admin/stats/api-keys/{id}/models and GET /admin/stats/users/{user_id}/models return a per-model token and request breakdown, and .../{id}/series and .../{user_id}/series return an ascending daily time series of total_requests, prompt_tokens, completion_tokens, and total_tokens keyed by the UTC completion date (from/to accept Unix-millis or RFC 3339; interval defaults to day and any other value returns 400). Four cardinality-bounded DashMaps on the StatsCollector (keyed by composite keys joined with a U+001F separator that cannot occur in ids, models, or dates) back the dimensions, a series_retention_days option (default 30, serde-defaulted) prunes daily buckets older than the cutoff, and the four maps are added to the persisted snapshot as #[serde(default)] so snapshots written before this change still load.

Fixed¶

• Harden the daily-series pruning lifecycle (#800, refs #799). Restore now counts every separator-bearing composite key, including the anonymous daily and model buckets that the runtime insert path treats as non-reserved, so pruning an expired bucket after a restart no longer decrements a slot that was never reserved, underflows the AtomicUsize, and silently folds new real buckets into the overflow row. A dedicated hourly series pruner is spawned when stats persistence is off (where the snapshot task that drove pruning never runs), so the default config physically drops expired daily buckets instead of accumulating them up to the cardinality ceiling.
• Record admin stats for /v1/responses traffic (#794), correcting the stats success and failure boundary for conversion failures and strengthening regression coverage for the registered per-key and per-user admin stats endpoints.
• Handle OpenAI input_audio content blocks in the native Gemini and Anthropic transforms (#778). The Gemini Responses converter now maps audio to an inline_data part with a mime_type derived from the format (wav to audio/wav, mp3 and other to audio/mp3), and routes a data: URL through the existing inline-data path, instead of discarding it. The Anthropic Messages API has no audio content type, so the native transform now returns a ValidationError surfaced as HTTP 400 at both the non-streaming and streaming entry points instead of silently dropping the block or forwarding it verbatim.

Dependencies¶

• Bump the minor-and-patch group with 5 updates (#769): uuid 1.23.2 to 1.23.3, regex 1.12.3 to 1.12.4, redis 1.2.2 to 1.2.3, aws-smithy-eventstream 0.60.20 to 0.60.21, and aws-smithy-types 1.4.9 to 1.5.0.

v1.10.2 - 2026-06-15¶

Added¶

• Kimi K2.7-Code and GLM-5.2 model metadata (#775). Kimi K2.7-Code is a 1T-parameter MoE (32B active) with a MoonViT vision encoder and 256K context that runs only in thinking mode and returns reasoning in the native reasoning_content field, so it carries no <think> marker config. GLM-5.2 is the GLM-5 line coding flagship with a 1M context window, 131K max output, and two thinking-effort levels (High and Max); it carries the standard <think>/</think> marker config like the rest of the GLM family. GLM-5.2 standalone API pricing was not published at launch, so its input/output rates are estimated from the GLM-5/GLM-5.1 tier.

Fixed¶

• Normalize the vLLM/OpenAI-compatible reasoning field to the canonical reasoning_content on /v1/chat/completions (#776, closes #774). Newer vLLM renamed its reasoning output field from reasoning_content to reasoning (streaming delta.reasoning, non-streaming message.reasoning), and the router relayed it unchanged, so clients reading reasoning_content silently dropped all reasoning text from self-hosted vLLM reasoning models. The rename runs across every OpenAI-compatible relay path (the streaming default and thinking transformers, the unix-socket relay, the mid-stream fallback relay, and the non-streaming proxy body), only when reasoning_content is absent so an upstream already using the canonical name is never overwritten, and is scoped away from the Gemini and Anthropic handlers and the Responses API.

Documentation¶

• Note in the reasoning-effort architecture reference (English and Korean) that the router normalizes the upstream vLLM reasoning field to reasoning_content.

v1.10.1 - 2026-06-15¶

Added¶

• Per-API-key and per-user usage statistics REST API: four admin endpoints GET /admin/stats/api-keys, GET /admin/stats/api-keys/{id}, GET /admin/stats/users, and GET /admin/stats/users/{user_id}, mirroring the existing /admin/stats/models shape under the admin auth router (#772, closes #770). The StatsCollector gains per_api_key and per_user dimensions recorded from the same spawned task that updates the Prometheus llm_tokens_total counter, so the in-memory dimensions no longer depend on the metrics feature and are attributed even for failed and zero-token requests. The api_key_id is the derived, non-reversible id resolved once off the request hot path; unauthenticated requests bucket under "anonymous", and ids beyond the 1000-per-dimension cardinality cap fold into an "unknown" overflow bucket so usage is still counted in aggregate. Each GET /admin/stats/.../{id} returns 404 when the id has no recorded usage, and the optional window query param is echoed back but does not filter, since the aggregates are all-time atomic counters like /admin/stats/models. The snapshot/persist format adds both dimensions as #[serde(default)], so snapshots written before this change still load without a format-version bump.

Changed¶

• Ignore root-level .sh scripts via .gitignore so local helper scripts are not accidentally committed.

Documentation¶

• Document the admin API key management endpoints and the per-API-key/per-user usage statistics in the Admin REST API reference for both English and Korean (#773, closes #770). A new "API Key Management APIs" section covers the eight /admin/api-keys endpoints (create, list, get, update, delete, rotate, enable, disable) with request/response schemas reflecting the sk-***abcd masking and the full-value-once-on-create behavior, the ApiKeyConfig fields, the permissive vs blocking api_keys.mode semantics, and runtime-key persistence via persistence_file with hot-reload. The Statistics section gains the four new stats endpoints, the window echo, the anonymous and unknown buckets, the per-dimension cardinality cap, and the api_key_id-to-issued-key linkage.

v1.10.0 - 2026-06-12¶

Added¶

• Dynamic model enumeration for Codex (ChatGPT OAuth) backends (#753, closes #752). The Codex backend has no standard /v1/models; the router instead queries the plan-gated GET <base>/models?client_version=<ver> endpoint with the loaded OAuth token during model discovery, keeps only user-facing entries (visibility: list and an available_in_plans that is empty or matches the account's chatgpt_plan_type claim), and uses the result to populate /v1/models and routing. On any failure (network error, non-2xx, empty list) the configured models: list remains the fallback, and the behavior applies only to Codex OAuth backends.

Changed¶

• Extract a shared parse_responses_body in src/services/responses/sse.rs and a shared build_openai_chat_request_core in src/http/streaming/handler.rs, eliminating the duplicated /responses SSE-aggregation parse path and the duplicated streaming request-construction logic that existed across the Codex, passthrough, and Anthropic handler variants (#766, closes #762). No behavior change; the consolidation removes drift risk where a fix to one copy would not propagate to the others.

Fixed¶

• Route Codex (ChatGPT OAuth) chat completions to the backend's /responses endpoint (#755, closes #754). build_responses_url appended /v1/responses to the .../backend-api/codex root, and the upstream edge rejects that path with a 403 HTML page that the router surfaced as an opaque authentication error despite a valid OAuth token; the Codex root now routes through the shared URL composer that strips the internal /v1. With the path fixed, Codex rejects a bare-string input with HTTP 400 "Input must be a list", so sanitize_codex_responses_request coerces a single-message string input into a one-element item list.
• Accept an RFC3339 expires_at in the OAuth token store (#756, closes #751). The field was a plain u64, so a token store that wrote the field as a datetime string failed to deserialize, the backend silently lost its OAuth strategy, and requests went out unauthenticated, surfacing as an opaque 403 that hid the real cause. expires_at now deserializes leniently (integer or float epoch seconds, a numeric string, or an RFC3339 datetime with offset, all normalized to epoch seconds) and rejects anything else with an error naming the accepted formats; the canonical on-disk shape remains a JSON number, so existing stores and the save/load roundtrip are unchanged.
• Strip local-engine-only fields (chat_template_kwargs, thinking_budget_tokens, enable_thinking, preserve_thinking, top_k, min_p, repeat_penalty) from /v1/chat/completions requests bound for cloud OpenAI, which rejects unknown top-level keys with HTTP 400 "Unknown parameter" (#760, closes #758). The strip mirrors the existing cloud Gemini behavior through a shared field_filter module gated on api.openai.com, runs at every cloud-OpenAI chat send site (non-streaming, streaming, and per-hop in the fallback loops), never touches extra_body or reasoning_effort, and is a no-op for local OpenAI-compatible engines, preserving the backend-passthrough contract.
• Respect the configured models: selection for Codex dynamic enumeration and stamp the clean owned_by on enumerated models (#765, closes #763). When live enumeration succeeded, the operator-configured models: allowlist was bypassed, so /v1/models exposed every enumerated Codex model; the post-filter now applies uniformly, with a non-empty list intersecting the enumerated set down to the selected subset and an empty list exposing the full set. Enumerated Codex entries also carried the raw backend name as owned_by; the owner is now resolved from the backend type (openai), matching every other backend.
• Return a valid non-streaming response for Codex (ChatGPT OAuth) backends on /v1/chat/completions with stream:false (#764, closes #761). Codex's /responses endpoint only accepts stream:true + store:false; sending stream:false produced HTTP 400 "Stream must be set to true". sanitize_codex_responses_request now forces stream:true and store:false unconditionally. A second fix makes SSE detection in PassthroughService::execute_request robust to Codex responses that carry an SSE body without Content-Type: text/event-stream: a looks_like_sse sniffer checks the first non-empty line for data:/event:/: prefixes, and a failed JSON parse retries SSE aggregation once before surfacing the original parse error.
• Reconstruct the assistant message from response.output_text.delta events for Codex store:false responses (#768, closes #767). When Codex runs with store:false the terminal response.completed event carries an empty output array and the assistant text arrives only as incremental response.output_text.delta events; the SSE aggregation now accumulates those deltas and synthesizes a message item when the completed event carries no assistant text, so non-streaming clients (including the title/summary utility path) receive the text instead of an empty response.

Documentation¶

• Document the post-v1.9.1 Codex and transport changes across the English and Korean manuals: the Codex Chat-to-Responses request handling and lenient expires_at parsing in backends.md, the cloud-OpenAI field strip in backend-passthrough.md, and item_reference resolution on the Responses API in api.md, and convert the remaining Korean pages to the polite -습니다 register so every page reads in one voice.

v1.9.1 - 2026-06-11¶

Added¶

• GET /version endpoint that returns { "version": "<CARGO_PKG_VERSION>" }, registered unconditionally on the base router (not behind the appproxy or any other Cargo feature) so it is present in the standard release binary, and a version field on the existing GET /health response (#750, closes #749). Both endpoints stay outside the API-auth boundary, matching /health, so a downstream consumer can probe the running router version for feature-gating instead of failing open against a 404.

v1.9.0 - 2026-06-10¶

Added¶

• AppProxy worker mode behind an opt-in appproxy Cargo feature, letting Continuum Router run as a Backend.AI AppProxy inference worker driven by an AppProxy coordinator (epic #709). The feature is deliberately left out of full, so default builds are unaffected.
• Foundation: the typed AppProxyWorkerConfig section (coordinator URL, shared api_secret/jwt_secret, redis_url, wildcard frontend parameters, heartbeat/reconcile durations, and an events toggle, with both bearer secrets redacted in Debug and ${ENV_VAR} references resolved through the same path as backends[].api_key), the SerializableCircuit/RouteInfo wire types and ProxyProtocol/AppMode/FrontendMode enums (snake_case and kebab-case tolerant, unknown fields ignored), and the module scaffold (#716).
• Coordinator REST client CoordinatorClient with register, heartbeat, deregister, list_circuits, and get_circuit, each carrying X-BackendAI-Token and a fresh per-call X-BackendAI-RequestID, and an error type that separates retryable (connection, timeout) from fatal (HTTP 4xx) failures (#717).
• Circuit-to-backend translation and reconcile: circuit_to_backends builds one BackendConfig per replica (named appproxy-<circuit_id>-r<route_key>, traffic-ratio mapped to a 1..=1000 weight that never drops a route to 0, vLLM detection from runtime_variant), and apply_circuits injects the translated backends through the existing hot-reload config_sender, namespaced by the appproxy- prefix so statically configured and admin-API backends are preserved (#718).
• Worker lifecycle service and a /status endpoint: run_worker registers with backoff, performs an initial circuit pull, then runs a heartbeat loop (kept under the coordinator's 30s LOST timeout) and a pull-reconcile loop (the always-on backstop for missed events), auto-discovering each circuit's model from a replica's GET /v1/models and deregistering on shutdown. The shared in-memory AppProxyRegistry is indexed by subdomain and circuit id (#719).
• Host/subdomain ingress resolver that turns a manager-issued endpoint subdomain into a concrete circuit and pins the request model so the existing selection path serves that circuit's replicas, with HS256 circuit-bearer verification that checks the decoded id against the circuit id and rejects an alg:none downgrade, an optional aggregation_hosts field for cross-circuit model aggregation, and a pure fall-through when no wildcard_domain is configured (#720).
• Redis Pub/Sub circuit-event overlay that gives legacy-mode coordinators sub-second circuit updates: a subscriber loop with exponential-backoff reconnect, a base64 + msgpack envelope codec, handlers for circuit_created / circuit_removed / circuit_route_updated, and the ack envelope that prevents the coordinator's E10001 "Proxy worker not responding" error (#721).
• Claude Fable 5 (claude-fable-5) and Mythos 5 (claude-mythos-5) model support (#747). Both are 1M-context, 128K-max-output models priced at \(10/\)50 per MTok; Mythos 5 is the same underlying model with safety classifiers lifted, shipped only through the limited Project Glasswing release. A new is_mythos_class helper routes both ids through the Anthropic capability gates: adaptive thinking is required (legacy budget_tokens is rejected with HTTP 400 and normalized to adaptive), temperature/top_p/top_k are dropped, the max effort level is supported (xhigh maps to max), and mid-conversation system messages are preserved. Both reject an explicit thinking.type == "disabled", so explicit_thinking_for_model now returns an Option and the router omits the thinking parameter entirely instead of forwarding a value that would 400. opus_supports_max_effort is renamed to supports_max_effort because the max effort level is no longer Opus-only. The same handling applies to the OpenAI Responses API conversion path.
• Gemma 4 QAT model metadata for the five quantization-aware-training checkpoints (E2B, E4B, 12B Unified, 26B-A4B MoE, 31B dense) with load-bearing -it-qat aliases and resolution/drift-guard tests (#723, closes #722).
• Gemma 4 12B Unified model metadata (#705).

Changed¶

• Log request-body extractor rejections at warn, so a malformed or oversized body that Axum rejects before the handler runs is visible in the logs instead of failing silently (#707).

Fixed¶

• Stop the retry loop from hammering the same upstream on HTTP 429 by distinguishing transient rate limits from non-transient quota/credit exhaustion and by honoring the upstream Retry-After hint (#742, closes #740). Previously every 429 was retried up to max_attempts (default 3) with a fixed exponential backoff that ignored the provider's Retry-After, so for a model served by a single upstream the router re-hit the same exhausted endpoint, amplifying load and adding latency before an inevitable failure. The RouterError::RateLimited variant now carries a retryable flag: non-transient 429s (OpenAI insufficient_quota / billing_hard_limit_reached, and clear credit-depletion language such as "prepayment credits are depleted") are classified as non-retryable so the router fails fast after a single call and passes the provider status and body through, while transient signals (bare RESOURCE_EXHAUSTED/RPM throttling, rate_limit_exceeded, rate_limit_error) stay retryable. The classifier is deliberately narrow: the over-broad "billing" and "exceeded your current quota" markers that Google reuses verbatim for transient throttling were removed so a recoverable Google 429 is no longer flipped to fail-fast. A retried 429 uses the upstream Retry-After for its backoff (capped to max_delay) and fails fast without sleeping when the requested interval would exceed the remaining total-timeout budget. The hint is preserved end to end (parsed from Google's RetryInfo.retryDelay and the integer-seconds Retry-After header, then reflected in the client-facing Retry-After header). The budget probe now uses saturating_add and the parsed hint is clamped to 24 hours (MAX_RETRY_AFTER_SECS), closing a remotely triggerable panic where a hostile upstream's near-u64::MAX Retry-After overflowed the Duration addition and aborted the request task.
• Persist the accumulated /v1/responses response on the streaming conversion paths (Anthropic, Chat-Completions/Gemini fallback) so a follow-up request that references a streamed output item via {"type":"item_reference","id":"item_..."} (the default behavior of the OpenAI and Vercel AI SDKs) resolves instead of returning HTTP 400, even when step 1 used store:true (#746, closes #745). The completed response is stored before the first response.completed event reaches the client; error-terminated streams that never emit response.completed are not stored, matching the non-streaming error paths, and passthrough streaming is unchanged because the upstream owns storage there.
• Resolve item_reference input items before strategy dispatch so /v1/responses no longer returns HTTP 400 for Anthropic/Claude backends on a multi-step tool round-trip that submits an item_reference instead of an inline item (#743, refs #741). References are rewritten to inline FunctionCall/Message/FunctionCallOutput items (de-duplicated by call_id, first-wins), build_context_for_user reconstructs stored function-call output items as proper tool_use/tool_result pairs, the OpenAI/Azure passthrough path still forwards references unchanged, an unresolvable reference returns a descriptive 400 naming the id, and a 256-item cap (MAX_ITEM_REFERENCES) bounds the per-request session-store scan.
• Propagate server.workers to the Tokio runtime (#736, refs #734). The value was documented and shipped in config.yaml.example but had no effect, because main used an argument-less #[tokio::main] and the runtime always ran with num_cpus::get() worker threads. main is now synchronous: it peeks server.workers from the config file, builds a correctly sized multi-thread runtime through the existing RuntimeConfig::build_runtime path, and runs the async body on it, falling back to the CPU count when the value is unset or 0.
• Emit conformant function_call output items and argument events on /v1/responses streaming for non-passthrough providers (Anthropic and chat-completions-backed routes), preserving text output payloads and tracking interleaved parallel tool-call arguments by upstream index (#725).
• Reuse the shared ApiKeyStore in the Files API routes instead of constructing a separate store, so a runtime-managed API key is recognized consistently across the Files API and the rest of the router (#706).
• AppProxy: preserve sibling circuits on single-circuit events (#737, closes #731). A worker serving two or more circuits previously wiped every unaffected sibling's appproxy-* backends on any single-circuit event (leaving them 404/502 until the next pull-reconcile, up to 15s), because RegistryEntry carried no route info and the rebuilt set held only the delta circuit. The full circuit is now cached on each RegistryEntry and unchanged siblings are rebuilt from it, so only the delta circuit's backends change.
• AppProxy: reach the fallback chain from wildcard subdomain ingress (#738, closes #735). A registered circuit whose replicas are all down is no longer a dead end; after per-circuit authorization it is pinned to its canonical model and handed to the normal pipeline, where FallbackService takes over (the "deployment went down, traffic goes to a cross-provider model" behavior operators expect). The fall-through is scoped to registered-but-down circuits, a truly unknown subdomain stays a 404, and the open-to-public / bearer-token / IP-allow-list gates still run first.
• AppProxy: preserve event-known models during periodic reconcile (#739). When the Redis event overlay learned a circuit's model before any successful pull probe, a reconcile could evict the registered-but-down registry entry and break scoped fallback; reconcile now reuses the shared registry's known model before probing replicas.
• AppProxy: align WorkerRegisterResponse deserialization with the Backend.AI coordinator's actual response shape, which carries slots as an array plus available_slots as the count (#726).

Documentation¶

• Rewrite the Zensical user documentation as a current-state manual: drop development-log narration, the roadmap, and "coming soon" entries; correct configuration-reference drift against the actual config structs (nonexistent sections and keys, retry field names, the admin auth shape, the environment-variable tables, and the config discovery order); document previously missing shipped behavior (seven CLI flags, the auth login subcommand, Windows AF_UNIX support and SSE over Unix sockets, and the Windows/musl and .deb release artifacts); and bring the Korean docs to parity with English.
• Add the AppProxy worker mode design document (#708).
• Condense the README "Recent Updates" list to one concise line per release.

Dependencies¶

• Remove the validator derive dependency and its transitive proc-macro-error2, clearing the RUSTSEC-2026-0173 advisory that previously needed a temporary cargo-deny ignore while validator had no safe upgrade path (#733, #732).
• Update Rust package versions (#732).

v1.8.2 - 2026-06-02¶

Fixed¶

• Stop forwarding the client Accept-Encoding header on the /v1/responses path (#702). When a client sent Accept-Encoding: gzip, deflate, br, the responses-path header filter omitted accept-encoding from its block list and forwarded it to the upstream backend, which then negotiated gzip and returned compressed bytes. Because reqwest disables automatic decompression once any Accept-Encoding header is set manually (the explicit .header("Accept-Encoding", "identity") call only appended a second value rather than replacing the forwarded one), the SSE transform received raw gzip bytes, parsed them as text, and dropped the leading response.created, output_item.added, content_part.added, and output_text.delta events, leaving only tail fragments with empty item_id and text. "accept-encoding" is now in FILTERED_HEADERS for both the primary convert path (src/http/handlers/responses.rs) and the responses-native passthrough path (src/proxy/responses_only.rs), restoring parity with the chat-completions proxy (src/proxy/backend.rs) so the upstream only ever receives Accept-Encoding: identity.
• Stop double-wrapping Responses SSE lines so the non-GPT /v1/responses streaming conversion emits single-layer OpenAI-compatible SSE records on the converted Anthropic and Chat-Completions paths instead of nested ones (#701).

Dependencies¶

• Bump uuid 1.23.1 → 1.23.2, redis 1.2.1 → 1.2.2, socket2 0.6.3 → 0.6.4, and serial_test 3.4.0 → 3.5.0 (#699).

Tests¶

• Hardening regression coverage for the production StreamService conversion processors, asserting single-layer SSE output on the converted Anthropic and Chat-Completions paths (#701).
• Integration regression for the /v1/responses streaming path with an Anthropic backend and a gzip-requesting client, asserting the upstream request receives only Accept-Encoding: identity and the transformed Responses SSE stream retains the full event sequence with populated text and item ids (#702).

v1.8.1 - 2026-05-29¶

Added¶

• Claude Opus 4.8 recognition with a claude_family_version parser that replaces the hardcoded opus-4-7/opus-4-6 substring gates (#693, part of #687). The four Anthropic capability predicates now compare a parsed (major, minor) version: uses_adaptive_thinking_api ≥ (4,6), model_requires_adaptive_thinking / model_forbids_sampling_params ≥ (4,7), and opus_supports_max_effort = Opus and ≥ (4,6). The parser treats the first integer token as the major and the next version-like token (1 to 2 digits, value < 100) as the minor, so an 8-digit date suffix like 20250514 yields minor 0 and is never mistaken for a version, and new minor releases are recognized without per-version edits. Adds the claude-opus-4-8 metadata entry (1M context, 128K output, \(5/\)25 pricing, adaptive thinking, Jan 2026 cutoff) and registers claude-opus-4-8 / claude-opus-4-8-latest in the built-in supported models and config samples. Behavior for 4.5/4.6/4.7 and Sonnet variants is preserved.
• Anthropic fast mode behind a per-backend anthropic_fast_mode opt-in (default off) (#694, part of #687). is_fast_mode_eligible returns true only for Opus 4.6/4.7/4.8 and later Opus minors; merge_beta_header comma-joins and de-duplicates beta tokens while preserving any client-supplied anthropic-beta. On the native /anthropic/v1/messages path, resolve_fast_mode_beta injects the merged fast-mode-2026-02-01 beta header only when the request is speed: "fast", the model is eligible, the backend is native Anthropic (never Bedrock), and the opt-in is enabled; when fast mode does not apply, speed is stripped from the outgoing body so it cannot trigger a spurious upstream 400. The OpenAI-compatible path forwards speed: "fast" and injects the beta header only for eligible, opted-in, native Anthropic targets. The Anthropic -> OpenAI and Anthropic -> Google fallback parameter mappings remove speed so a fast-mode request that falls back to a non-Anthropic backend does not leak the native-only field. usage.speed is preserved on the response.
• Mid-conversation system messages for Claude Opus 4.8+ (#695, part of #687). A role:"system" entry inside the messages array (which earlier Claude families reject with HTTP 400) is now accepted, gated on a new supports_mid_conversation_system(model_id) predicate that reuses claude_family_version and matches family version ≥ (4,8). The native handler round-trips the entry unchanged to a native Anthropic backend; the cross-provider transforms map the System role onto the OpenAI system role and preserve it as user-role text for Gemini and Responses. The OpenAI-compatible transform emits mid-conversation system/developer messages (after the first user turn) as in-array role:"system" entries for supporting models, while leading system messages still fill the top-level system field. Non-supporting models (Opus 4.7 and below, all Sonnet/Haiku, Bedrock-prefixed ids, non-Claude ids) keep the historical flattening into the single top-level system.
• Refusal stop_details and the refusal stop reason propagated through the full Anthropic response pipeline (#696, part of #687). map_anthropic_finish_reason maps "refusal" to "content_filter"; the non-streaming transform and the streaming message_delta handler attach the stop_details object to the choice when stop_reason is "refusal", and omit the key (rather than forwarding a null) when upstream sends an explicit null.

Fixed¶

• Accept an input_image content part that references a Files API upload via file_id instead of an inline image_url on POST /v1/responses (#686, refs #681). Because the parent enums are #[serde(untagged)], a {"type":"input_image","file_id":"file-..."} part previously failed deserialization with a generic untagged-enum error and Axum returned HTTP 422. image_url is now optional with an added file_id, mirroring input_file; a shared resolve_local_file_to_data_url helper resolves a local file_id to an inline base64 image_url data URL through the same metadata, ownership, size, load, and base64 sequence, honoring ownership and the 10MB size limit. The OpenAI/Anthropic/Gemini converters handle the optional image_url, emitting the image when resolved and warning + skipping an unresolved file_id. validate_request walks message content and rejects an input_image with neither image_url nor file_id with a clear 400 before file resolution.
• Harden the Claude Opus 4.8 routing gates so fast-mode speed is only forwarded when the transport has confirmed native-Anthropic opt-in and beta-header injection, non-Opus Claude families keep mid-conversation system messages flattened, and OpenAI-compatible Anthropic responses preserve usage.speed (#698, refs #687).

Documentation¶

• Document Claude Opus 4.8 support in English and Korean (#697, closes #692, part of #687): add claude-opus-4-8-* to the adaptive-thinking model list and the sampling-params-deprecated warning in reasoning-effort.md; add the Claude Opus 4.8 model detail, an Anthropic Fast Mode section, a Mid-Conversation System Messages section, and the refusal stop_reason -> content_filter mapping to backends.md; and add a changelog entry covering model recognition, fast mode, mid-conversation system messages, and refusal stop_details.
• Document input_image file_id support in api.md (English and Korean), noting that exactly one of image_url or file_id is required and that file_id is resolved to an inline base64 data URL before reaching the backend under the same ownership and 10MB size limit as the input_file path (#686, refs #681).

Tests¶

• claude_family_version and supports_mid_conversation_system boundary tests (4.7 false, 4.8 true, Sonnet/Haiku/older false, Bedrock-prefixed and cross-region/ARN ids false), fast-mode eligibility, beta-header merge/dedup, speed/usage.speed (de)serialization, resolve_fast_mode_beta gating with client-beta merge, and speed non-leak versus preservation across fallback providers (#693, #694, #695).
• Refusal coverage: non-streaming and streaming refusal with and without stop_details, the explicit-null omission path, and regression tests for end_turn/max_tokens/stop_sequence/tool_use (#696).
• input_image file_id-only and image_url-only deserialization, the full failing request payload as a regression, FileResolver resolution honoring ownership and the size limit, converter output across all three backends, and the neither-field validation case (#686).

v1.8.0 - 2026-05-28¶

Added¶

• Per-API-key backend access control via an optional allowed_backends allow-list on client API keys (#677, closes #674). When the list is non-empty, requests authenticated with that key may only route to the named backends; an empty or absent list keeps the existing unrestricted behavior. The field is integrated end-to-end: config file and hot-reload, the runtime ApiKey and AuthContext, the backend-selection chokepoint and the Responses / Anthropic selection paths, cross-provider fallback, the Admin REST API (create/update/get/list), runtime-key persistence, and the models-listing endpoints.
• select_backend_with_retry and the Responses (StreamService), Anthropic-native, count_tokens, and image handlers filter candidates by the key's allow-list. When the model exists but the allow-list rejects every candidate, the request is rejected with a new RouterError::Forbidden variant mapped to 403 with error_type = "permission_error" (non-retryable), distinct from the 401 AuthError.
• /v1/models, /v1/models/extended, and /anthropic/v1/models are filtered to models served by at least one allowed backend when a restricted key is authenticated; GET /v1/models/{model} returns 404 for a model the key cannot reach. Unauthenticated or unrestricted callers see the full list.
• A new api_optional_auth_middleware is layered in permissive mode. It validates a presented bearer token on a best-effort basis and attaches AuthContext without ever rejecting, so per-key restrictions apply to authenticated callers while anonymous and invalid-token callers pass through unrestricted. The existing blocking-mode api_auth_middleware is unchanged; the two are never layered together.
• Config validation warns (does not hard-fail) when a key's allowed_backends references an unknown backend name, so a backend rename does not brick the router before operators update the keys.
• fallback.mid_stream_enabled config field (default true) so operators can keep cheap pre-stream backend re-selection on the initial connection while turning off the per-stream mid-stream buffering (#680, closes #676). Previously fallback.enabled was all-or-nothing: on meant both pre-stream and mid-stream fallback (with a per-stream StreamAccumulator buffering roughly 100 to 200 KB), off meant no fallback at all. Memory-constrained or high-concurrency hosts now have a middle ground.
• The streaming dispatch becomes a three-way decision factored into a pure decide_streaming_fallback_dispatch helper. With fallback.enabled true and a chain configured, mid_stream_enabled = true keeps the buffering path (handle_streaming_with_mid_stream_fallback), mid_stream_enabled = false routes to the revived handle_streaming_with_pre_stream_fallback (no StreamAccumulator or MidStreamFallbackContext allocation; mid-stream failures surface as a normal stream error), and otherwise the standard no-fallback path runs.
• The previously dead handle_streaming_with_pre_stream_fallback and advance_to_next_fallback are now live; the #[allow(dead_code)] markers are removed and the per-key allow-list is threaded through fallback re-selection so the newly-live path does not bypass per-API-key access control.

Changed¶

• Removed the streaming-local copies of transform_payload_for_openai and its requires_max_completion_tokens helper from src/http/streaming/handler.rs; both call sites now resolve to the canonical implementations in crate::proxy::utils (#679, closes #660). The two copies were byte-for-byte identical, creating a drift risk where a change to one would not be mirrored to the other. The streaming-local duplicate unit tests are removed since the canonical tests in src/proxy/utils.rs already cover the same contract.

Fixed¶

• Enforce the per-API-key backend allow-list in the default mid-stream fallback handler when it re-selects a backend after a mid-stream failure (#683). With fallback.mid_stream_enabled = true (the default), a restricted key whose fallback chain mapped to a disallowed backend was transparently switched to it on a mid-stream failure, an access-control bypass that the new mid_stream_enabled work in PR #680 had only closed on the pre-stream path. handle_streaming_with_mid_stream_fallback now takes an owned allowed_backends: Option<Vec<String>> (moved into its spawned streaming task), and a new resolve_allowed_backend_name_for_model helper wraps resolve_backend_name_for_model and applies the same allowed_backends.filter(|l| !l.is_empty()) + exact-name membership semantics used everywhere else in the module (try_get_healthy_backend_for_model, get_backend_for_model_streaming, get_healthy_backend_for_streaming). All three fallback re-selection sites resolve through it; a disallowed or unresolvable candidate returns None, so the existing warn + continue arm skips it and the chain index advances. When every remaining candidate is filtered out the loop terminates via the existing chain-exhausted paths and surfaces an error to the client. An empty or None allow-list is byte-for-byte the prior unrestricted behavior.
• Resolve an allowed fallback backend before rebuilding the fallback payload in mid-stream fallback, so a disallowed chain entry can no longer mutate current_payload before being skipped (#684).
• Anthropic-native x-api-key callers now supply the same allowed_backends policy as Authorization: Bearer-authenticated callers when no AuthContext is present, covering Messages, count_tokens, and models listing (#684). The previously documented limitation that the per-key allow-list was unenforced on the native Anthropic surface is now closed.
• Close a HIGH-severity authorization-bypass on POST /v1/responses/compact found in the post-merge security audit of #677. The endpoint was the one client-facing model-routing handler that never read AuthContext from request extensions, so a key scoped to backend set A could reach a disallowed passthrough backend B (OpenAI / Azure) through compaction whenever B served the requested model. compact_response now mirrors create_response exactly: it derives the per-key allow-list via allow_list_from_auth, filters the model's candidate backends against it before any health-check or passthrough, and returns a deterministic 403 permission_error when the model exists but the key cannot route to any backend serving it. The filter precedes backend forwarding, so the rejection happens without contacting an upstream.

Documentation¶

• Clarify in config.yaml.example, src/services/streaming/mid_stream_config.rs rustdoc, and docs/en/configuration/advanced.md that mid_stream_fallback.enabled does not disable mid-stream buffering or reduce memory: regardless of its value the StreamAccumulator is still constructed and still buffers up to roughly 100 KB per stream, and mid-stream fallback still activates on backend failure (#678, closes #675). The flag only selects continuation versus restart mode for the fallback request. The real kill-switch for the buffering and memory is fallback.enabled: false or omitting the model from fallback.fallback_chains. The Korean docs (docs/ko/configuration/advanced.md) had no corresponding Mid-Stream Fallback section so no Korean change is made for the clarification, but #680 separately added a translated "미드스트림 버퍼링 비활성화" subsection covering the new toggle.
• Document the per-API-key allowed_backends allow-list in the security docs (English and Korean) including the now-closed x-api-key Anthropic Messages limitation (#677, #684).

Tests¶

• Deterministic unit tests for resolve_allowed_backend_name_for_model: allow-list excludes the resolved backend returns None, includes it returns Some, empty list returns Some, None matches the unfiltered resolver, unresolvable model returns None (#683).
• Integration tests in tests/per_key_backend_access_test.rs driving /v1/responses/compact behind blocking auth: a restricted key requesting a model served only by a disallowed backend gets 403 permission_error (the security case), and a restricted key requesting an allowed backend's model passes the filter (#677).
• Unit test for the new decide_streaming_fallback_dispatch helper covering the three-way decision matrix between fallback.enabled, mid_stream_enabled, and chain presence (#680).

v1.7.1 - 2026-05-28¶

Fixed¶

• Anthropic web_search_20250305 server-tool emulation no longer masks provider failures as empty result sets. A missing API key, HTTP 401/403/429, timeout, or parse error now surfaces to the client as a web_search_tool_result_error content block with a mapped error_code ("too_many_requests" for HTTP 429, "unavailable" for all other failures). A genuinely empty-but-successful search continues to emit an empty content: [] array, so the two outcomes remain distinguishable. Both non-streaming and streaming SSE code paths are covered, with an explicit SSE event-ordering assertion guarding the streaming error path. Serper response-shape drift (a body that parses cleanly but omits the organic key) is now detected and logged with a provider-tagged warn! that records only the observed top-level keys, never the user query text. (#671, #672, #673)

Dependencies¶

• Bump lru 0.16 → 0.18, reqwest 0.13.3 → 0.13.4, and rusqlite 0.39 → 0.40 (libsqlite3-sys 0.37 → 0.38, unpinned now that CI runs Rust 1.95), plus a transitive lockfile refresh (aws-lc-rs 1.16.3 → 1.17.0, h2 0.4.13 → 0.4.14, http 1.4.0 → 1.4.1, and hyper/axum/reqwest knock-on revisions); cargo audit reports zero vulnerabilities across the dependency graph (#669, #670).

v1.7.0 - 2026-05-26¶

Added¶

• AWS Bedrock Claude backend Phase 1 (bedrock-mantle) over a Bearer token (#616, closes #613)
• New type: bedrock backend with serde aliases (aws-bedrock, bedrock-anthropic, AwsBedrock, ...). is_commercial() returns true and owned_by() returns Some("anthropic") so OpenAI-shaped clients see the expected model lineage.
• endpoint_type: mantle (default) speaks the native Anthropic Messages API at the region-templated https://bedrock-mantle.{region}.api.aws, routes to /anthropic/v1/messages, uses Authorization: Bearer, and omits anthropic-version (Bedrock returns HTTP 400 if present). An explicit url: field overrides the template for proxies and tests; empty or uppercase regions are rejected at load time.
• model_ids.rs recognizes plain (anthropic.<family>), geographic (us./eu./jp./au.), global (global.anthropic.<family>), and full-ARN identifiers, forwarded unchanged with no automatic alias mapping because the geo prefix carries real billing and residency consequences.

• The existing OpenAI/Anthropic body transforms and the Anthropic SSE stream transformer are reused unchanged, so per-model quirks (Opus 4.7 sampling-param ban, adaptive thinking) apply identically to Bedrock without duplication. endpoint_type: runtime is reserved here and implemented in Phase 2 below.

• Request-path rate limiting is now enforced, plus a Redis storage backend (#635, #632, closes #626)

• state.rs previously dropped the MiddlewareLayer returned by initialize_rate_limiting, so the rate_limiting.* config was a silent no-op. The layer now flows through ServiceHandles → ContinuumRouterBuilder::build_router → Router::layer and attaches to the assembled Axum app, so configured budgets actually return 429.
• All five rate_limiting dimensions are now optional: per_client, per_backend, and global join the already-optional per_api_key/per_model. Operators may omit any dimension and load with it disabled; the Default impl retains all three so existing deployments are unaffected (#632).

• New redis-cache-gated rate_limit_v2::redis_backend runs a token-bucket and a sliding-window Lua script, each as a single atomic EVAL, reusing the shared create_redis_pool helper with keys like cr:rl:per_client:10.0.0.1. On any Redis failure (pool unavailable, timeout, Lua error) the backend reports BackendUnavailable and the caller falls through to the in-process token bucket, degrading to per-replica enforcement rather than dropping requests.

• Cross-provider fallback is now wired into request dispatch and hot-reload (#631, #637, #665)

• The completed src/core/fallback/ module (~4,900 lines, 36 unit tests) was never called from the request path, so configured fallback.fallback_chains were a silent no-op. FallbackService now runs for chat_completions (non-web-search), completions, embeddings, rerank, sparse_embeddings, and image generation, with an execute_with_optional_fallback wrapper (no overhead when fallback is unconfigured), X-Fallback-* response headers, and a From<RouterError> to TriggerReason mapping that satisfies the executor bound (#631).

• fallback.fallback_chains and fallback.fallback_policy changes now apply at runtime through the hot-reload subscriber via FallbackService::update_config; toggling fallback.enabled remains restart-only and is documented as such in config.yaml.example (#637, #665).

• POST /v1/models/refresh force-refresh endpoint for interactive desktop use (#593, #664)

• Clears the ModelCache (all_models key) and synchronously re-aggregates from all configured backends before responding, returning the same {"object":"list","data":[...]} shape as GET /v1/models. Desktop clients (e.g. backend.ai-go "Refresh models" button) can use the response immediately without a second round-trip.
• Rate-limited per verified API key, with anonymous or invalid-token callers sharing one global anonymous bucket: 3 requests per 5-second burst window, 12 per minute. Callers that exceed the limit receive 429 Too Many Requests. The limit is intentionally tighter than the regular list endpoint because each call triggers an upstream fetch from every configured backend.
• Gated by a new model_aggregation.allow_force_refresh: bool config field (default true). Setting it to false makes the endpoint return 403 Forbidden, suitable for hardened deployments where clients must rely on TTL-based expiry.
• Each refresh logs the verified API key ID when available, or anonymous otherwise, at INFO level for audit correlation.
• config.yaml.example extended with desktop-embedded guidance: cache_ttl: 10, soft_ttl_ratio: 0.5, and allow_force_refresh: true for backends.ai-go style embedded proxy.

• New force_refresh(state) helper on ModelAggregationService encapsulates the clear-then-aggregate flow. allow_force_refresh() accessor exposes the config flag to handlers.

• AWS Bedrock Claude backend Phase 2: bedrock-runtime with SigV4 + AWS binary event-stream (#614)

• New endpoint_type: runtime value on type: bedrock backends targets https://bedrock-runtime.{region}.amazonaws.com/model/{modelId}/invoke[-with-response-stream]. The router signs each request with AWS Signature V4 (service: "bedrock"), wraps the OpenAI → Anthropic body with "anthropic_version": "bedrock-2023-05-31", strips the top-level "model" field (Bedrock takes the model ID from the URL path), and percent-encodes the model identifier into the path so versioned foundation IDs (anthropic.claude-3-5-sonnet-20240620-v1:0) and full ARNs round-trip cleanly.
• Streaming responses arrive in the AWS application/vnd.amazon.eventstream binary frame format. A new bedrock::event_stream::EventStreamDecoder reassembles frames that span multiple TCP reads, base64-decodes each chunk payload, and emits synthetic event: <type>\ndata: <json>\n\n SSE bytes for the existing AnthropicStreamTransformer to translate into OpenAI-shape SSE. Exception frames (ThrottlingException, ValidationException, ...) surface as synthetic event: error SSE chunks instead of being silently dropped.
• AWS credentials resolve in this order: inline auth.aws.access_key_id + auth.aws.secret_access_key (+ optional session_token), then a named profile via auth.aws.profile, then the standard AWS chain (env vars, shared config, IMDS, IRSA / EKS pod identity, ECS task role). The resolver is fronted by aws_credential_types::provider::SharedCredentialsProvider so temporary credentials refresh transparently between requests.
• New BackendAuthType::Sigv4 variant on BackendAuthConfig, plus an AwsAuthConfig sub-block under auth.aws. Both are wired through Debug redaction so static credentials never leak into logs. BackendAuthType accepts sigv4, aws_sigv4, and aws-sigv4 as YAML spellings.
• Health check for runtime probes POST /model/{probe_model}/invoke with a single-token body; HTTP 2xx, 400, 401, 403, and 429 all count as healthy because they prove the AWS surface is reachable and the operator can address auth/billing issues separately.
• All AWS SDK crates (aws-sigv4, aws-smithy-eventstream, aws-credential-types, aws-config) sit behind a new optional bedrock-sigv4 Cargo feature. Default builds do not pull them in; configuring endpoint_type: runtime without the feature returns a clear error pointing at the rebuild flag instead of failing at the AWS edge. The Phase 1 mantle path is unaffected and works with or without the feature.
• Proxy header policy in src/proxy/backend.rs and src/http/handlers/anthropic/handler.rs splits on endpoint_type: Bedrock-mantle keeps Authorization: Bearer, Bedrock-runtime suppresses the static-Bearer injection because the Backend trait implementation signs each request with SigV4.

• English and Korean docs in docs/{en,ko}/configuration/backends.md extended in place with the new endpoint_type: runtime configuration, build requirement, credential chain, IAM policy snippet, geo/global profile behaviour, and a streaming-pipeline overview.

• EXAONE 4.0 (vLLM) registration with a request-gated hybrid-model thinking transform (#640, refs #639)

• EXAONE 4.0 (e.g. EXAONE-4.0-32B-FP8-RNGD) is a hybrid reasoning model: in reasoning mode it streams chain-of-thought inline in content ended by a lone </think>; in non-reasoning mode it emits a plain answer with no </think>.
• The assume_reasoning_first (unterminated_start) transform is now gated on the request actually enabling thinking (chat_template_kwargs.enable_thinking or top-level enable_thinking; conservative default false) at both the HTTP and Unix-socket streaming decision points, so non-reasoning mode no longer emits the whole answer as reasoning_content with empty content. Standard-pattern models keyed off a real <think> marker are unaffected.

• Registers exaone-4.0-32b with the unterminated_start config; the served -RNGD name resolves via the hardware-suffix peel below.

• NPU/accelerator hardware-variant suffix normalization in model-id matching (#662)

• Adds a HARDWARE/ACCELERATOR category (rngd, warboy, atom, atommax, rebel) to is_recognized_format_token() so FuriosaAI (RNGD/WARBOY) and Rebellions (ATOM/ATOMMAX/REBEL) serving-target suffixes normalize to the canonical base metadata entry through the existing layered peel chain, without a per-model alias. layered_format_strip() lowercases before peeling, so runtime-emitted upper-case names resolve correctly.
• Exact-id and exact-alias phases run before the peel, so any model legitimately registered as *-atom or *-rebel wins via exact match first; a grep of shipped model-metadata.yaml confirms zero current collisions. EXAONE-4.0-32B-FP8-RNGD now normalizes to exaone-4.0-32b by peel (-rngd then -fp8).

Changed¶

• Narrow the fallback handler's LM Studio compatibility shim so only / and /v1/models return 200; all other unmatched routes now return 404. The JSON error body shape is preserved unchanged, so any consumer already reading the body continues to work (#628).

Fixed¶

• Strip seven non-OpenAI top-level fields (chat_template_kwargs, thinking_budget_tokens, enable_thinking, preserve_thinking, top_k, min_p, repeat_penalty) before forwarding to Gemini's /v1beta/openai/chat/completions endpoint, which returns HTTP 400 INVALID_ARGUMENT for unknown keys; the extra_body escape hatch is untouched and reasoning_effort stays (Google maps it to thinking_level). Also extend the 3.5-flash thinking-disable and is-thinking matchers, since 3-flash did not substring-match 3.5-flash (#642).
• Wire request-stats recording into every Anthropic handler code path (native HTTP/Unix, Bedrock mantle/runtime, OpenAI-compatible, Responses API) for both streaming and non-streaming, adding an AnthropicStreamUsageTracker that accumulates input/output tokens from raw passthrough SSE (#627, #634).
• Use the configured timeouts.request.streaming.chunk_interval instead of a hardcoded 60s for mid-stream inactivity, and emit bounded keep-alives so a silent backend now advances to the next fallback model via StreamOutcome::Failed rather than emitting keep-alive comments forever (#633).
• Accept partial model_overrides.<model>.streaming/standard blocks via new StreamingTimeoutOverride/StandardTimeoutOverride structs whose Option<String> fields merge over the base config, fixing YAML parse failures on the --generate-config output path (#630).
• Gate admin/metrics/metrics-persistence/webui imports and functions behind their Cargo features, and add a #[cfg(not(feature = "metrics"))] no-op metrics stub mirroring the public surface used by always-compiled callers, so feature-reduced builds compile cleanly (#629, #666, closes #636).
• Harden force-refresh rate limiting so anonymous and invalid-token callers share one global bucket, preventing spoofed Authorization/X-Forwarded-For/X-Real-IP headers from bypassing the budget.
• Metrics history query limiting, UTF-8-safe metric label truncation, and Bedrock runtime routing through the typed SigV4 implementation (follow-up to #608, #609, #613, #614).

Documentation¶

• Document the Bedrock backend (region selection, geographic vs global inference profiles, model ID format, credential chain, IAM policy snippet, and streaming pipeline) in docs/{en,ko}/configuration/backends.md; add a Force-Refresh Models section to docs/en/api.md; and extend config.yaml.example with desktop-embedded model-aggregation guidance and fallback hot-reload annotations.

Tests¶

• Negative and positive case coverage for transform_payload_for_openai (#661).
• Rate-limit middleware hot-reload tests with documented bucket-reset behaviour, plus router_wiring_tests that build a real ContinuumRouter and assert 429 fires when the burst is exhausted (#635, #638, #667).
• Verify web_search injection interacts correctly with the passthrough contract (#663).
• MLxcel streaming passthrough integration test (#659).
• Bedrock unit and integration coverage: serde aliases, URL templating, header policy, model-ID parsing (geo/global/ARN), runtime SigV4, and event-stream frame decoding driven against a wiremock server (#616, #614).

Dependencies¶

• Bump tokio 1.52.1 → 1.52.3, tower-http 0.6.8 → 0.6.11, dashmap 6.1.0 → 6.2.1, serde_json 1.0.149 → 1.0.150, aws-config 1.8.16 → 1.8.17, aws-sigv4 1.4.3 → 1.4.4, and aws-smithy-types 1.4.7 → 1.4.8 (#619, #658).

v1.6.3 - 2026-05-12¶

Added¶

• Per-API-key LLM token usage metrics (#608, #610)
• New Prometheus llm_tokens_total{api_key_id, model, backend, kind} counter that records actual prompt and completion token consumption per API key, model, backend, and token kind. The hot-path counter's label set is intentionally minimal — extra dimensions live on the companion info-metric below.
• Companion api_key_info{api_key_id, ...} info-metric exposes a configurable allowlist of per-API-key annotation labels (e.g. email, team, environment) so dashboards can group/filter the token counter via standard PromQL * on(api_key_id) group_left(...) joins without bloating the hot-path counter's label set.
• derive_api_key_id returns either the configured id (when the auth layer matched the request) or a SHA-256 first-12-hex prefix k_<hex> of the raw bearer token. The raw key is never used as a label. A dedicated ApiKeyCardinalityTracker (default cap: 1000 unique key IDs) prevents label-cardinality explosion.
• ApiKeyConfig and the in-memory ApiKey gain an annotations: HashMap<String, String> field. MetricsConfig gains annotation_labels: Vec<String> — the allowlist that materializes as labels on api_key_info. Reserved canonical annotation keys are documented (email, uuid, owner, team, environment); operators may add custom keys.
• Streaming and non-streaming paths record at the existing usage parse site through a new StreamObservabilityContext field on StreamTransformConfig, threaded through handle_anthropic_streaming / handle_gemini_streaming / handle_successful_backend_response so OpenAI-compat / Anthropic / Gemini / thinking-pattern streaming response builders all emit the counter without duplicate parsing. The router already injects stream_options.include_usage=true for OpenAI-compat backends, so streaming metrics work uniformly regardless of client opt-in.
• api_key_info is initialized once at startup from metrics.annotation_labels; label names are frozen at registration (Prometheus does not allow renaming labels). Annotation values hot-reload through the existing config-watch path via ApiKeyStore::refresh_info_metric, called from load_from_config, add_key, and remove_key_by_id so admin operations stay in sync.
• All label values flow through CardinalityManager / sanitize_label_value. Annotation values use a slightly less strict sanitize_annotation_value that preserves @, +, and : so emails and namespaced identifiers round-trip cleanly.
• Persistent local metrics log backed by SQLite with configurable retention (#609, #611)
• New MetricsStore async trait + bundled rusqlite v1 implementation under src/metrics/persistence/ with WAL mode, prepared-statement cache, and PRAGMA user_version schema versioning (store.rs / sqlite.rs / snapshot.rs / snapshot_task.rs). Histograms and summaries are fanned into row-per-sample form.
• Counters and gauges are NEVER restored on startup — the persistent log is a separate read path so the live /metrics endpoint keeps Prometheus monotonic-counter semantics. Historical samples are read through a new GET /admin/metrics/history?metric=...&from=...&to=... surface (src/admin_metrics_history.rs) that returns 404 when persistence is disabled at runtime and 503 when the feature is not compiled in. No PromQL in v1.
• Hot-reload pipeline in src/server/serve.rs translates config changes into PersistenceCommand::{SetSnapshotInterval, SetRetentionDays, SetCompaction} messages, atomically rebuilding the ticker and prune cutoff without dropping in-flight snapshots.
• Compaction schedule honors a minute hour * * * cron subset to avoid pulling in a full cron crate for what is effectively a daily timer.
• Defaults to enabled: true; switch off via metrics.persistence.enabled: false. The redb and duckdb variants are reserved keywords in the YAML schema and return NotImplemented at startup until they get implementations.
• Disk usage: measured ~119 bytes/sample on a synthetic 100-series × 10-snapshot workload (see tests/metrics_persistence_test::disk_usage_smoke_check_under_synthetic_load). Documented formula in docs/en/persistent-metrics.md and config.yaml.example.

Fixed¶

• Coerce token-usage label values to &str in with_label_values so the release build no longer fails type inference. Mixing &String label variables with a &str literal ("prompt" / "completion") made the compiler pick &[&String] and reject the literal — regression introduced in #610.

Documentation¶

• Korean translations for the two metrics features (#612)
• docs/ko/metrics.md: new ### API 키별 LLM 토큰 사용량 section covering llm_tokens_total, api_key_id derivation, annotation_labels allowlist, api_key_info info-metric, PromQL examples, Grafana panel, and verification steps.
• docs/ko/persistent-metrics.md: new page translating docs/en/persistent-metrics.md (SQLite-backed snapshot semantics, configuration fields, disk-usage formula, /admin/metrics/history surface, schema layout, operational notes).
• docs/ko/admin-api.md: insert ## 지속 메트릭 로그 API section between Stats and Response Cache, plus a matching TOC entry.
• zensical.ko.toml: add 지속 메트릭 로그 nav entry under 운영 so the page is reachable from the Korean sidebar.
• New docs/en/metrics.md ### Per-API-Key LLM Token Usage section covering metric definition, api_key_id derivation rules, annotation config schema, cardinality and hot-reload semantics, example PromQL (tokens-per-email, top-10 keys, per-team rollup), a Grafana panel example, and verification steps. config.yaml.example gains a documented metrics.annotation_labels block and an annotations: example under each API-key entry.

Tests¶

• Per-API-key token-usage unit coverage: derive_api_key_id priority (configured id wins; otherwise hash; otherwise anonymous), determinism, hash format ^k_[0-9a-f]{12}$, annotation-label normalization, info-gauge one-time-init, refresh atomicity, cardinality bounds, email-preserving annotation sanitizer; streaming-transformer write-through; middleware annotation-snapshot exposure; integration coverage in tests/metrics_integration_test.rs (4-label counter with both kinds, anonymous fallback, hash regex). (#610)
• Persistent-metrics SQLite store unit coverage (insert, query by time range, retention deletion, idempotent open, unknown-kind round-trip) and integration coverage in tests/metrics_persistence_test.rs (snapshot task lands rows in SQLite, retention prunes only old samples, retention hot-reload preserves in-flight snapshots, disk-usage smoke check). (#611)

v1.6.2 - 2026-05-10¶

Fixed¶

• /v1/responses and /v1/chat/completions now accept the OpenAI reasoning-API developer role (#603, #605, #606)
• Add MessageRole::Developer with a serde lowercase rename so "developer" deserializes as a first-class variant. The previous failure surfaced as a misleading did not match any variant of untagged enum ResponseInput rather than naming the unknown role; the implicit-message deserialization error now names the offending role string and lists the valid roles.
• Per-backend translation: pass through as developer for OpenAI-compatible servers; merge into the Anthropic top-level system parameter (concatenated with \n\n when both system and developer text are present, fixing a pre-existing overwrite bug); merge into Gemini system_instruction; map to system for Ollama (older builds reject developer).
• Chat Completions → Responses converter recognizes developer as instruction-bearing: the first occurrence becomes top-level instructions; subsequent occurrences remain as input items with their original role preserved on the wire.
• Treat developer and system equivalently in cross-cutting string-based recognition sites: prefix-cache key extraction, cross-provider fallback translation, the OpenAI-to-Anthropic transform's system-content extraction, the global-prompt injector's existing-system-message lookup, and the smart-routing classifier / LLM prompt builder.

Documentation¶

• Migrate the docs site from MkDocs to Zensical and restore brand styling (#602)
• Remove mkdocs.yml and mkdocs.ko.yml in favor of native zensical.toml and zensical.ko.toml, both rooted under the [project] namespace per Zensical's TOML schema; per-extension options live inside [project.markdown_extensions] as a dict (Zensical's config loader ignores any separate mdx_configs table).
• Replace docs/en/shared and docs/ko/shared symlinks with rsync -a --delete docs/shared/ docs/{en,ko}/shared/ invoked before each build, since Zensical does not follow symlinks for asset directories.
• Register the lablup brand color via Zensical's documented primary = "custom" mechanism plus a [data-md-color-scheme="default"][data-md-color-primary="custom"] selector in docs/shared/stylesheets/extra.css that defines the orange CSS variables.
• Mermaid is registered as a pymdownx.superfences custom fence rather than relying on the now-incompatible mermaid2 plugin; favicon falls back to logo.png when missing.
• Restore Zensical render output for icons, diagrams, and brand color (#604)
• Re-enable pymdownx.emoji with the zensical.extensions.emoji twemoji index/generator (replaces the removed materialx) so :material-*: icon syntax stops rendering as literal text.
• Reimplement the <!-- diagram: PATH --> ... <!-- /diagram --> ASCII-replacement as a Python-Markdown extension (docs/hooks/diagram_extension.py); the prior MkDocs on_page_content hook does not run because Zensical exposes no MkDocs hook lifecycle. Add docs/__init__.py and prefix builds with PYTHONPATH=. so the extension is importable from Zensical's console-script entry point.
• Set --md-primary-bg-color on the custom palette and override .md-header / .md-tabs so the orange brand band paints on top of Zensical's modern layout.
• Move the nav table above the first [project.X] sub-table in both TOMLs so it stops being silently parsed under [[project.extra.social]] (alphabetical fallback was producing an unsorted top menu and wrong prev/next footer neighbors).

Tests¶

• Regression coverage for system/developer concatenation in the Anthropic transform on both streaming and non-streaming paths, plus per-backend converter mapping for the developer role across all five backends and the Chat Completions → Responses converter's developer-then-system ordering (#605, #606).

Dependencies¶

• Bump redis 1.2.0 → 1.2.1 (#598).

v1.6.1 - 2026-05-07¶

Fixed¶

• Claude Opus 4.7 (claude-opus-4-7) now routes correctly through the Anthropic backend (#599, #600, #601)
• Extended the adaptive thinking API gate (uses_adaptive_thinking_api) to include 4.7-series model IDs. Claude Opus 4.7 requires thinking.type == "adaptive" + output_config.effort; sending the legacy budget_tokens shape produces HTTP 400.
• Added model_requires_adaptive_thinking and model_forbids_sampling_params predicates for 4.7-series request-shape rules: explicit manual thinking is normalized to adaptive thinking and temperature, top_p, and top_k are dropped unconditionally before forwarding.
• Extended opus_supports_max_effort to include Opus 4.7 so xhigh reasoning effort maps to output_config.effort = "max" on Opus 4.7.
• Added claude-opus-4-7 and claude-opus-4-7-latest to the built-in supported-models list and to model-metadata.yaml; the speculative claude-sonnet-4-7 entry is intentionally not advertised until Anthropic publishes it (defensive request-shape matching is retained for user-supplied configurations).

Documentation¶

• Update reasoning-effort docs (EN + KO) and backends.md to cover the Claude 4.7 family adaptive-thinking requirement and unconditional sampling-parameter deprecation (#600).

Tests¶

• Responses API regression coverage for Opus 4.7 adaptive thinking and unconditional sampling-parameter stripping; both transform paths (Chat Completions and Responses) for the 4.7 family with negative regression on Opus 4.6 / Sonnet 4.6 / Haiku 4.5 / Haiku 3.5 (#600, #601).

v1.6.0 - 2026-05-04¶

Added¶

• ChatGPT subscription / Codex backend authentication via OAuth device flow (#551, #592)
• continuum-router auth login --backend <name> runs the OpenAI Codex three-step headless device-code flow: POST /api/accounts/deviceauth/usercode to mint a one-time user_code, POST /api/accounts/deviceauth/token polling, and a PKCE exchange at /oauth/token. Standards-compliant RFC 8628 device flow remains available for any future provider that implements it; the new OpenAICodexDeviceFlowClient is selected automatically for provider: openai.
• Tokens are wrapped in SecretString, written to the configured token_store with mode 0600 on Unix using an O_CREAT|O_EXCL open + atomic rename; a random tempfile suffix prevents concurrent saves from colliding, and a partial write is unlinked on failure so secret material does not linger on disk.
• Access-token expiry is parsed from the JWT exp claim (with a 1-hour fallback for non-JWT tokens) and clamped to a useful minimum so a degenerate expires_in from the provider cannot trigger a refresh storm.
• Proactive refresh fires 60 s before expiry, single-flighted with a tokio::sync::Mutex. A 401 from the upstream backend triggers exactly one forced refresh and a single retry; the previous refresh token is preserved race-free when the provider omits refresh_token from a refresh response.
• The strategy reports an identity_fingerprint() (backend name, client_id, token_store) so that hot-reload rebuilds the strategy when any of those rotate, instead of silently keeping the prior in-memory state.
• The CLI strips C0/C1 control characters from verification_uri_complete and user_code before printing, so a hostile provider response cannot inject ANSI escapes that rewrite the terminal.
• Every device-flow and runtime request to auth.openai.com / chatgpt.com/backend-api/codex carries originator: codex_cli_rs (configurable via auth.oauth.originator) and a codex_cli_rs/<version> User-Agent (configurable via auth.oauth.user_agent), matching the official Codex CLI so Cloudflare admits the traffic instead of returning a 403 JS challenge.
• auth.type: oauth is accepted in YAML alongside the legacy o_auth snake_case rendering. client_id and scope default to the public Codex CLI values; only token_store is required for the ChatGPT-subscription case.
• Anthropic Messages and Chat Completions surfaces both transparently route to the ChatGPT Codex backend (#592)
• Any backend whose auth.type is oauth and whose provider uses the Codex flow (currently openai) is forced through the Responses API for every request, regardless of per-model responses_only metadata. chatgpt.com/backend-api/codex exposes /responses only — no /chat/completions — so chat-shaped models (e.g. gpt-5.5, alias-mapped claude-haiku-4-5) and unknown model IDs all dispatch through /v1/responses → …/backend-api/codex/responses. Non-OAuth OpenAI backends continue to honor the per-model responses_only flag.
• New core::url_utils::compose_backend_url centralizes backend URL composition for the three OpenAI-compatible roots (/v1, /openai, /backend-api/codex). Replaces ad-hoc ends_with("/v1") || ends_with("/openai") checks across proxy/backend.rs, http/handlers/responses.rs, http/streaming/handler.rs, services/responses/stream_service.rs, and the Anthropic handler so the /backend-api/codex rule applies uniformly.
• The proxy hot path (proxy/backend.rs, proxy/responses_only.rs, proxy/image_gen.rs, proxy/image_edit.rs) now flows through a backend-name-keyed AuthStrategyRegistry exposed on AppState via src/proxy/oauth_helper.rs. The helper looks up the strategy, calls refresh_if_needed() before sending, replaces the static-bearer header with one derived from the strategy, and force-refreshes + retries once on a 401. Static api_key auth continues to work unchanged when no strategy is registered.
• The Anthropic-compatible handler (src/http/handlers/anthropic/handler.rs) consults the same registry. Client-supplied Authorization: sk-ant-… and x-api-key headers are dropped when the backend has an OAuth strategy, instead of being forwarded to OpenAI as the bearer.
• Model fetcher detects OAuth-authed backends and falls back to the configured models list rather than probing /v1/models, since chatgpt.com/backend-api/codex does not expose a models endpoint.
• Codex-compatible Responses API extensions (#536, #537)
• POST /v1/responses/compact endpoint for context compaction — passthrough to OpenAI / Azure OpenAI native /v1/responses/compact; other backend types return 501.
• store field on ResponsesRequest (defaults to true) controls upstream session persistence; Codex sends store: false for ephemeral requests.
• output_text content part type alongside input_text so converters can differentiate assistant vs. user content in input items. All converters (OpenAI, Anthropic, Gemini) handle the new variant.

Documentation¶

• Sync Codex / Responses-API extensions across the root CHANGELOG.md and the Korean docs (ko/configuration/backends.md, ko/configuration/advanced.md, ko/api.md, ko/architecture.md); resolve all zensical build warnings on both EN and KO builds and preserve unicode in toc anchor slugs via pymdownx.slugs.slugify (#596).
• Clean up AI-slop patterns across English and Korean mkdocs sources — replace em dashes in prose, remove filler/slop words, rewrite trailing participial clauses and inflated verbs, collapse colon+bullet AI-style intros, and replace closing summary slop with concrete next-action links (#597).

CI/CD¶

• Bump apple-actions/import-codesign-certs from 6 to 7 (#590).

Dependencies¶

• Bump tokio 1.51.0 → 1.52.1, axum 0.8.8 → 0.8.9, reqwest 0.13.2 → 0.13.3, clap 4.6.0 → 4.6.1, fastrand 2.4.0 → 2.4.1, uuid 1.23.0 → 1.23.1, rand 0.10.0 → 0.10.1, and lru 0.16.3 → 0.16.4 (#595).

v1.5.6 - 2026-04-29¶

Fixed¶

• /v1/chat/completions returned HTTP 502 responses_parse_failed for responses_only reasoning models (gpt-5.4-pro, gpt-5.5-pro). OpenAI's /v1/responses payload for these models contains output items shaped like { "id": "rs_...", "type": "reasoning", "summary": [] }, but OutputItem::Reasoning required content and status, so serde rejected the payload with missing field 'content'. The Anthropic Messages surface bypassed the strict variant on a different conversion path, masking the bug until directly tested. content and status are now optional on OutputItem::Reasoning; reasoning items are dropped before reaching Chat Completions clients (per existing project policy), so body shape is irrelevant beyond successful deserialization. (#594)

Changed¶

• Realign gemini-3.1-pro-preview as the canonical metadata id for the Gemini 3.1 Pro family in model-metadata.yaml, with gemini-3.1-pro (and existing -latest / -customtools forms) demoted to aliases. Matches what generativelanguage.googleapis.com actually serves today — the canonical gemini-3.1-pro form returns 404 from upstream — and avoids implying GA availability that does not exist yet. The metadata cache still resolves both forms to the same entry. Note: alias-to-canonical rewriting on the upstream-bound payload is out of scope for this release; clients calling with the gemini-3.1-pro alias will still hit upstream 404 until that work lands. (#594)
• Sample config.yaml registers the newly-available pro / 5.5 family models so the responses_only dispatch path can be exercised end-to-end against real upstreams (gpt-5.4-pro, gpt-5.2-pro, gpt-5.5, gpt-5.5-pro, claude-opus-4-7, gemini-3.1-pro, gemini-3.1-pro-preview); duplicate claude-haiku-4-5 entry removed.

v1.5.5 - 2026-04-27¶

Added¶

• Transparent Responses-API routing for OpenAI Pro models (epic #581)
• New responses_only: true capability flag in model-metadata.yaml and the built-in OpenAI registry marks gpt-5.2-pro, gpt-5.4-pro, and gpt-5.5-pro as served only on /v1/responses upstream (#574, #582)
• /v1/chat/completions requests for responses_only models are dispatched to the upstream /v1/responses endpoint and translated back into a strict-mode chat.completion (or chat.completion.chunk for streaming) envelope, transparent to the client. Stream usage is gated by stream_options.include_usage, and per-model latency / success counters are recorded for the responses_only path (#578, #584)
• /anthropic/v1/messages requests for responses_only models are converted to the Responses API shape, dispatched to /v1/responses, and translated back into Anthropic Messages JSON (or the Anthropic SSE event sequence for streaming) — tool-call round-trips, web-search emulation, and Unix-socket transports all branch on the flag (#575, #577, #583, #585, #586)
• Anthropic Messages <-> Responses request transformer covers system → instructions, tools, tool_choice (including disable_parallel_tool_use → parallel_tool_calls: false), max_tokens → max_output_tokens, reasoning effort derivation, and multi-turn tool round-trips; the response transformer preserves thinking/text/tool_use ordering and stop-reason fidelity (#575, #583)
• SSE streaming bridge (AnthropicResponsesStreamTranslator) maps Responses API events to Anthropic Messages events while preserving Anthropic's strict event-ordering invariants (single message_start, paired content_block_start/content_block_stop, terminal message_stop); handles mid-stream error / response.failed / response.cancelled, response.incomplete → stop_reason: max_tokens, deferred input tokens, and graceful early-close synthesis (#576, #585)
• Only OpenAI and Azure OpenAI backends serve /v1/responses; pairing a responses_only model with another backend type produces a 400 invalid_request_error before any upstream call (rejection fires on both /v1/chat/completions and /anthropic/v1/messages surfaces) (#577, #589)
• The first dispatch per (backend, model) pair logs at info level so operators can confirm Responses-API routing without enabling debug logs
• Anthropic Messages → Responses requests explicitly send store: false to avoid upstream side-effects (#589)
• 22 deterministic, in-process integration tests covering the {Anthropic, Chat} × {gpt-5.4-pro, gpt-5.2-pro} × {non-streaming, streaming} × {plain, tool-call, reasoning} matrix, mid-stream backend-failure negatives on both surfaces, and an upstream byte-fragmentation regression guard (#579, #588)
• Documented in docs/en/configuration/advanced.md (Responses-API-only Models section split into Models-marked-out-of-the-box, Marking-a-new-model, Dispatch-behavior, and Backend-type-constraint subsections), docs/en/architecture.md (Responses-API Routing data-flow diagram), and the docs/en/api.md Chat Completions and Anthropic Messages surface notes with a Transparent-Responses-API-routing subsection (#580, #587)

Fixed¶

• Chat Completions responses-only routing now rejects incompatible-only backend configs before upstream dispatch and chooses a compatible OpenAI/Azure Responses backend when available (#589)
• Chat assistant tool_calls[] are preserved as Responses function_call input items for stateless tool-result turns over /v1/chat/completions (#589)

v1.5.4 - 2026-04-25¶

Changed¶

• Refresh model-metadata.yaml for late-April 2026 frontier model releases (#572, #573)
• Add GPT-5.5 ($5/$30 per 1M, 1M context, knowledge cutoff 2025-12, omnimodal, leads Terminal-Bench 2.0 at 82.7%) and GPT-5.5 Pro ($30/$180 per 1M, Responses API only, deep reasoning) — released 2026-04-23
• Add DeepSeek V4 Pro (1.6T total / 49B active MoE, 1M context, 384K max output, three reasoning effort modes) and DeepSeek V4 Flash (284B total / 13B active MoE, 1M context, 384K max output) with deepseek-chat and deepseek-reasoner retained as deprecated aliases per official API docs — released 2026-04-24
• Add gpt-image-2 (token-billed instead of per-image: text $5/$30, image $8/$30 per 1M tokens; 1K/2K/4K resolution tiers; ~99% text accuracy in any language; built-in reasoning before generation; context-aware multi-turn editing; gpt-image-2-latest alias) — released 2026-04-21
• Add Claude Opus 4.7 ($5/$25 per 1M, 1M context, 128K max output, knowledge cutoff 2026-01, high-resolution image support up to 2576px / 3.75MP, new tokenizer with ~1.0–1.35× token usage vs prior models, new xhigh effort level) — released 2026-04-16
• Promote Gemini 3.1 series from preview to GA, retaining -preview suffix as alias for fallback compatibility (#573)
• gemini-3.1-pro-preview → gemini-3.1-pro (with gemini-3.1-pro-preview, gemini-3.1-pro-preview-customtools, and gemini-3.1-pro-latest aliases)
• gemini-3.1-flash-image-preview → gemini-3.1-flash-image (with gemini-3.1-flash-image-preview, nano-banana-2, and gemini-3.1-flash-image-latest aliases)
• gemini-3.1-flash-lite-preview → gemini-3.1-flash-lite (with gemini-3.1-flash-lite-preview and gemini-3.1-flash-lite-latest aliases)
• Updated gemini-3-flash-preview deprecation note to point to the new GA gemini-3.1-pro id

v1.5.3 - 2026-04-23¶

Added¶

• HuggingFace repo-prefix stripping as a new matching phase (phase 5) in src/models/pattern_matching.rs (#555)
• try_strip_hf_repo_prefix() validates a vendor/repo (or org/team/repo) prefix against a MAX_PREFIX_SEGMENTS = 3 bound, rejects empty segments (/repo, vendor/, vendor//repo), and rejects any ASCII whitespace before returning the residual
• Phase 5 re-enters phases 1-4 on the stripped residual with a structurally-enforced recursion depth of exactly 1 (the re-entry call clears the allow_prefix_strip gate), so prefix stripping composes with the existing layered suffix peel in a single lookup — the motivating case unsloth/Qwen3.6-35B-A3B-GGUF now resolves to qwen3.6-35b-a3b without any hand-registered alias
• Phase 5 runs before the wildcard phase; the blast-radius audit confirmed no *-bearing alias in model-metadata.yaml contains /, so the ordering change is behavior-neutral for existing routing
• Phase numbering in tracing output realigned to match the documented phase chain (previous code emitted phase = 7 for the namespace fallback while comments called it phase 6)
• 12 new unit tests covering standard HF form, composition with suffix peel, case-sensitive vendor, registered-alias precedence, unresolvable residual, three-segment form, segment-cap rejection, no-slash input, whitespace rejection, empty segments, re-entry bounding, and alias-phase precedence
• 9 new integration tests in tests/format_suffix_normalization_test.rs exercising the full RouterConfig / BackendConfig public API through phase 5
• Pipeline doc updated in docs/en/configuration/advanced.md (and Korean counterpart) with a new "HuggingFace repo-prefix stripping (phase 5)" section covering the composition semantics, security bounds, and out-of-scope list (hyphen prefixes, HF API discovery)

Changed¶

• Replaced the previous phase-6 namespace fallback with the new phase-5 HuggingFace prefix-strip layer. The previous phase was case-sensitive and did not compose with suffix peel; the new phase applies stricter input validation (segment cap, empty-segment rejection, whitespace rejection) but composes with phase 4's case-insensitive peel through the bounded re-entry. Pathological inputs above MAX_PREFIX_SEGMENTS (3) — such as provider/deep/nested/model — are now rejected by phase 5 rather than silently matched via recursive rsplit_once fallback (#555)
• Aliases currently classified as vendor-prefix in the #560 audit (e.g., Qwen/Qwen3.6-35B-A3B, MiniMaxAI/MiniMax-M2.5) are now peel-coverable-adjacent post-#555: phase 2 still wins on the explicit alias, but phase 5 + phase 4 together reach the same metadata. Retroactive removal is deferred to a follow-up audit per #555 design section 7

Fixed¶

• POST /anthropic/v1/messages now works when the selected backend is configured with a unix:// URL (#567)
• Native Anthropic backends and OpenAI-compatible backends both work over Unix sockets, for both non-streaming and streaming requests
• Socket paths containing spaces (e.g. macOS ~/Library/Application Support/...) are handled correctly
• Auth header selection (x-api-key for Anthropic backends, Authorization: Bearer for OpenAI-compatible backends) is correct on the Unix socket path
• anthropic-version header is added automatically for Anthropic backends on the Unix socket path, matching the HTTP path behavior

v1.5.2 - 2026-04-21¶

Added¶

• Regression tests locking down the transport-layer passthrough contract for llama.cpp and MLxcel backends (#562)
• New tests/llamacpp_passthrough_test.rs and tests/mlxcel_passthrough_test.rs covering all four passthrough call sites: direct backend execute_chat_completion, factory-built backend (BackendFactory -> LlamaCppBackend), proxy/backend.rs HTTP handler, and the streaming handler
• New test_mlxcel_factory_backend_passthrough_nonstandard_fields asserts that BackendFactory -> LlamaCppBackend::execute_chat_completion preserves non-standard fields byte-for-byte at transport time
• Anthropic input test (tests/anthropic_input_test.rs) extended with explicit passthrough coverage
• docs/en/architecture/backend-passthrough.md and its Korean counterpart docs/ko/architecture/backend-passthrough.md documenting the passthrough contract, the four guarded call sites, and the list of router-side transforms that run before transport (global_prompts, transform_payload_for_openai for o1/o3/gpt-5*, web_search injection) (#562, #563)
• docs/reports/alias-audit-2026-04.md classifying every alias in model-metadata.yaml into peel-redundant, peel-redundant-but-kept, and peel-independent categories, with an "aliases vs peel" policy section added to docs/en/configuration/advanced.md (and the Korean counterpart) explaining when to prefer each mechanism (#560)

Changed¶

• Narrowed the passthrough contract from an implied "byte-equivalent" global guarantee to a transport-layer scope — the router may still run global_prompts injection, o1/o3/gpt-5* payload transforms, and web_search tool injection before transport, but no provider-specific rewriting happens at the transport boundary (#563)
• Comment-only clarifications in src/http/streaming/handler.rs, src/infrastructure/backends/factory/backend_factory.rs, src/infrastructure/backends/llamacpp/backend.rs, and src/proxy/backend.rs
• Audited model-metadata.yaml aliases for peel-normalization redundancy: removed aliases that differ from the canonical ID only by suffixes already handled by the layered peel (-4bit, -q4_k_m, -fp8, -gguf, -mlx, -awq, etc.), while preserving aliases that encode canonical flavor variants (-qat, -instruct) or disambiguate parameter counts (#557)
• New tests/alias_audit_helper.rs and tests/format_suffix_normalization_test.rs enforce the peel-vs-alias boundary going forward

CI¶

• Target Ubuntu 26.04 LTS (Resolute) instead of 25.10 (Questing) in the Debian build workflow
• Fall back to createdAt when release publishedAt is null in debian/update-changelog.sh to prevent changelog regression when the latest release is still in draft

v1.5.1 - 2026-04-20¶

Added¶

• Built-in web_search tool for self-hosted LLM backends (#553)
• Router-level tool transparently injected into chat completion requests for vLLM, Ollama, llama.cpp, MLxcel, LM Studio, Continuum Router, and Generic backends
• Pluggable SearchProvider trait under src/services/search/ with SerperProvider implementation; Exa and Brave scaffolded behind the same trait
• Configurable inject_policy (auto/always/never) with per-backend overrides; commercial backends (OpenAI, Azure, Gemini, Anthropic) left untouched so their native web_search continues to flow through unchanged
• Bounded non-streaming tool-execution loop parses web_search tool calls, executes the provider, appends tool-role results, and re-invokes the backend up to max_tool_iterations rounds
• New BackendTypeConfig::is_self_hosted / is_commercial helpers covered by unit tests enforcing the commercial/self-hosted partition invariant
• API keys redacted in Debug output and never logged; hot-reload friendly WebSearchConfig with ${ENV} substitution
• Prometheus counters for tool calls, injections, and iteration-cap hits under src/metrics/web_search
• Layered quantization and format suffix normalization for model metadata lookup (#549)
• New layered_format_strip() in src/models/pattern_matching.rs iteratively peels allowlisted quantization/format/flavor tokens from the right side of a model ID, retrying exact-id/alias/date-suffix matches after each peel
• Token categories: BIT_WIDTH, GGUF_QUANT, FP_FORMAT, INT_FORMAT, LIBRARY, IMATRIX, UNSLOTH, CONTAINER, FLAVOR (all case-insensitive)
• Parameter-count suffixes preserved: -Nbit stripped as quantization; -Nb, -aNb, -eNb, -0.6b kept as parameter counts
• Canonical base IDs ending in allowlisted flavors (e.g. gemma-3-12b-qat) win via exact-id match before peel runs
• Normalization pipeline wired into find_matching_config, BackendConfig::get_model_metadata, RouterConfig::get_model_metadata, RouterConfig::get_thinking_pattern_config, resolve_model_tier (routing), and get_model_profile (admin)
• Model metadata for GLM 5.1, Qwen 3.6, and MiniMax M2.7 (#548)
• Teams release notification posted to Microsoft Teams via Power Automate webhook after build and Docker jobs

Changed¶

• Migrate documentation toolchain from MkDocs + Material for MkDocs to Zensical — reads mkdocs.yml natively and bundles required extensions

Fixed¶

• Security: Cap layered peel phase with MAX_MODEL_ID_LEN=256 and MAX_PEEL_ITERATIONS=8 to eliminate DoS via pathological model IDs (previously O(n²) allocation on inputs like -4bit-4bit-4bit-...)
• Security: Enforce 256-char model field length at /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/embeddings/sparse (parity with existing /v1/responses check)
• Consolidate 7-phase metadata matching pipeline into a single implementation (find_matching_config_slice) with thin adapters at each call site, eliminating drift between BackendConfig, Config::get_model_metadata, Config::get_thinking_pattern_config, and find_matching_config
• Replace cfg.to_ascii_lowercase() == peel with str::eq_ignore_ascii_case on the hot path (~4000 fewer per-request String allocations)
• Pin Pygments <2.20 to fix MkDocs build failure (superseded by Zensical migration)

CI¶

• Bump softprops/action-gh-release from 2 to 3 (#544)
• Bump actions/github-script from 8 to 9 (#545)
• Bump actions/upload-pages-artifact from 4 to 5 (#554)

Documentation¶

• Document suffix-order ambiguity (-qat-4bit vs -4bit-qat) and internal peel phase bounds in docs/en/configuration/advanced.md
• Add pattern_matching.rs to Model Aggregation Service module listing in docs/en/architecture.md with cross-reference to suffix normalization section
• New docs/en/web-search.md feature documentation; config.yaml.example extended with web_search section

v1.5.0 - 2026-04-11¶

Added¶

• Smart routing system with model tier & capability profile registry (#525, #531)
• Rule-based request classifier & smart routing policy engine (#526, #532)
• Load-aware dynamic tier adjustment (#527, #533)
• LLM-based request classifier with hybrid mode (#528, #534)
• Smart routing observability, admin API & documentation (#529, #535)
• Codex-compatible Responses API extensions (#536, #537)

Changed¶

• Upgrade core dependencies — axum 0.8, sha2 0.11, rand 0.10 (#523)
• Add Gemma 4 model family metadata (#538)

Fixed¶

• Complete smart routing integration gaps
• Increase DefaultTransformer PDF size limit from 20MB to 32MB (#542)

CI¶

• Bump actions/deploy-pages from 4 to 5 (#521)

Dependencies¶

• Bump the minor-and-patch dependency group with 4 updates (#539)

Documentation¶

• Add Codex-compatible Responses API gap analysis report

v1.4.5 - 2026-03-27¶

Fixed¶

• Return 400 error when file references are used without file service configured (#519)

Changed¶

• Add GLM-5-Turbo model metadata (#516)

Documentation¶

• Fix Korean anti-AI-slop violations in ko/ documentation
• Fix slop word and transition word in api.md

v1.4.4 - 2026-03-18¶

Fixed¶

• Fix Anthropic thinking failing for high/xhigh reasoning effort — budget_tokens (32768) exceeded default max_tokens (16384), causing API rejection (#514)
• Auto-adjust max_tokens to budget_tokens + 4096 when thinking is enabled and budget exceeds max

Changed¶

• Add GPT-5.4 model family: gpt-5.4, gpt-5.4-pro, gpt-5.4-mini, gpt-5.4-nano with 1M context window (#515)
• Update Gemini 3 series: add gemini-3.1-pro-preview, gemini-3-flash-preview, gemini-3.1-flash-lite-preview; mark gemini-3-pro-preview as deprecated
• Recognize Gemini 3 Flash and 3.1 Flash-Lite as thinking models for include_thoughts auto-injection
• Update Claude 4.6 models: context window to 1M (GA), fix Sonnet 4.6 max_output to 64K, correct knowledge cutoffs
• Update config examples and documentation with latest model names across 8 files

v1.4.3 - 2026-03-18¶

Fixed¶

• Fix Gemini thinking models (2.5 Pro, 3 Pro, etc.) not returning reasoning_content in streaming responses through the router (#513)
• Replaced transform_payload_for_gemini() with transform_request_gemini() across all three Gemini streaming paths to ensure include_thoughts: true auto-injection

v1.4.2 - 2026-03-17¶

Changed¶

• Change mid-stream fallback default to enabled for improved streaming reliability (#504)
• Breaking: Mid-stream fallback is now enabled by default; set mid_stream_fallback.enabled: false to restore previous behavior

Documentation¶

• Add failover latency tuning guide for optimizing fallback behavior

v1.4.1 - 2026-03-17¶

Added¶

• Mid-stream fallback for streaming inference (#497) — when a backend fails mid-stream during SSE streaming, the router transparently retries with a fallback backend

Changed¶

• Decouple pre-stream fallback from mid-stream fallback (#500) — each can now be independently enabled/disabled
• Bump dependency versions to latest major releases

Fixed¶

• Fix streaming config changes not detected in hot reload system (#503)
• Fix mid-stream connection errors leaking to client during fallback (#502)
• Remove unused config crate dependency

CI¶

• Bump dorny/paths-filter from 3 to 4 (#493)
• Bump actions/create-github-app-token from 2 to 3 (#494)

v1.4.0 - 2026-03-14¶

Added¶

• Prefix-aware routing: PrefixAwareHash selection strategy with Consistent Hash with Bounded Loads (CHWBL) (#455, #457, #461)
• Response caching: SHA256-based cache key computation with streaming response buffering and post-completion caching (#456, #459, #462)
• Multi-tier CacheStore: in-memory backend (#466), Redis/Valkey backend with connection pooling (#467), and S3-backed tiered L1/L2 cache (#483)
• KV cache index: shared data structure (#470), KV event consumer for vLLM backend streams (#471), prefix overlap scoring integrated into backend selection (#473), configuration/metrics/admin endpoints (#474)
• Tiered KV cache with storage-tier awareness (GPU hot / external warm) (#484)
• Disaggregated prefill/decode orchestration with external KV tensor transfer (#485)
• Anthropic cache_control breakpoint auto-injection (#460)
• Multimodal embedding support for Gemini Embedding 2 (#492)
• Shared cache configuration and operational metrics (#468)
• 30 new models added to model-metadata.yaml (#472)

Changed¶

• Rename VAST-specific identifiers to generic S3/external storage names (#490) — update configuration files if using VAST-specific field names

Fixed¶

• Make RequestExecutor transport-aware for Unix socket paths with spaces (#488)
• Replace relative source tree links with GitHub URLs in docs

CI¶

• Bump docker/setup-qemu-action from 3 to 4 (#428)
• Bump docker/metadata-action from 5 to 6 (#426)
• Bump docker/setup-buildx-action from 3 to 4 (#429)
• Bump docker/build-push-action from 6 to 7 (#430)
• Bump docker/login-action from 3 to 4 (#427)

Documentation¶

• Comprehensive KV cache feature documentation, benchmarks, and config examples (#477)
• VAST Data connection guide and integration examples (#486)
• Sync Korean documentation with English counterparts
• Split monolithic configuration.md into 6 smaller files

v1.3.0 - 2026-03-12¶

Added¶

• Agent Communication Protocol (ACP) support with JSON-RPC 2.0 protocol layer and stdio transport (#414, #420)
• ACP session management with protocol lifecycle, initialize/shutdown handshake (#415, #421)
• ACP-to-LLM inference pipeline with streaming support (#416, #422)
• ACP tool call reporting and permission delegation (#417, #423)
• MCP-over-ACP bridge for MCP server tunneling (#418, #424)
• ACP agent registry with metadata and configuration support (#419, #425)
• ACP integration tests for protocol lifecycle and session management

Fixed¶

• Resolve clippy field_reassign_with_default warnings in ACP integration tests

CI¶

• Bump actions/upload-artifact from 6 to 7 (#398)

Documentation¶

• ACP architecture documentation with MkDocs integration
• ACP practical usage guide with IDE integration examples
• KV cache integration plan for router-level caching strategies

v1.2.1 - 2026-03-07¶

Added¶

• MLxcel backend type support for MLX-based model serving (#412, #413) — fully API-compatible with llama-server, reusing the same backend implementation for health checks, model discovery, and proxying

v1.2.0 - 2026-03-06¶

Added¶

• Admin Statistics API with comprehensive request-level statistics collection and reporting (#409)
• Endpoints: GET /admin/stats, GET /admin/stats/models, GET /admin/stats/backends, POST /admin/stats/reset
• Time-windowed queries, token usage tracking, latency percentiles (p50, p95, p99)
• Statistics persistence with configurable snapshot path, interval, and staleness checks (#410, #411)
• Atomic writes, restore on startup, final snapshot on graceful shutdown

Documentation¶

• Add admin stats and persistence to configuration guide
• Add post-refactoring benchmark report for v1.1.0 (#407)

v1.1.1 - 2026-03-04¶

Added¶

• Embeddable library crate (Phase 1) — use continuum-router as a Rust dependency (#394)
• Type-safe config builders for programmatic library usage (#400)
• Cargo feature flags for optional library dependencies (#399)
• Persistent storage for runtime API keys (#405)
• New LLM model metadata entries (#403)

Fixed¶

• Fix Gemini-specific transforms incorrectly applied in Anthropic handler (#404)

v1.1.0 - 2026-03-01¶

Added¶

• Embedded WebUI for configuration management and API key administration (#388)
• Windows AF_UNIX socket support via socket2 crate (#390)
• Nano Banana 2 (Gemini Image Generation) support

Fixed¶

• Resolve compilation error in ClientAddr::is_unix for tuple variant matching
• Resolve Windows AF_UNIX socket accept failure and config validation
• Accept Windows absolute paths in Unix socket config validation (#393)
• Resolve Windows compilation errors in Unix socket tests and transport parsing (#392)

v1.0.0 - 2026-02-19¶

Added¶

• Continuum Router federation — router-to-router chaining as a new backend type (#385)
• LM Studio as a dedicated backend type (#381)
• Anthropic adaptive thinking effort parameter (output_config.effort) (#384)
• Adaptive thinking and auto reasoning effort level across backends (#378)
• Cohere/Jina-compatible rerank and sparse embedding endpoints (#374)
• BGE-M3 and multilingual embedding model support (#373)
• Claude Opus 4.6 model metadata
• Qwen3-Coder-Next, Qwen3-VL-30B/8B model metadata