Changelog¶

All notable changes to Continuum Router are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

v1.17.0 - 2026-08-03¶

Execute hub-governed guardrail policy for the first time, enforce Hub cost-center V2 monthly limits locally, bound file resolution and Hub envelope decoding, make the per-key and per-client rate-limit dimensions fire on live traffic, and stream Files API transfers instead of buffering them.

Security¶

• Bind Responses file and session authorization to the authenticated API-key principal instead of the client-supplied X-User-ID header (#1244). File resolution, stored-session creation, previous_response_id replay, GET, and DELETE all derive the requester from AuthContext::user_id(), X-User-ID is filtered before upstream dispatch, and the spoofable cache and session fallback namespace is gone. Owned files and sessions on both the Responses and Anthropic paths now fail closed when no authenticated requester is present, and a Responses AccessDenied on file resolution answers 403 instead of forwarding the raw file_id. Deployments that partitioned Responses state by X-User-ID must move to per-key partitioning; the English and Korean guides carry the migration steps.
• Redact backend URLs and other reqwest-derived transport detail from client-visible error bodies (#1235). A shared client_safe_reqwest_error helper consumes reqwest::Error::without_url() at the response boundary, covering the Responses, Messages, count_tokens, image edit, multimodal embedding, and responses-only passthrough builders that previously interpolated the raw display text; the responses-only passthrough mapper now matches the Responses mapper and returns a static backend connection message. HTTP status, error type, error code, circuit-breaker accounting, and the full operator logs are unchanged.
• Escape and cap the untrusted log values #1161 left out of scope (#1179). Client-supplied multipart field names on POST /v1/files, the image edit and variation parser, and the admin Files API upload reached message through format-string interpolation, where EscapeGuard passes \n and \r through and nothing bounded the length. Backend-supplied values in the Anthropic subtree that #1161 did not touch, including the /v1/responses parse-failure body head and the full upstream error body during streaming, get the same escape-and-cap treatment.
• Cap the client-supplied values echoed into the disabled-thinking rejection body (#1183). disabled_thinking_effort_error_message interpolated the raw effort twice and the raw model id once with no length bound, and all three ingresses turn that message into a 400 body. Neither gate bounds its input first: effort_exceeds_high compares a trimmed, lowercased copy while interpolating the original, and claude_family_version inspects only the leading tokens. The cap now lives in the shared builder, so every ingress inherits it.
• Bound Responses file_id resolution per request (#1180, closes #1174), the same defect class #1169 fixed on the Anthropic path. The resolver enforced only the per-file 10 MiB MAX_INJECTION_SIZE, so any number of individually small references resolved with no aggregate ceiling, and each one holds its own base64 copy resident until dispatch. A count pre-pass now rejects above 100 references before the first metadata lookup, an aggregate byte budget is charged per reference from metadata.bytes before the content read (so a repeated reference is charged again), and the whole resolution phase carries the 30 second timeout the other two paths already had. A new src/core/files/limits.rs holds the bounds all three paths agree on, replacing two duplicated declarations and the comment asking that they be kept in step.
• Bound chat-completions local file_id resolution with the same 32 MiB aggregate raw-byte budget used by Anthropic Messages and Responses (#1240). The chat path now charges every reference from metadata.bytes before reading content, including repeated references to the same file, and returns a 400 invalid_request_error when one request exceeds the budget instead of forwarding partially resolved payloads. Chat-completions also now applies the 10 MiB per-file inline ceiling to PDFs as well as images; larger PDFs remain uploadable under files.max_file_size but are no longer expanded into a chat-completions JSON body.
• Bound the Hub policy envelope decode, which previously had no size cap (#1193). sync_policy now decodes through the same bounded reader as config sync under a derived ~152.6 MiB ceiling (MAX_POLICY_SYNC_RESPONSE_BYTES), written down term by term against the Hub's own distribution budgets (a documented 100,000-entry key-table allowance for the one member the Hub does not cap per org, 1,024 tiers with the 4 MiB aggregate request_params budget, the 256 KiB guardrail canonical budget, 5,000 cost-center assignments plus 1,024 capped centers, and explicit allowances for the members the Hub does not bound), sized so a legitimate tenant at every ceiling at once still decodes, which a new ceiling-scale round-trip test locks in. An over-limit refusal is loud and typed rather than a silent parse-class failure: a distinct HubError::ResponseTooLarge carrying the limit and the observed size, an error! log naming both (never body content), and a new control_plane_policy_sync_failure_total{reason="envelope_too_large"|"other"} counter separating "envelope too large" from an ordinary sync failure, because a persistent refusal leaves a restarted router permanently fail-open while an ordinary failure heals on its own. Both reason series are instantiated at zero at registration so a fleet alert can see them before any refusal, and the refusal count with its last-occurrence time is surfaced as policy_sync.oversize_refusals / last_oversize_refusal_ms on GET /admin/control-plane/status, where a nonzero count beside a null last_sync_ms is the cold-start fail-open signature. The policy store preserves the last-known-good snapshot across a refusal and never widens enforcement; a cold-start refusal leaves the store exactly fail-open. The WebSocket policy stream now sets its message and frame ceilings to the same constant, replacing tungstenite's 64 MiB default, which sat below the poll bound and would have refused on one path a legitimate large envelope the other path accepts. The remaining unbounded Hub response decodes (enroll, push_usage, push_probes) each get their own derived bound with the same typed refusal, the usage ack bound derived from the 10,000-record MAX_USAGE_BATCH_MAX_RECORDS clamp rather than the 500-record default, and an unreadable oversize usage ack is terminal for its already-processed batch rather than retried forever; heartbeat, report_config_apply_result, and push_batch_status decode no response body and are unchanged.

Added¶

• Add local enforcement of Hub cost-center monthly limits under the corrected V2 convergence contract (#1162; hub issues #688, #735, #736, epic #646). The vendored continuum-protocol crate syncs the complete cost-center surface from the hub at rev 98bb8ac, byte-for-byte with the hub's committed fixtures pinned by a new wire test, so the V1 member (cost_centers, cost_center_policy_digest) stays parse-compatible while remaining enforcement-inert and unacknowledged, and the V2 member (cost_centers_v2) becomes the enforced contract: a per-router stable allocation statement (Hub-pre-resolved key_id -> cost_center_id ownership, router fallback, provider/model and stable-backend fallbacks, per-center caps) plus the persisted dynamic budget snapshot it references (consumed-to-date counters, Hub-authored per-dimension verdicts, and the per-router included_router_usage_seq watermark). The statement is one atomic apply unit validated for bounds, duplicates, references, the router-scoped digest, the deterministic cursor, snapshot staleness, and watermark possibility, refused whole with a typed code (invalid_cost_center_policy, digest_mismatch, stale_budget_snapshot, invalid_usage_watermark) while the last-known-good unit stays enforced, and acknowledged exactly (stable cursor/digest plus dynamic budget cursor) on both the applied and rejected side of heartbeat status. Enforcement is local and the hub is never in the request path: the admission gates engage only on request paths that can produce a usage record the hub will meter; the key and router tiers are refused and atomically reserved (one request plus a deterministic token allowance: a bounded byte-derived prompt estimate plus the request's stated output maximum clamped to the tier's request_params ceiling and a hard bound) at the admission middleware, with the reservation riding the response body to the last streamed frame and counting toward every other admission so concurrent at-limit requests cannot all pass, while a single request is refused only at the cap and never on nothing but its own estimate; the lower tiers (stable backend, then provider/model, in the hub's own precedence) are refused at admission when every candidate serving backend for the effective post-rewrite target allocates to a cost center that is itself over cap, the condition under which the pre-dispatch decision cannot diverge from the identity that serves and is accounted, since every identity the request could serve under is exhausted (the candidates need not resolve to the same cost center: requiring that was sufficient but not necessary, and it let a client keep serving past a cap by naming a model whose backends spread across two exhausted centers); and accounting for every tier settles exactly once at the usage seam, where the single usage-loop consumer stamps the persisted monotonic router_usage_seq on each record entering the push buffer and ledgers its metered usage under the cost center computed from the final record's own identities, the same inputs the hub allocates from at ingest. Accepting a newer snapshot discards exactly the ledger entries at or below the hub-included watermark (unreflected usage is never reset, included usage is never double-counted); a hub watermark naming sequences a fresh runtime never assigned (a lost sidecar beside a surviving credential) recovers by a durable forward jump of the sequence space instead of wedging into permanent rejection; a populated statement arriving without a budget block is the documented budget-unavailable state (hub #750) and drops the held counters and hub-authored verdicts rather than applying them under new caps; a record permanently dropped from the push buffer, an unsplittable oversize batch, or a hub per-record rejection retires its ledger entry; and the sequence counter, last-known-good unit, applied watermark, and unreflected deltas persist in an owner-only sidecar next to control_plane.state_file (<state_file>.cost-center.json, 0600, atomic rename) so restart resumes the sequence space instead of restarting it. Sell spend is Hub-authored only: the router never prices, decrements, or infers sell spend locally, and the sell_spend_status verdict is the only sell-spend input (exhausted refuses; an unknown verdict from a newer hub never does). cost_center_limits_v2 is advertised only while the complete contract is live (sidecar attached, identity bound, sequencing and enforcement wired); the legacy cost_center_limits_v1 string is never advertised, and the documented overshoot stays bounded rather than claimed away, with four stated terms: the snapshot cadence times the fleet size, the reservation-estimate error, one allowance per concurrently admitted request, and the in-process handoff between a response finishing and its record entering the push buffer. Documented in the English and Korean control-plane architecture references.
• Add hub-managed guardrail policy, so a Continuum Hub can govern content-safety enforcement across a fleet while every router keeps ownership of execution (#1094, closing #1105, #1106, #1107, #1108, #1109, #1110, #1111, #1112, and #1122). v1.17.0 is the first release that executes hub-governed guardrail policy, which is the version the hub compatibility matrix should pin (lablup/continuum-hub#603); an older router ignores the field, never advertises the guardrail_policy_v1 capability, and is derived as unsupported rather than left pending forever. Governance travels in the existing policy envelope under control_plane.policy.enabled and is a closed, typed, integer-only surface: mode, category_thresholds (unsigned microunits, 1000000 is exactly 1.0), stages, inspect_reasoning, and block_behavior, with at most 64 route overrides keyed by resolved model id. Route scope covers mode and category_thresholds; a route-scoped stages, inspect_reasoning, or block_behavior has no slot in the local route schema and is declined with a warning naming the route and the knobs rather than folded into the global setting, because folding would widen a route-scoped decision to every route. Deployment wiring is never sourced from the hub: the providers list itself, provider endpoint / backend: / api_key_env, per-provider and global timeout_ms and on_error, every streaming_* knob, bypass_api_keys, the audit block, per-route enabled and providers subsets, and the global and per-route allow / deny match lists stay local and sovereign, and the policy type has no free-form payload field anywhere, so no prompt, completion, matched span, or redacted value can travel on it in either direction. Composition is strictest-wins per knob (enforce beats monitor from either side, the lower threshold floor wins per category and a stated category with no local counterpart is added, stages union into every provider row, inspect_reasoning ORs, and a stated block_behavior wins as a governance choice rather than a strictness ordering), recomputed whenever either input changes and applied atomically, so a local edit never drops the hub layer, a hub update never freezes local edits out, and a request is always evaluated against one whole policy generation. Delivery is tri-state rather than binary: an absent envelope member keeps the last policy, a stated policy replaces it, and an explicit clear returns the router to local-only configuration and is acknowledged with the canonical digest of the cleared policy rather than reported as absent, so a cleared tenant converges instead of resting at pending or stale. A router that started with guardrails.enabled: false, or with no guardrails: block at all, materializes its guardrail service at runtime when an enforceable policy arrives, through the same factory path startup uses and with local provider wiring only. A policy demanding enforcement this deployment cannot serve, meaning nothing that can gate at all or a demanded stage no runnable provider covers, is rejected whole with the typed code guardrails_unavailable and is never reported as active, so a Fleet view cannot show enforcement that is not happening; a body that fails validation is refused with invalid_guardrail_policy, which keeps "this router is older than the policy" distinguishable from the shared digest_mismatch. A refusal clears without hub involvement: the wire layer keeps the policy applied and re-evaluates on every local configuration change, so adding the missing provider row or deny rule makes the same policy take effect on the next reload. guardrail_policy_v1 is advertised only once a live guardrail-policy reconciler has attached, from the single capability list every status-writing path shares, because the capability is a promise about execution and not a build flag. The guardrail and tier tracks stay independent in both directions. Documented in the English and Korean guardrail guides (governance scope, local sovereignty, the composition table with a worked example, tri-state delivery, refusal codes, and the hub status derivation), the control-plane architecture reference, and the configuration-assistant skill doc (schema 1.30.0 to 1.31.0).
• Report the serving backend's stable configuration id on control-plane usage records and heartbeat inventory, so a backend can be an allocatable cost center (#1153, hub lablup/continuum-hub#686). The new optional backends[].backend_id is entirely operator-owned and nothing derives it: name is a display label an operator may rename at any hot reload, and a (provider, model) pair is not a backend because several backends can serve one pair, so a derived value would silently merge two distinct cost owners that share a pair and silently split one cost owner across a rename, both invisible once the usage has been ingested. A backend without an id reports none and its usage falls through to provider/model attribution exactly as before, which is a normal state rather than an error. Values are 1 to 128 characters from the delimiter-safe ASCII set [A-Za-z0-9._-], the same alphabet fallback.fallback_chains documents, because the value travels as an opaque key the hub matches byte for byte and never normalizes; whitespace, /, :, ,, control characters, and non-ASCII including bidi overrides are refused at the source rather than left to change which cost center a request charges. Uniqueness is required within one router and only within one router, since the hub keys membership on (authenticated_router_id, backend_id), so two routers may reuse one local spelling. The bound runs on the shared ingestion gate in infrastructure::config::validator, not only in the Validate impl, because the real load path validates sections explicitly and never invokes Config::validate(); rejecting the whole candidate there is also what keeps a hot reload atomic, leaving the running router on its last known good rather than a half-applied identity map. The /admin/backends handler family does not reach that gate at all (it publishes straight to the hot-reload watch channel through propagate_config_change), so it enforces the same bound at its own boundary, and POST /admin/config/validate checks it too so the pre-save dry run cannot report a green that the following write rejects. The id is resolved at each serving seam against the config view the selection actually used, and carried on the usage event rather than looked up by backend name when the record is pushed, because a hot reload in that window renames the backend and a later lookup would resolve to nothing and drop the attribution for usage already served: a fallback hop overwrites it exactly as it overwrites the TTFT origin so a failed-over request is charged to the backend that produced the response, the mid-stream relay resolves each hop from the snapshot the request was routed against, a provider batch carries the id captured at submit so a completion hours later still names the backend that executed it, /v1/responses streaming resolves at context construction because it has no dispatch seam, and a local cache hit dispatched no backend and reports nothing. The display name is never substituted when an id is absent. On the wire UsageRecord.backend_id is a verbatim copy of the upstream field and BackendInfo.backend_id is additive ahead of an upstream re-sync, since the hub holds no backend inventory and heartbeat is the only way an operator can discover which ids exist to assign; both are optional with serde defaults, so a router with no ids configured serializes byte-identically to the previous wire and PROTOCOL_VERSION stays 0. stable_backend_identity_v1 is advertised unconditionally because, unlike guardrail policy, there is no separate executor that may or may not attach: config validation, inventory, and the usage taps are compiled and wired as one unit, so the capability describes the implementation rather than the data and a router with no ids configured still advertises it and reports none, which is the state the hub must be able to tell apart from a router that cannot report at all. Documented in the English and Korean configuration and architecture guides, config.yaml.example, and the configuration-assistant skill doc (schema 1.32.0 to 1.33.0).
• Report metadata-only guardrail counters in the heartbeat inventory, so the hub can see fleet-wide guardrail activity without scraping every router (#1120). RouterInventory.guardrails carries checks_total, blocks_total, blocks_by_category, verdicts by (stage, mode, result), stream_buffer_cap_trips by (strategy, outcome), and a counters_since_ms stamped from the same process-start value as SupplySummary, so both counter families reset together. The counters come from an in-process tracker written at the same seams that emit the Prometheus families rather than from the registry itself, because metrics is optional and reporting zeros from a build without it would tell the hub that an actively blocking guardrail is idle. Every label space is closed, so a third-party moderation provider cannot expand the hub's stored key space or carry provider-authored text into it: any category outside the known vocabulary folds into other, and an unrecognized stage, mode, result, strategy, or outcome folds onto an unknown slot instead of being dropped. The block is reported whenever a guardrail service exists, even before the first check, and omitted entirely when the router runs none, so absent means "guardrails are off" while zeros mean "on and quiet". Counts and bounded ids only: no prompt text, no completion text, no matched span, no matched rule, and no redacted value.
• Add an optional in-flight request ceiling, so the router's per-request memory budgets finally imply a per-process bound (#1173). Every memory bound the router enforces is per request: the 10 MiB GLOBAL_BODY_LIMIT, files.max_file_size (512 MiB by default, buffered whole in memory), and the 32 MiB aggregate file-resolution budget added by #1169 and #1174. Worst-case resident memory is the product of those and the number of requests in flight, and nothing bounded the second factor. A rate limit does not supply it: it bounds arrivals per unit time, not residency, and by Little's law the steady-state in-flight count is the arrival rate times the mean duration, so per_client: 10 requests per second against completions averaging 30 seconds implies roughly 300 concurrent from one client entirely within policy, which is about 12.6 GiB at the file-resolution ceiling. The shipped default configures no rate limiting at all, in which case the ceiling is whatever a client can open. server.max_concurrent_requests is unset by default, so no existing deployment starts shedding on upgrade; set it and the router admits that many requests at once and answers the rest with 503, a Retry-After, and a JSON body naming the limit. It sheds rather than queues, because queueing trades an out-of-memory kill for unbounded latency while still holding the memory of everything queued. Implemented as a tokio::sync::Semaphore acquired with try_acquire_owned rather than tower's ConcurrencyLimit / LoadShed, which would have queued by default, needed a Buffer with a background task, given no control over the refusal body or the retry hint, and pulled new tower features into the embed graph that deliberately excludes iOS-hostile dependencies; this adds no dependency at all. The permit rides the response body, so a streaming completion holds it until the last frame is delivered, which means a ceiling tuned for short non-streaming traffic will starve streaming traffic and should be sized against the longest requests. The layer is mounted application-wide but inside both CORS and the rate limiter, so a shed 503 is readable in a browser and a request the limiter would refuse anyway never occupies a permit; it covers the separately-nested /v1/files group, which carries the largest per-request term of all, and exempts /health, /healthz, and the configured metrics.path, because shedding a liveness probe is how a correctly-degrading replica gets restarted mid-overload. SupplySummary::max_concurrency and shed_requests now report the real ceiling and the real shed count instead of promising the hub that no ceiling exists; an unconfigured ceiling still reports as absent rather than 0, since 0 would mean the router admits nothing. Memory-aware admission (charging a request its declared cost rather than counting requests uniformly) was evaluated and deferred: it needs a per-request cost estimate available before the body is read, and the only pre-read signal is Content-Length, which is absent under chunked encoding and does not predict cost anyway, since the #1168 vector was a 57 KiB body carrying 800 file references. The capacity arithmetic, the per-request terms with their config keys, and the interaction with rate limiting are documented for operators in the English and Korean deployment and performance guides, which also raise the Kubernetes example's 512Mi memory limit that a single default-size upload already exceeds.
• Add metadata for five models that shipped without entries, so /v1/models serves display names, pricing, capabilities, and limits for them instead of nothing (#1091, closes #1090): Claude Opus 5 (claude-opus-5, claude-opus-5-latest, which also joins the native Anthropic backend's built-in supported-models list), Kimi K3, Qwen 3.7-Max, Qwen 3.8-Max Preview, and Solar Open 2. Opus 5 needed no capability-gating code, since claude_family_version() already parses the id to (5, 0); new unit tests pin extended thinking, adaptive thinking, sampling-parameter handling, max effort, mid-conversation system prompts, and fast-mode eligibility so a refactor cannot silently regress it the way a substring gate once regressed Sonnet 5 (#854). Qwen 3.8-Max Preview keeps zero rates because preview access is credit-based with no published per-token rate, and Kimi K3 and Solar Open 2 omit knowledge_cutoff because neither vendor published one. solar-pro-2 also gains the tool-use capability and the Japanese language support its own summary already implied.
• Implement the files.retention_days startup sweep, which was configurable and documented but never ran (#1234, closes #1229). A nonzero value now deletes stored files whose metadata sidecar created_at is at least the configured age, and retention_days: 0 still keeps files forever. FileService::sweep_retained_files deletes metadata first, treats a missing content object as benign, restores the metadata if the content delete fails, logs each sweep, and reports file_retention_deletes_total. The read race boundary, the startup-sweep semantics, and the config examples are documented in the English and Korean guides, the Admin configuration schema, and the configuration-assistant skill.
• Complete BackendConfigBuilder coverage for BackendConfig (#1208). Setters were added for backend_id, internal, role, region, endpoint_type, auth, org_id, model_configs, retry_override, health_check, anthropic_auto_cache_control, anthropic_fast_mode, and external_storage; timeout and max_retries now mutate the same retry_override field as the explicit setter; backend_id format is validated in BackendConfigBuilder::build and uniqueness in ConfigBuilder::build. An exhaustive no-rest-pattern field inventory guard fails compilation when a new BackendConfig field is added without a builder decision.

Changed¶

• Reject a disabled thinking config paired with an effort above high on Claude Opus 5, at the router's own boundary (#1103). Anthropic accepts thinking: {"type": "disabled"} on Opus 5 only while output_config.effort stays at high or below and returns HTTP 400 at xhigh or max, so the invalid pair was previously forwarded and the client learned about it after a full round trip. The resolution is fail-fast rather than a rewrite: clamping the effort or dropping the thinking config would both silently rewrite an explicit client instruction, and disabled thinking on Opus 5 has two documented failure modes of its own (tool calls emitted as plain text instead of structured tool_use blocks, and <thinking> tags leaking into the visible response), so neither is a neutral fallback. The 400 states the constraint and names both remedies. The guard runs at every ingress: the shared OpenAI-to-Anthropic transform, both Anthropic-shape streaming entry points (TCP and Unix socket), the typed /anthropic/v1/messages handler after alias resolution and request_params application, and the Responses converter's independently built payload.
• Remove the dead Backend::max_concurrent_requests() trait hook and every production and test-double implementation (#1224). Nothing read it, so reporting the old hardcoded provider constants to the hub would have published an unenforced fiction. Per-backend control-plane inventory stays None, with the comment now pointing at the real router-level server.max_concurrent_requests admission cap added in #1173.
• Stream Files API uploads and downloads instead of materializing whole files in memory, and decide download authorization before any content is read (#1145). FileStorageBackend now has store_stream and retrieve_stream as its primitives, with the slice-based store and retrieve kept as provided wrappers, so peak resident memory per transfer is a 64KB chunk buffer rather than roughly 1.5x the file size. LocalFileStorage streams each upload into a per-process, per-upload temporary file and fsyncs plus renames it into place only after every check has passed, so a reader never observes a partially written .bin, and an upload that is rejected, errors, or is cancelled mid-flight leaves neither a partial file nor a leaked temporary. Content validation became incremental while reaching the same verdict as before: the first 13 bytes cover the image and executable magic-byte checks, and a new non-allocating Utf8Validator carries a partial multi-byte sequence across chunk boundaries, so a character split between two network chunks is judged exactly as it would be in a single buffer and a body ending mid-sequence is still rejected. GET /v1/files/{id}/content and GET /admin/files/{id}/content now resolve authorization from metadata alone before opening the file, closing a path where a caller who knew a file id but did not own it could force a full-size read and allocation and still receive a 403, and both then stream the response body instead of buffering it. The 413 semantics, the exact File too large: exceeds maximum N bytes message, and every other observable Files API behavior are unchanged. Because the temporary now exists for the whole transfer, an abrupt death (SIGKILL, OOM kill, node eviction, power loss) can leave one behind where no in-process cleanup can run, and nothing else in the product could see it: the orphan scanner matches only .bin and .meta.json, while the user-data retention sweep uses metadata sidecar created_at and ignores temporary upload dotfiles. Starting the router therefore reclaims every temporary older than 24 hours, ungated by cleanup_orphans_on_startup because an aged temporary is unambiguously garbage rather than possibly-recoverable data, and FileService::detect_orphans now reports the count so the leak is observable before the next restart. The 24 hour threshold is what keeps the sweep from deleting an upload another process may still be streaming; a temporary's mtime advances with every write, so a live transfer keeps refreshing itself out of range. commit also fsyncs the parent directory, so the rename is durable and not just the bytes it points at.

Alongside this, the upload routes' outer body limits are derived from files.max_file_size, clamped to the documented 1KB to 5GB range, instead of a hardcoded 512MB that silently capped anything larger, and a value outside that range is now reported as a startup warning on the real config-load path (warning rather than a hard error, because max_file_size: 0 is accepted today and a fatal check would stop a working router from starting; the clamp is what protects the routes). Relatedly, a body the transport-level cap truncated now answers 413 file_too_large rather than 400 invalid_request: both handlers classified every multipart failure as malformed input, which was academic while the cap was a fixed 512MB and became reachable once it derived from max_file_size. The previously unwired file_uploads_total, file_upload_size_bytes, file_upload_duration_seconds, file_downloads_total, and file_download_duration_seconds metrics are registered and emitted: an upload records its outcome from the session's drop, which is the only point that also sees an oversize rejection, a missing purpose part, or a client that disconnects mid-body, and a download is counted a success only when its body reaches the end without error rather than by virtue of having started. A startup warning fires when files.max_file_size exceeds a quarter of the cgroup v2 memory limit, staying silent when there is no limit, when it reads max, and on platforms without cgroups.

Fixed¶

• Stop two of the four self-hosted classifier templates from reading unrecognized guard-model output as a clean allow (#1203, follows PR #1182). verdict_llama_guard and verdict_yes_no took the first token of the model's answer and returned GuardrailVerdict::Allow for anything that was not their single positive token (unsafe, yes), so an error string, a refusal to comply, a chat-template artifact, a generation truncated by the 32-token cap, or an answer in an unexpected language was served as safe. It was silent: the parser produced an ordinary Ok(Allow), ProviderOutcome::failure stayed None, and under mode: enforce the Prometheus error family, the heartbeat's errors_total / fail_open_total / fail_closed_total, and the audit record all described an inspected, clean request. The lenient parsers were also the default path: an omitted template, an unrecognized template, granite_guardian, and shieldgemma all resolve to verdict_yes_no, so a single typo pointed a Llama Guard deployment at a parser where unsafe is not yes and every check allowed. Both parsers now recognize both poles explicitly, safe alongside unsafe and no alongside yes, and treat only genuinely unrecognized output as a check error, which is what Qwen3Guard and custom_classifier already did; the four templates now agree on the rule. Behavior change to check before upgrading: a deployment whose guard model emits output the configured template cannot read no longer passes silently. It now fails per the provider's effective on_error, which under the default fail_open means unchanged traffic with new error counts on guardrail_errors_total and the heartbeat's errors_total / fail_open_total, and under fail_closed means refused traffic. With api_format: completion the classified text is the entire prompt with no instruction wrapper, so a completion-served guard model that merely continues that text is the configuration most likely to see this, and it now fails loudly where it used to allow silently.

To avoid narrowing what a working deployment accepts, the verdict token is normalized deliberately rather than by fallthrough: surrounding whitespace, ASCII case, punctuation or markdown emphasis wrapping the token (**unsafe**, Unsafe:, Yes,), and a completed <think>...</think> reasoning block ahead of the verdict are all accepted. Whitespace and case were already accepted; the other two used to read as silent allows even when the model had answered unsafe, so they are now read as the positive verdicts they are. A reasoning block truncated before it closed carries no verdict and fails the check, which is the likely outcome at the 32-token cap these templates use. Nothing past the first token is scanned for a verdict word, because guard models restate both poles inside their prose ("this is not safe") and a scan would read an explanation or a refusal as a verdict, which is the failure this change exists to remove. The two allows that come from a real classification are untouched: an unsafe verdict whose reported hazard codes are all excluded by categories, and a positive verdict suppressed by a category_thresholds entry.

A second behavior change, in configuration validation. An unrecognized options.template on a self_hosted_classifier (or classifier) provider is now refused by Config::validate, where it previously fell back to granite_guardian with only a startup warn!. That fallback is what turned a typo such as llama-guard or shield_gemma into a total silent bypass, and because the failure direction was always permissive it could never produce the false block an operator would notice. A configuration carrying an unrecognized template value therefore stops loading, and continuum-router config validate reports it; fix the value or drop the key, since an omitted template is still valid and still means granite_guardian. The provider-side fallback is kept as defense in depth for construction paths that did not validate. The accepted values and the verdict-reading rules are documented in the English and Korean guardrails guides.

• Stop leaking the router's filesystem layout into client-visible error bodies on the Files API and the two file_id resolution paths (#1191). FileError::Storage(_) carries a message the router writes itself at roughly twenty sites in src/services/files/storage.rs and src/services/files/metadata.rs, every one of them a format! that interpolates path.display(), and map_file_error handed that string straight to the client as the message of a 500. A download of a file whose backing object had become unreadable answered with the absolute path under files.storage_path, the two-level sharding subdirectory, the stored .bin filename, and the underlying std::io::Error; a failed upload answered with the temporary filename, which temp_file_name builds as .{id}.{pid}.{sequence}.tmp and which therefore also disclosed the router's process id (1 under a container, itself a deployment tell) and the per-process count of uploads begun since start. Under the default files.auth.method: api_key the audience is every holder of a valid key with the files scope, including one whose enforce_ownership confines it to its own files, so a tenant that cannot read another tenant's bytes could still read the deployment's storage layout out of an error body; method: none widens it to anyone who can reach the port. The substitution is made once at each response boundary through a new FileError::client_message, not at the construction sites, because the path is exactly what an operator debugging a storage failure needs and the log is where it belongs: Display is untouched, so every error! and warn! site keeps the full text at unchanged fidelity, and the tests assert both halves through capture_logs. The status, the type, and the code are all preserved (500 / server_error / storage_error and io_error), so a client that branches on them sees no change and only the human-readable message becomes the fixed storage error: the file operation could not be completed. The client-caused variants keep their messages verbatim, since those name the caller's own file_id, purpose, or limit. Three client-facing surfaces carried the same string and all three are covered: the public /v1/files* handler, the chat-completions file_id path (where the resolution warnings are joined into FileResolutionError::PartialResolutionFailed and written into a 404 body, so the redaction happens where each warning is built and the full error is logged there instead), and the Anthropic /v1/messages path (where FileResolverError::ReadError reaches a 502 as File service error: ...). The Responses API needed no change and was left alone, since try_resolve_files degrades every non-limit error back to the original request so only LimitExceeded and Timeout ever reach a client. Two adjacent defects found by verifying the per-variant claim rather than assuming it are fixed in the same change. FileError::NotFound raised by the storage backend carried {shard}/{id}.bin rather than the file id, so a download whose backing object had been deleted disclosed the sharding scheme through a 404; FileService now relabels it to the id the caller supplied, which is both safer and a better message. And build_error_response in the admin Files API hardcoded "type": "invalid_request_error" in every body it built, including the 500s that map_file_error routed through it, so the admin API labeled a server-side storage failure as client-caused and disagreed with the public handler, which takes the type from error.error_type(); the type now follows the error, with tests pinning one admin 500 and one admin 4xx so the two classes cannot silently collapse again. The admin surface redacts on the same terms as the public one rather than keeping the detail behind the admin boundary: the operator's answer is in the log either way, and one policy for both surfaces is what stops them drifting apart, which is how the type bug arose. src/admin_config/prompts_api/handlers.rs was surveyed as part of the same sweep and carried a narrower instance: its PathTraversal rejection for a parent directory outside the base rendered the resolved absolute path, while the sibling check a few lines below already reported the caller's relative path, so it now matches; PromptFileError::IoError and InvalidPath gained the same client/log split.
• Stop a failing Azure Content Safety sub-check from discarding the other sub-check's refusal (#1194, follows PR #1182). AzureContentSafetyGuardrail::check_input_text inspects the input stage with two independent endpoints, Prompt Shields (text:shieldPrompt) and Content Safety text analysis (text:analyze), and #1182 combined them with ? when it converted the sub-checks to the reportable GuardrailCheckResult shape. The ? short-circuited: a Prompt Shields failure meant text analysis never ran, the whole check became an error, and under the default on_error: fail_open the service resolved that error to Allow, so content Azure text analysis would have refused was served instead. The symmetric case discarded a jailbreak block when text analysis was the endpoint that failed. Both sub-checks now always run, concurrently rather than back to back so a slow endpoint cannot eat the timeout budget the other one needs, and the outcome is chosen by whether the survivor already refuses: a surviving Block is returned and enforced, while a survivor that does not refuse (Allow, or a non-blocking Flag) still returns the failure so the fail policy decides the disposition and the counters from #1182 (errors_total, fail_open_total / fail_closed_total, and guardrail_errors_total{kind="error"}) still move. The test is "does the survivor already refuse" rather than "did the survivor find anything" because the adapter deliberately cannot see on_error and so has to return something correct under both policies: a Block is exactly as strict as the strictest verdict the fail policy could synthesize, while a Flag maps to allow at the gate, so returning one would serve a request fail_closed was configured to refuse and would leave the endpoint outage uncounted. That keeps both invariants at once rather than a plain "return whichever succeeded": a half-completed inspection that refused nothing never reads as a clean pass, and a genuine refusal is never swallowed because the sibling endpoint happened to be down. fail_closed behavior is unchanged, since an error and a block both refuse the request. The audit re-confirmed this was the only call site with that shape: check_output runs text analysis alone (Prompt Shields is input-only), and the Bedrock provider routes both stages through a single ApplyGuardrail call.
• Report the per-backend inventory identity under the hub's stable_id wire key, repairing a silent v0 wire defect from #1172 that made the entire backend cost-center tier inert (#1162 review, PR #1190). #1172 shipped BackendInfo.backend_id while the hub's independently written counterpart (hub #732) reads stable_id; both sides carry serde defaults with no rename and no deny_unknown_fields, so the hub parsed the router's key as absent, no backend ever became assignable cost-center inventory, and backend_assignments was always empty in practice. Verified against the hub clone at the pinned rev rather than inferred. The vendored field is renamed to the hub's key with the upstream doc comment adopted, the operator-facing backends[].backend_id config key and the separate (correct) UsageRecord.backend_id field are unchanged, and the hub's committed heartbeat_stable_backend_identity.json fixture is copied byte for byte and pinned in both directions with a guard that the identity never serializes under the local config key again. The same re-sync vendored the hub's committed policy_envelope.json as an envelope-level value pin, which immediately caught and fixed a second parse-fidelity gap: KeyEntry.project_id (hub #685) was silently dropped on re-serialization and is now carried verbatim, enforcement untouched pending #1152.
• Stop a dead backend from staying in the per-request rotation under load, and stop non-listed statuses from being recorded as circuit-breaker successes (#1148). Killing a primary backend mid-traffic preserved availability, because streaming fallback rescued every request, but first-token p95 stayed degraded for the whole outage: the dead backend was re-attempted on every request. Three distinct gaps produced that metastable state, and the reporter's proposed mechanism (that fallback's success prevents failures from accruing) was not one of them, since every fallback hop already records the primary's transport failure before advancing the chain. First, CircuitBreaker::record_failure converted a status outside failure_status_codes into record_success, which in the Closed state zeroed failure_count and pushed a success into the sliding window. That is the real defect the issue's title names: a backend interleaving 429s or 404s with genuine 5xx could never reach its threshold, and one answering only 429 looked healthy forever. Those statuses now record a neutral outcome that releases the half-open probe slot and touches nothing else, reported through a new OutcomeRecord return so the Prometheus mirror stops counting them as successes too. 429 stays out of the default failure_status_codes: a transient rate limit means the backend is alive and throttling, opening its circuit would convert throttling into an outage, and the retry path from (#740)/(#742) already honors the upstream Retry-After and fails fast on non-transient quota exhaustion. Second, the breaker is off by default, so every recording seam was a no-op and exclusion fell entirely to health checking, whose 90s default window (interval 30s x unhealthy_threshold 3) contains the reported degradation entirely. The breaker stays opt-in, because switching it on by default would change request routing for every existing deployment; instead the router now emits a startup warning when fallback.fallback_chains is configured without one, naming the failure mode and computing the exposure window from the configured health_checks values, and the three shipped config templates that configure fallback gained a circuit_breaker section. Third, a large set of dispatch paths never touched the breaker at all: image generation and image edit, the Responses-only passthrough bridge, native Gemini multimodal embeddings, the whole Anthropic Messages ingress (native, OpenAI-compatible, Responses-backed, Bedrock Runtime, Unix-socket, and web-search emulation), the Responses ingress (all four conversion strategies plus passthrough and compact), and Anthropic count_tokens. All of them now admit and record through a new proxy::circuit::Admission guard, which pairs admission with exactly one outcome and releases the half-open probe slot on drop, so the multi-exit OAuth-retry paths cannot leak probe capacity. Crucially, admission is never applied on its own: proxy::selection::select_admissible_backend filters candidates through filter_admissible, selects, admits, and re-selects on a lost race, returning the backend together with its guard. Pairing a dispatch-time gate with a selection step that does not know about circuits would have converted requests a healthy peer could serve into router-generated 503s. HalfOpen is the one state where a neutral outcome still advances the circuit toward closing, because that state asks whether to stop rejecting traffic rather than whether the backend is healthy; without it a backend answering only 429 would be pinned at its probe ceiling with no timeout to end it. The unwired second circuit state machine in services::health_service is documented as off the data path rather than mistaken for the real one.
• Converge the heartbeat guardrails block on the hub's shape, so a guardrail-active router delivers inventory at all (#1150). The router's GuardrailSummary and the hub's were written independently and did not match, and because RouterInventory.guardrails is a nested struct rather than a tolerated blob, serde failed the whole struct: a router with guardrails active and control_plane.enabled true delivered no inventory whatsoever, not backends, not health, not load, not supply, not policy_status. Verified against hub 58bb3d0 rather than inferred. GuardrailSummary now carries the six scalars the hub declares required (transforms_total, flags_total, errors_total, fail_open_total, fail_closed_total, stream_buffer_cap_trips_total), and the category breakdown is the hub's by_category object keyed by category id rather than the blocks_by_category array this router used to send, which is the one genuinely breaking wire change and is safe precisely because no hub could parse the old shape. transforms_total, flags_total, and stream_buffer_cap_trips_total are derived in GuardrailSnapshot::into_summary from the breakdowns they summarize rather than counted twice, so the scalar the hub reads and the breakdown a later hub will read cannot drift; a test asserts each against its sum. errors_total, fail_open_total, and fail_closed_total had no source in the control-plane tracker at all and are now recorded by GuardrailTracker::record_error from the same seam in src/services/guardrail/service.rs that records guardrail_errors_total, deliberately not read back from the Prometheus registry: metrics is optional, and reporting zeros from a build without it would tell the hub that a guardrail silently letting traffic through is idle. The two fail-policy counters split by the disposition that actually applied rather than by the configured on_error: monitor mode never gates, so a failed check under monitor is traffic that was served uninspected and counts as fail-open whatever the policy says, and reporting it as fail-closed would zero out the one counter that surfaces an enforcement hole. errors_total is a floor rather than a total, because only a provider timeout reaches that seam: a provider that fails fast resolves its own transport, status, or parse failure into a verdict internally, a gap the Prometheus surface shares and which needs an explicit failure marker on ProviderOutcome to close. The router-only verdicts and stream_buffer_cap_trips breakdowns stay as additive optional fields, ignored by today's hub because it uses no deny_unknown_fields. Category ids stay bounded in count, byte length, and charset by construction, since the only way a key enters by_category is a GuardrailCategoryId literal, and is_bounded() now checks all three against the hub's own ingestion rule. The hub's committed guardrail_summary.json is pinned here byte for byte, with the same "re-copy on any hub change, never regenerate from local types" rule the guardrail policy fixtures carry, because both repositories previously round-tripped only their own types and that is exactly how the shapes diverged while both stayed green.
• Report the guardrails configuration section as an immediate hot reload rather than restart-only (#1127). ConfigSection::Guardrails declared HotReloadCapability::RequiresRestart, which the Admin API surfaced to operators and which had been wrong for the mode flips, threshold edits, route changes, and provider rows that GuardrailService::update_config has always applied live. The enabled toggle no longer needs a field-level exception either: since the guardrail service handle became late-settable, a startup-disabled router materializes a service the first time a reload or a hub policy enables guardrails, and a runtime disable is expressed as enabled: false on the live service rather than tearing it down, so both directions are live. Nothing in the section requires a restart, and the section is now classified accordingly.
• Rewrite response-scoped backend tool-call ids to a reversible, per-response-unique form at the /anthropic/v1/messages boundary (#1143, closes #1142). tool_use.id is conversation-scoped in the Anthropic protocol, but vLLM synthesizes {tool_name}:{index} for Kimi-K3 with the index restarting on every response, so an agent calling the same tool on two turns was handed read_file:0 twice; Claude Code's ensureToolResultPairing then treated the replay as corrupted, substituted a synthetic "tool use interrupted" result, and ended the turn, with the router reporting nothing but 200s throughout. Outbound ids that are not already known conversation-unique (the toolu_, call_, fc_, and chatcmpl-tool- prefixes, and only when the whole id is charset-safe) are now rewritten to crt_<nonce:12><base64url_nopad(raw_id)>; the 12-character UUID v4 nonce is what actually breaks the collision, since base64url alone is deterministic and would reproduce the same client-visible id on every turn. The raw id is restored byte-exact on the way back in rather than charset-cleaned, because a backend has been reported to dead-stop when replayed a mutated form of its own id template. Decoding never fails a request: anything that doesn't match the encoded shape passes through unchanged. New src/http/handlers/anthropic/tool_id.rs; native Anthropic backends stream passthrough SSE and never reach it, and Gemini already synthesizes conversation-unique toolu_ ids of its own.
• Stop POST /admin/config/export with format: "toml" from failing unconditionally (#1159, closes #1138). Most optional Config sections carried #[serde(default)] without skip_serializing_if = "Option::is_none", so an unset section serialized to an explicit null, and the toml crate has no representation for a null; a default export carried 19 such nulls. The 21 remaining top-level optional Config fields now skip when unset, along with the nested optionals that were still leaking a null into a default export (ServerConfig.workers, ServerConfig.connection_pool_size, ResponseCacheConfig.{redis,l1,l2,tiered}, DisaggregatedServingConfig.default_external_storage, S3CacheLayerConfig.ttl_override, and RequestTimeoutConfig.image_generation), so GET /admin/config/full, POST /admin/config/export, and config-history snapshots now omit an unconfigured optional section instead of carrying it as null. get_section_value maps that absence back to Value::Null, so GET /admin/config/{section} on a known-but-unconfigured section still answers 200 with a null body rather than 404, and PATCH still merges onto that null base instead of failing with 500. The WebUI's structural-diff and history-diff helpers (config.js, history.js) now treat an absent key and an explicit null as the same state, so pasting an older export against the new response shape no longer renders a vanished null key as a spurious addition or removal.
• Truncate log strings on UTF-8 character boundaries instead of fixed byte offsets, so a non-ASCII value can no longer abort the whole router process (#1158, closes #1157). Eighteen byte-index &str slice expressions across twelve log-truncation sites in proxy, core::files, http::middleware, http::handlers::anthropic, infrastructure::backends::anthropic, and services::smart_routing panicked whenever the fixed offset landed inside a multi-byte character, and the release profile sets panic = "abort" (Cargo.toml:311), so the panic terminated the process instead of failing the one request. Two sites are reachable by an authenticated client without any unusual deployment: proxy::utils::sanitize_file_id_for_log truncates a raw file id once FileId::from_string has already rejected it, and a byte-for-byte duplicate that used to live in core::files::transformer truncated a validated id, because FileId::from_string's Unicode-aware alphanumeric check accepts non-ASCII values; both panicked on ids such as file-일이삼사오육. infrastructure::backends::anthropic::transform_openai_file_to_anthropic (a malformed multi-byte file_data preview) and http::middleware::admin_audit::mask_username (a multi-byte Basic auth username) are reachable the same way. New src/core/text_utils.rs adds truncate_on_char_boundary and truncate_tail_on_char_boundary, which back off to, or advance to, the nearest valid boundary instead of slicing blindly; every affected call site now routes through them, and the core::files::transformer duplicate was deleted in favor of the shared sanitizer. Old and new output were diffed exhaustively across every ASCII length from 0 to 700 for four alphabets rather than checked by inspection, so existing log formats and doctested output are byte-identical.
• Escape and cap client-supplied values recorded in Anthropic handler log fields, so an authenticated caller can no longer forge additional log records by embedding a newline in a value (#1161, closes #1151). field = %value wraps a value in DisplayValue, and tracing_subscriber's default formatter writes any field other than message as a bare {:?} with no wrapping, so the Display output landed in the log line raw; interpolating into the format string fared no better, because the EscapeGuard applied to message only rewrites ANSI and C1 controls and lets \n and \r through. Roughly 40 client-influenced sites across src/http/handlers/anthropic/ (tool names, tool-call and file ids, effort strings, the request model, the anthropic-version header, and enum discriminants such as AnthropicToolChoice::Simple) now record the value as a named field without the % sigil, so it routes through Visit::record_str and is formatted with Debug for str, which quotes the string and escapes \n, \r, \t, ", \, and non-printables into a single log line. Escaping alone stops forging but not bloat, so every one of those values also passes through a new core::text_utils::cap_client_value_for_log, which truncates at CLIENT_LOG_VALUE_MAX_BYTES (256 bytes) through the existing boundary-safe truncate_on_char_boundary rather than a byte-index slice. Router-controlled values (backend.name, url, status, typed enums, numerics) and two mime_type fields provably constrained to a fixed allow-list keep the % sigil, since escaping bytes that can never be client-chosen would only hide a real regression.
• Preserve live backend credentials when a masked configuration export is re-imported, instead of overwriting them with their own mask placeholder while reporting success (#1160, closes #1139). Exporting a configuration with the default include_sensitive: false and importing it straight back, the flow the WebUI's Configuration page presents, wrote every masked credential through as its own placeholder and answered success: true; the breakage surfaced later as 401s from every backend, with recovery meaning re-entering each secret by hand. The old placeholder shapes (sk...(21 chars), the bare ***MASKED*** for a value of four characters or fewer, ${***VAR***} for an environment reference) were derived from the secret itself, so none of them told a masked value apart from a real credential of the same shape. Every masked value the Admin API returns is now wrapped in a fixed sentinel, ***CONTINUUM-MASKED:<hint>*** (a value of four characters or fewer now reports (N chars) where it previously showed the bare ***MASKED***), and only that sentinel is read as "keep the value that is already live". POST /admin/config/import, POST /admin/config/apply with a config candidate, PUT/PATCH /admin/config/{section}, and the POST /admin/config/validate pre-save dry run all resolve placeholders the same way, so the dry run predicts exactly what a save will do; a write takes the configuration modification lock before splicing so a concurrent change cannot move a secret out from under the resolution. Import reports every preserved path through a new preserved_secret_paths response field, and every write path, including the dry run, reports the same set as MASKED_SECRET_PRESERVED validation warnings. Array entries resolve by id or name rather than position, so reordering backends still restores correctly; a list with neither, such as guardrails.bypass_api_keys or rate_limiting.bypass_keys, has only position to go on, so adding or removing an entry is refused rather than guessed, since two keys from the same provider can share a prefix and length and a shifted index would otherwise splice in the wrong one. A placeholder that cannot be matched to a live value fails the whole write with MASKED_SECRET_UNRESOLVED and the failing path named, rather than borrowing a neighboring secret. The three legacy placeholder shapes are still recognized, but only to refuse the import with LEGACY_MASK_PLACEHOLDER: a document exported by a router built before this fix is refused rather than silently trusted, and the way out is to re-export from the upgraded router or replace the flagged value with the real secret. Both the preserved-path and refused-path lists are capped at 500 entries with a MASKED_SECRET_REPORT_TRUNCATED marker, so a document filled with placeholders cannot inflate the response. Backend PUT/PATCH and the WebUI's Configuration and History pages recognize the new sentinel too. Documented in the English and Korean Admin API and WebUI guides.
• Enforce the per-API-key and per-client (IP) rate-limit dimensions on live traffic, which had never fired in production (#1164, closes #1147). Three wiring defects compounded: the limiter read a bare SocketAddr request extension where axum inserts ConnectInfo<SocketAddr>, so client_ip was permanently None and the per-client dimension, the IP whitelist, and the trusted-proxy X-Forwarded-For path were all dead; the limiter was layered on the whole application outside the API-key authentication that populates AuthContext, so the per-key dimension had no identity to bind to; and rate_limiting.limits.per_api_key was never read at all. Only the global dimension worked, so a single flooding tenant was not shed and collapsed a well-behaved tenant from roughly 22ms to 5600ms first-token latency. The limiter is now split by dimension across two mounts sharing one store: the global dimension is enforced by a single application-wide mount outside authentication and CORS, so every request is charged against limits.global exactly once, including API requests authentication later rejects with 401 (key guessing stays metered), failed admin authentication, CORS preflights, and the 404 fallback; the per-client dimension is enforced on the API routes outside authentication and the remaining identity dimensions (per-key, per-model) inside it (mount placement subsequently refined by #1170). Because the mounts enforce disjoint dimensions, no request is ever counted twice against any bucket. On refusal the refusing dimension's numbers are authoritative, and when the global and an identity dimension would both refuse, the client sees the global refusal because it is checked first. Two behavior changes for existing configurations. rate_limiting.limits.per_api_key is now the enforced default for every recognized key that has no api_keys[].rate_limit of its own, so a configuration that already carried that block will start shedding keys that were previously unlimited; an explicit rate_limit still overrides it and rate_limit: 0 exempts a key. And the 429 body is now a single valid JSON document (error.message, error.type, error.code, and error.details.limit_type), where it was previously a JSON error document re-embedded unescaped inside a second envelope and therefore unparseable; refusals now also carry retry-after and x-ratelimit-*, with Retry-After floored at one second per #1146. Dimensions skipped for missing identity (per-client on a Unix-socket listener, per-key for an anonymous caller in permissive auth mode) are now recorded in the new rate_limit_skipped_total{dimension} counter and a debug log.
• Stop the rate limiter from disclosing caller identity and shared fleet state through its response headers, bind unauthenticated floods on the dimension that exists for them, and stop a full bucket map from refusing traffic (#1170). Enforcement is unchanged throughout: every configured dimension still charges and still refuses exactly as before, and refusal attribution, Retry-After, and the 429 body are untouched. What changed is what the router reports and where two mounts sit. x-ratelimit-* now describes the caller's own quota and nothing else. The previous contract reported whichever dimension had the fewest remaining tokens, which made the numbers a function of who the caller was: with limits.per_api_key burst 7 and limits.global burst 100, a valid key read 7/6 and an unrecognized one read 100/99, the only key-recognition signal the router emits in permissive auth mode. The same headers also handed any unauthenticated caller a live read of the shared global bucket at one token per probe, on 401s, fallback 404s, and unauthenticated 200s such as /version, turning a global-exhaustion flood from blind into precisely timed; and because a bypass_keys holder produced the exemption sentinel and therefore no headers at all while any other token got them, with both responses 401, the limiter alone confirmed bypass-list membership. The rule is now that only quota tied to an authenticated identity is reported, which today means per_api_key: the caller has proved they hold the key the bucket is keyed on, so the numbers tell them nothing they do not own. Everything else is withheld on an admitted response. global, per_model, and per_backend are drawn on by the whole fleet, and per_client is drawn on by everyone behind one source address, so no shared level is the caller's quota; per_client is additionally evaluated before authentication, where the router cannot name whom it would be reporting to, and reporting it would have reopened the very oracle finding 3 describes, since a bypass_keys holder gets the exemption sentinel and no headers while any other token would get real ones on an otherwise identical 401. A response that dimension did not evaluate carries no x-ratelimit-* at all, uniformly, which removes the discriminator from the 401, 404, unauthenticated 200, whitelist, and bypass-key paths together. A refusal still reports the refusing dimension's numbers, including a shared one, because the client needs the retry hint and "the bucket is empty" is already implied by the 429. Two consequences worth checking before upgrading. Admitted responses now carry quota headers only when per_api_key evaluated the request; configure that dimension if your clients read them. And the permissive-mode residual is accepted rather than fixed: a recognized key's response carries per-key headers while an unrecognized token's carries none, so their presence still distinguishes the two. Suppressing them for unrecognized callers would only move the signal to their absence, so the decision is to state plainly that permissive mode is a development and trusted-network convenience rather than an authentication boundary, since it already serves valid and invalid keys identically; set api_keys.mode: blocking where key validity is sensitive. per_client moved outside authentication on the API routes, from inside it. The dimension keys on the peer address, which the connection supplies before authentication runs, so grouping it with the dimensions that genuinely need AuthContext left it unable to constrain the exact pattern it exists for: twelve bogus-key requests from one address in blocking mode all returned 401 and none were shed, leaving the flood metered only by the shared global bucket and shedding every other tenant as it drained. Mounting it on the same route group but outside auth binds the flood on its own bucket without widening its reach to WebUI, admin, or static traffic. The request path on an API route now reads global -> per-client -> authentication -> per-key -> handler, and the three mounts still enforce disjoint dimensions, so nothing is double-charged. The metrics exemption is derived rather than fixed. Matching the literal /metrics had both halves backwards: it exempted a route that did not exist when metrics were disabled, leaving an unmetered 404 sink, and it limited a scrape an operator had moved to a custom metrics.path, shedding exactly the monitoring needed during an incident. The exemption now follows the effective metrics.path and applies only while metrics.enabled is true. Keeping scrapes un-sheddable is deliberate, since a limiter that starves monitoring under load is worse than an unmetered scrape endpoint, so the residual is accepted and stated: a scrape's cost grows with metric cardinality and the limiter will not bound a flood of them, which is why metrics.auth and network policy matter. /health and /healthz stay exempt unconditionally so a saturated router keeps answering its own liveness probe instead of being restarted mid-overload. A full bucket map now fails open instead of closed. per_client, per_api_key, and per_backend each cap at 100,000 entries with a 600 second TTL, and hitting the cap returned a Capacity refusal; the client and backend checks had no already-present guard, so once full they refused callers that already owned a bucket too. Refusing saved no memory, because the refusal path inserted no bucket either, while 100,000 distinct source addresses are easy to source from a single IPv6 /64, so failing closed let an attacker convert a bounded map into a ten-minute outage for every caller on that dimension. A full map now skips that dimension for the request, leaving the others enforcing, and records it on rate_limit_skipped_total{dimension} plus a per-dimension warning rate-limited to one line per minute; the inline expiry sweep is throttled to at most once per second per map, so a flood of unique keys cannot force a 100k-entry scan on the hot path while the background sweep every 60 seconds keeps doing the steady-state pruning. Per-client buckets still key on the full address, so an IPv6 source varying its low 64 bits both evades the dimension and saturates the map; bound that at the edge. A 429 is readable in a browser whichever mount produced it. The global mount sits outside the CORS layer, which is what makes preflights chargeable, so its refusal carried no Access-Control-Allow-Origin at all and a browser turned the one refusal a client most needs to act on into an opaque network error. The mount does not move; its short-circuited refusal now carries an allow-origin echo for an origin listed in cors.allow_origins, Vary: Origin, and the credentials flag when configured. Separately, retry-after and the x-ratelimit-* headers are now always added to the CORS layer's Access-Control-Expose-Headers, because without them cross-origin script could read a refusal's status but not its retry hint, which is the actionable half, and that gap applied to every mount rather than only the outer one. Documented in the English and Korean rate-limiting guides.
• Bound Anthropic file_id resolution per request, so one small request backed by one uploaded file can no longer expand into gigabytes of resident memory (#1169, closes #1168). src/http/handlers/anthropic/file_resolver.rs fanned out over messages, content blocks, and blocks nested inside a tool_result with three unbounded try_join_all calls, and enforced its 10 MiB MAX_INJECTION_SIZE per file rather than per request, so roughly 57 KiB of request body carrying 800 references to a single 10 MiB upload resolved into about 10.7 GiB of base64 held simultaneously. Deduplicating repeated file_id values would not have fixed it: resolve_image_source and resolve_document_source call BASE64_STANDARD.encode once per reference, so every reference keeps its own encoded copy resident whether the bytes were read once or 800 times, which is why the aggregate budget added here counts every reference including duplicates rather than summing unique files. Three bounds now apply, each closing a different term. MAX_TOTAL_RESOLVED_BYTES (32 MiB of raw bytes per request, about 43 MiB once base64 expands it) is charged in load_file from metadata.bytes before any content is read, through a resolver-owned AtomicUsize whose running total is tested on the value fetch_add returns rather than with a check-then-add that would race the other resolutions in flight, so the bound is exact rather than approximate: a read-modify-write always reads the value written immediately before it in the atomic's single modification order, which makes every observed running total a prefix sum, so the references that pass the test are exactly a prefix of that order and their sizes cannot sum past the budget under any interleaving. MAX_FILE_REFERENCES_PER_REQUEST (100) is applied in a single pre-pass over all three traversal levels before the first metadata lookup; it is deliberately five times the chat-completions limit of 20 (src/proxy/files.rs) because the Messages API replays the whole transcript on every turn, so a long multimodal session legitimately accumulates references a single chat-completions call never would, and the count cap is a cheap pre-filter rather than the memory bound. MAX_CONCURRENT_FILE_LOADS (8) is enforced by a tokio::sync::Semaphore on the resolver, because the three fan-out sites nest and a per-level buffer_unordered(8) would multiply to 512; the permit is held across the storage read only and never across the recursive resolution above it, so a tool_result can never wait on a permit held by one of its own ancestors. The resolution phase also gained the 30 second timeout the chat-completions path already had, mirrored rather than imported because proxy::files is private to crate::proxy. Both new limits surface as 400 invalid_request_error and a timeout as 504, through a new FileResolutionResult::into_request_or_response that the streaming and non-streaming handlers now share in place of their two duplicated match blocks, so a resolution outcome cannot be handled on one path and silently missed on the other. The pre-existing per-file FileTooLarge still answers 502 and was left alone. Documented in the English and Korean API guides.
• Keep a classifier verdict that precedes a closing reasoning tag (#1241, follows PR #1214). strip_reasoning_prefix keeps everything past the last </think> on the rule that a reasoning block sits ahead of the verdict, so output where the tag lands after the answer (unsafe\nS1</think>) stripped to an empty head. Since #1203 made unreadable output a failed check, a real refusal became a failed check and, under the default on_error: fail_open, was served. That is the outcome #1203 exists to prevent, relocated rather than removed.
• Report a fast-failing guardrail provider as a failure rather than a clean allow (#1182). A transport error, a non-success HTTP status, or an unparseable body was resolved into a fail-policy verdict inside each adapter, so the telemetry seam, which fired only on outcome.timed_out, never saw it: under fail_open a moderation outage produced a heartbeat byte-identical to a healthy guardrail on clean traffic, and under fail_closed a flood of uncategorized safety blocks. The Guardrail trait now returns Result<GuardrailVerdict, GuardrailCheckError>, so a new adapter physically cannot swallow a failure and the fail policy is applied at the single service seam shared by the buffered and streaming paths, resolved per check from the live config snapshot instead of frozen at construction. The trait change is breaking for out-of-tree adapters. The PII provider keeps a local on_error for its documented degrade path only, where an external-recognizer failure under fail-open completes the check with builtin detections, which is a reduced-fidelity inspection rather than a failed check.
• Enforce the guardrails.allow and guardrails.deny match lists, which were parsed, regex-validated at startup, settable through the Admin API, and documented as enforcement features while no code in the request path ever read them (#1104). A new matchlist module compiles the global and per-route lists once per configuration snapshot, so no regex is compiled on the request path, and the config snapshot became a PolicySnapshot carrying the config and its compiled rules behind one lock, so a check can never read config from one generation and rules from another. Precedence is deny, then allow, then providers: a deny match blocks before any provider runs, an allow match short-circuits with no provider call, and route lists extend the global lists rather than replacing them, so a route override cannot weaken a global deny rule. exact entries are case-insensitive literal substring matches, since a case-only variation must not evade a security control; blank literals and empty patterns are dropped with a warning rather than matching everything. Matches are recorded under the reserved pseudo-provider labels match_list_deny and match_list_allow, so a monitor-mode match is measurable before enforcing, and the block reason never echoes the matched text or the rule. Enforce mode now requires at least one provider or at least one deny rule, where it previously rejected a deny-list-only policy that genuinely gates.
• Correct and complete the built-in OpenAI model catalog, which stopped at GPT-5.2 and left every newer tier to name heuristics (#1177). A discovered gpt-5.3-codex was advertised with a 256K context window, a 4096-token max output, and a fabricated \(10/\)30 per 1M price instead of 400K/128K at \(1.75/\)14. The GPT-5.5, GPT-5.4, and GPT-5.3 families plus gpt-5.2-codex were added to the built-in table, gpt-5.3-codex-spark and the three earlier codex tiers to model-metadata.yaml. gpt-5.1 was wrong in both catalogs in different ways and is now 400K/128K at \(1.25/\)10 with a 2024-09 cutoff and no audio capability it never had; gpt-5.1-codex and gpt-5.1-codex-mini are corrected likewise. Every codex tier is documented as Responses-API-only, so all four now carry responses_only and the router bridges Chat Completions requests for them to /v1/responses. Retirements were decided by cross-checking OpenAI's deprecations page against the Azure Foundry retirement schedule.
• Correct solar-pro-3 pricing, capabilities, and aliases (#1097, closes #1095). The entry advertised zero rates, which is this file's convention for open-weight models with no hosted API cost, so /v1/models reported a paid hosted model as free and any downstream cost estimation under-reported Solar Pro 3 spend to nothing; Upstage charges $0.15 per 1M input and $0.60 per 1M output tokens. The reasoning capability was missing while the entry's own summary claimed improved reasoning accuracy. The solar-pro3 and solar-pro3-260323 names Upstage actually publishes resolved to nothing, since neither the 4-digit nor the 6-digit date suffix is touched by the 8-digit date-suffix normalization, so both are now listed explicitly. solar-pro-2 gains its own solar-pro2 and solar-pro2-251215 aliases for the same reason (#1099, closes #1098).
• Rebuild the runtime state a configuration write drops, so an Admin API change no longer publishes a config with broken backend lookups and model metadata (#1100 closes #1089, #1136 closes #1116). put_config_section and patch_config_section round-trip the running configuration through JSON, which silently resets the #[serde(skip)] runtime state (backends_index, model_metadata_cache, response_defaults_cache) to its type default, and nothing rebuilt it, so a successful section PUT or PATCH broke backend lookup and metadata resolution until the next full config load or file-based hot reload. Both handlers now rebuild the index and reload metadata in the same order reload_config_from_file uses, and the follow-up extends the same treatment to the full-config apply, import, and rollback publication paths and centralizes index rebuilding across the create, update, delete, weight, and models mutations.
• Fail a hot reload when model-metadata assembly fails, instead of warning and publishing a revision with empty caches (#1102). publish_config_revision assigns the whole Config, so model_metadata_cache and response_defaults_cache were replaced wholesale and the router kept serving with pricing, context windows, capabilities, and /v1/models response defaults silently reverted to backend-reported values, with no restart to explain it. Metadata assembly was the one reload step that warned and continued where path validation, the oversize check, parse failure, env overrides, secret processing, and config validation all return Err and leave the previous revision serving intact; it now does the same. Only genuinely fatal conditions reach that path: an unreadable or unparseable base file, a merged document that does not match the metadata schema, and the MAX_METADATA_LAYERS (64) and MAX_TOTAL_METADATA_BYTES (8 MiB) caps. A missing base file and a drop-in that fails to read, parse, or validate are still warned and skipped. The cap breaches are what made this newly reachable, since anyone with write access to a model-metadata.d/ directory can add a 65^th file without touching an existing one.
• Validate every /admin/backends hot-reload candidate through the shared config validator before publishing it to the live watch channel (#1206), so the backend-specific Admin API agrees with the file and config ingestion paths. Validator failures map to 400 while watch-channel send failures keep their server-error behavior, and the backend_id-only Admin API validation helpers were removed because format and uniqueness now flow through the same whole-config gate as the rest of BackendConfig. Regression tests on create, update, delete, weight, and models prove a rejected candidate is neither published nor recorded in config history.
• Allow an empty backend pool at startup in both factory paths, honoring the existing zero-backend deployment contract, while preserving fail-fast behavior for a non-empty configuration whose backends all fail (#1134, resolves #1133).
• Defer half-open circuit admission to the dispatch seam, so selecting a backend for local preflight, cache, or guardrail work no longer consumes a probe slot (#1236, follows #1148). A deferred SelectedBackend token is produced only by circuit-filtered selection and consumed into an Admission immediately before upstream I/O, with a late admission race reselecting over the remaining eligible candidates; Anthropic count_tokens, Anthropic Messages, Responses, compact Responses, and the responses-only Chat-to-Responses bridge all moved to inspecting selected backend metadata before admission. Half-open progress is also split from genuine success evidence, so a probe window carrying only neutral outcomes or timeouts returns the circuit to Open with a fresh cooldown, while a window mixing neutral outcomes with real successes may still close.
• Deduplicate CORS exposed response headers by retaining the first parsed header name, so an operator-configured duplicate of a built-in header no longer appears twice (#1220). The previous Vec::dedup was adjacency-dependent and missed a configured retry-after, whose built-in counterpart is not adjacent in the combined list.
• Preserve per-key in-memory stats attribution when Prometheus metrics are disabled (#1209, closes #1196). API-key attribution and label sanitization moved into an always-compiled core module, and both the local and hosted CI matrices gained metrics-off regression coverage.
• Surface guardrail dry-run provider failures in POST /admin/guardrails/test and the Guardrails WebUI test console, so a fail-open substitute is distinguishable from a genuine allow (#1221). Sanitized failure metadata now travels through GuardrailService::dry_run, and the console renders fail-open and fail-closed result chips with the failure detail. Documented in the English and Korean WebUI guides.
• Surface PII external-recognizer degradation as local telemetry (#1223). The new guardrail_degraded_total{provider,kind} records kind="external_unavailable" when the PII provider falls back to builtin detections under fail-open because its optional external recognizer is unreachable, which is a reduced-fidelity inspection that guardrail_errors_total deliberately does not count. The built-in-only disposition and the fail-closed error behavior are unchanged.
• Restore the four BackendConfig struct literals #1172 broke, and close the CI gap that hid them (#1200). #[serde(default)] covers deserialization and does nothing for a Rust struct literal, so adding the required backend_id field required updating every complete literal in the same change; the two inside a get_health_check_config rustdoc example survived because nothing in CI compiled doctests, and the two Bedrock integration literals survived because bedrock-sigv4 was built by no job and no release binary. .github/workflows/ci.yml and scripts/local-ci.sh each gain two mirrored steps, cargo test --doc and an all-features cargo check --all-targets --keep-going, the second chosen over a narrow --features bedrock-sigv4 because it covers every unpinned feature at once and keeps covering features added later without a matrix edit.
• Add the no-op FileMetrics::record_retention_delete to the metrics-disabled compatibility surface (#1238), restoring compile parity for every no-default feature graph that includes the always-compiled Files service. The gap appeared only once #1234 and #1233 were combined on main.
• Clarify what the guardrail heartbeat verdicts and errors_total fields count (#1227), so they are read against the right units for a multi-provider stage and for fail-closed substitute blocks. The continuum-protocol rustdoc and the English and Korean guardrails guides now describe failed checks explicitly, bound the fail-open reading as an upper bound on uninspected allow traffic, and document where verdicts and blocks_total diverge under fail-closed.

Dependencies¶

• Enable the io feature on tokio-util for the streaming Files API paths (#1189), and make arc-swap a non-optional dependency now that the guardrail runtime service materialization path depends on it unconditionally (#1126). No crate was added to or removed from the dependency graph in this release.

v1.16.0 - 2026-07-28¶

Run the streaming output guardrail gate on every streaming path, make it redact instead of only block, keep it checking past its 4 MiB buffer cap, give monitor mode something to observe, and add metadata download with layered model-metadata drop-in directories.

Added¶

• Add layered model-metadata drop-in directories so a vendor baseline and operator customizations stop competing for the same file (#1069). model_metadata_file had exactly one slot, which forced a choice between never updating the shipped copy and hand-merging every upstream change, and made metadata download (#1067) unsafe next to local edits because it replaces that file wholesale. Metadata is now assembled from a layered search path, lowest precedence first: the base model_metadata_file, then /etc/continuum-router/model-metadata.d/, then <user config dir>/continuum-router/model-metadata.d/, then ./model-metadata.d/, then every directory in a new model_metadata_dirs config key in the order given. That is the inverse of find_config_file's first-match-wins order and deliberately so: expressed as layers, the more specific location wins by being applied last. model_metadata_dirs supplements the conventional locations rather than replacing them and its entries are used verbatim, so a Kubernetes ConfigMap mounted at /etc/router/metadata/ is listed as that path; it defaults to empty, so existing configs deserialize and behave exactly as before. Within a directory only regular .yaml and .yml files are read, in lexicographic filename order so the 10-/50-/90- convention works, with no recursion into subdirectories, dotfiles and *~ and *.swp skipped, symlinks resolved but required to land on a regular file, and a missing directory treated as empty rather than an error. Merging happens on the parsed serde_yaml::Value before deserialization, which is load-bearing rather than incidental: mappings merge recursively key-wise so an override carries only the fields it changes, sequences and scalars are replaced wholesale so an inherited aliases or capabilities entry can be removed, models entries are keyed by id so an existing id merges in place and a new id is appended, and responses_only keeps working because a plain bool with #[serde(default)] is indistinguishable from absent after deserialization, so a typed merge would let an override file that never mentions the key silently clear an inherited true. A new field on ModelMetadata merges correctly with no new merge code for the same reason. The merged document is deserialized and validated once, and backend model_configs still take precedence over it, which still takes precedence over the built-in OpenAI registry. A drop-in that fails to read, parse, or validate is skipped with a loud warning and the rest of the search path still loads, matching the router's existing warn-and-continue posture at startup and on reload so one malformed operator file cannot take down a running router; a malformed base model_metadata_file stays fatal exactly as before. Blame is assigned once, to the layer that was merging when the document stopped matching the schema, so an already-invalid base file is reported as the cause instead of every innocent drop-in layered on top of it. --resolved also stops exiting 1 on an invalid response_defaults, mirroring the router's own warn-and-fall-back behavior there instead of refusing to print the document the router actually serves, and both report renderings surface a config file that was found but could not be loaded, or that could not be found at all, as a # WARNING: header line and a config_warning JSON field rather than silently describing a search path that omits that config's model_metadata_file and model_metadata_dirs. At most 64 files totalling at most 8 MiB are assembled, and exceeding either cap is an error rather than a truncation, because applying a partial override set silently is worse than refusing to assemble. Because skipping is invisible to clients, a new continuum-router metadata show [--resolved] [--json] prints the search path, the applied layers in application order with every model id a later layer took over and the file it came from, and every skipped file with its reason; the default mode prints the raw merge verbatim so a typo is visible, --resolved prints the effective typed document after response_defaults validation and sanitization, and the human header is written as YAML comments so redirecting the output still produces a loadable file. The applied file list and every model-id override are also logged at startup. Reading a layer, including the base file, opens the path once with O_NONBLOCK on unix, checks it is a regular file through fstat on that same descriptor, and reads it through a take bounded by the remaining byte budget, so the type check, the byte cap, and the bytes actually read all describe the same object instead of three separate lookups of the same path; a fifo or device node dropped into an enumerated drop-in directory, or swapped in as the base file, can no longer block open with no timeout, and a file that grows or misreports its length after being stat'd can no longer defeat the size cap. Merging the models list is linear rather than quadratic in model count, indexing the accumulated entries by id once per layer instead of rescanning the whole list per overlay entry. ConfigSection::ModelMetadataDirs is also wired into FromStr, closing a gap where the section was advertised by the section listing and the admin JSON schema but rejected as unknown by the per-section admin GET/PUT handlers. metadata download still writes only the base file and never reads from or writes into a drop-in directory, which is what makes it safe to run repeatedly, and the --model-metadata flag keeps its meaning: it sets the base file, it does not disable drop-ins. Both model_metadata_file and model_metadata_dirs remain requires_restart. libc is now a cfg(unix) dependency for O_NONBLOCK, already present in the lock file, so no new crates are resolved. Documented in the man page, the English and Korean configuration guides, the example config, and configuration-assistant guidance (schema 1.27.0 to 1.28.0).
• Add continuum-router metadata download (visible alias metadata update) to fetch the canonical model-metadata.yaml from the repository and install it at the path the router actually reads (#1067). A release archive, Debian package, or container ships without the file, and an existing copy goes stale silently as metadata-only commits land new frontier models, so there was no supported way to pick those up short of cloning the repo. The destination resolves in a documented order (--output, then the global --model-metadata, then model_metadata_file from the loaded config with the same tilde expansion the loader uses, then model-metadata.yaml next to the discovered config file, then ~/.config/continuum-router/model-metadata.yaml) and is printed on the failure paths as well as the successful ones. --model-metadata sits above the config field because that is exactly what it does for the router itself, so --model-metadata /etc/cr/meta.yaml metadata download installs the file where that instance will actually read it. The download is parsed and run through the loader's response_defaults validation before anything on disk is touched, and unlike the loader (which warns and falls back to defaults) the command treats a validation failure as fatal, so a malformed or schema-invalid response exits 1 and leaves a working deployment byte-for-byte unchanged. Replacement is atomic through a temp file in the destination directory, created exclusively under a per-run unique name so a symlink planted at a predictable sibling path in a directory another user can write to cannot steer the write and two concurrent runs cannot interleave into one buffer file, preserves an existing file's mode (0644 for a new file, not the 0600 of the token store), and keeps the previous file as <file>.bak unless --no-backup. Content is compared by SHA-256 so an unchanged file is skipped without a write unless --force. --check writes nothing and exits 2 when an update is available, so cron and CI can branch on staleness without treating it as an error. Only HTTPS sources are accepted, redirects that leave HTTPS are refused rather than followed by reqwest's default policy, and the streamed body is capped at 8 MiB rather than trusting an advisory Content-Length. While the repository is private a GitHub token is required, because an unauthenticated raw request returns 404 for every ref: the token is read from CONTINUUM_GITHUB_TOKEN, GITHUB_TOKEN, or GH_TOKEN in that order, first non-empty value wins, whitespace trimmed, and there is deliberately no --token flag because a secret in argv leaks into shell history and ps output. The credential is attached only when the request host is exactly raw.githubusercontent.com or api.github.com, compared case-insensitively rather than by suffix, so a --url mirror or a lookalike such as raw.githubusercontent.com.evil.test never receives it, and it is never logged, printed, or placed in the --json report, which names the source variable instead of the value. Basic-auth credentials embedded in a --url are stripped the same way from every rendering of the source, so a private mirror's password does not reach the human report, the --json source_url field, or an error message. A 404 no longer conflates the two causes: without credentials it names the three variables and says the repository may be private, with credentials it says the ref or path may not exist or the token may lack access. --ref pins to a branch or tag, --url overrides the source for mirrors and air-gapped installs, and --json emits the same report as an object. The report covers source URL, resolved ref, destination, model count before and after, added and removed model ids, backup path, and both hashes, plus a reminder that updating the file does not hot-reload a running router. No new crate dependencies. Documented in the man page, the English and Korean configuration guides, and the configuration-assistant CLI quick reference.

Changed¶

• Mirror release container images to cr.backend.ai/product/continuum-router alongside ghcr.io (#1068). Each Debian and Alpine multi-arch tag is now pushed to both registries in one build step, and the final imagetools create sources its manifest from Harbor so a tag is only ever created from a manifest Harbor is known to hold, regardless of ghcr-side state. Downstream deployments get a source that does not depend on ghcr availability or per-user PATs. Nothing changes on the ghcr side.

Fixed¶

• Run the streaming output gate on every streaming path, in each path's own wire format (#1087, closes #1076). StreamingOutputGate had exactly one production construction site, the OpenAI chat streaming handler, so output guardrails silently did not run on any other streaming surface: /anthropic/v1/messages in all five reachable arms, Gemini through the SSE pipeline, mid-stream fallback, all four /v1/responses streaming strategies, the thinking-pattern transform, Bedrock runtime and converse, every Unix-socket streaming constructor, and the responses_only chat bridge. Each is now gated on what the client actually receives. A new Anthropic-wire driver renders a well-formed Anthropic terminal on a block (close the open block, refusal text block, message_delta, message_stop, or a single error event under block_behavior: error); the SSE pipeline gains an output-gate stage between the observability tap and the caching tap, with a shared veto so a blocked or masked stream is never stored; MidStreamFallbackContext carries one gate across every fallback hop so text held through a backend switch is checked as one completion, and a policy block completes the stream rather than triggering a fallback retry; the Responses passthrough arm defers the response.output_text.done and response.completed mirror events past the end-of-stream check and rewrites them from the gate's client-visible text, so a mirror cannot leak what the deltas masked. Gate construction also stopped hardcoding the global guardrails.mode: a shared build_streaming_gate resolves the per-route mode, the per-route enabled, and the bypass_api_keys allowlist, so a route overriding to enforce under a global monitor now gets an enforcing gate instead of a logged check that never blocks, and a disabled route or a bypassed key builds no gate at all instead of one that delays the whole stream only to short-circuit every check to Allow. stream_with_auto_backend_selection is the one out-of-scope entry point (library-only, no HTTP route registers it); its rustdoc records the decision and it warns once per process if an embedder reaches the ungated arm with guardrails configured. Monitor mode costs one output provider call per stream on each newly gated surface, which the guardrail guides now state.
• Apply Transform verdicts on the streaming output path, so output PII masking is no longer silently bypassed on the default traffic shape (#1075, closes #1074). finalize and run_window_check collapsed every verdict to is_block(), so the Transform a mask action produces fell into the allow branch: the original plaintext chunks were released and the masked text discarded, while the same request masked correctly when non-streaming. Held chunks are now released rewritten to carry the redacted text, with per-chunk source-shape detection so one gate serves both OpenAI choices[].delta.content and native-Anthropic content_block_delta payloads, and framing events (role opener, tool-call deltas, finish_reason, usage, message_start, ping, message_stop, [DONE], and anything unparseable) pass through byte-for-byte and in position. Under buffer_full that is a complete redaction, since the whole completion is held. Under chunked the masked window is stripped of its longest common prefix against the already-released context before release and spliced back over the checked region of the accumulator; without that splice the next window's context re-includes the original text, the provider redacts it a second time, and the client sees a duplicated placeholder. The built-in PII provider now overrides check_streaming_chunk instead of inheriting a default that returns Allow for every chunk, which had left it inert under streaming_mode: chunked. streaming_stream_first: true, and enforce with streaming_mode: passthrough, release before checking, so a masking verdict has nowhere to land; the gate warns once per stream and leaves the sent text alone rather than re-sending a masked copy. For OpenAI n > 1 the masked text collapses onto choice 0, which is documented: mangled for multi-choice streams, never leaking.
• Keep checking streamed output past the 4 MiB gate buffer cap (#1082, closes #1078). Reaching the cap failed open: the gate switched to passthrough, flushed everything it was still holding with no check having run, and released the rest of the stream unchecked, so one long completion turned output blocking and PII masking off for that entire response with a single warn! line as the only trace. The cap now degrades to chunked instead. The full-text check runs over everything held and the verdict is applied in full (a block cuts the stream, a Transform redacts the held chunks before release, only a clean verdict releases them), stream_first is forced off for the remainder because buffer_full was configured on the promise that output is checked before release, and the already-checked history is compacted down to the trailing streaming_context_size window so memory stays bounded while live checking continues. chunked streams now compact rather than degrade, so their behavior is otherwise unchanged. A new guardrail_stream_buffer_cap_trips_total{strategy,outcome} counter and one warning fire once per stream. The gate also stopped keeping a third full copy of the stream that existed only to measure the other two.
• Observe streamed output in monitor mode (#1086, closes #1079). Monitor performed zero output-stage evaluation on streaming responses: StreamingOutputGate::new folds every streaming_mode down to Passthrough in monitor (correctly, since monitor must never hold or cut), that arm forwarded without recording, and finalize's passthrough arm is guarded on a non-empty accumulator, so nothing was ever checked. Monitor is how a team decides whether a policy is safe to enforce and streaming is the default for chat UIs and agents, so a team could run monitor for a week, see no output-stage verdicts, and conclude their traffic was clean. The passthrough arm now accumulates when the mode is monitor, keeping only extracted text and never a held chunk, so nothing is held, delayed, cut, or rewritten and the end-of-stream check produces the verdict the rustdoc and the guardrail guides already promised. Enforce with streaming_mode: passthrough stays a true zero-check, zero-provider-call path. Monitor cannot take the cap's chunked fallback, because checking a window before releasing it means holding the stream, so it stops observing at 4 MiB and the truncation is counted as guardrail_stream_buffer_cap_trips_total{strategy="monitor",outcome="truncated"}. is_active() is no longer a safe skip signal and its rustdoc now says so: a monitor gate answers false and still needs every chunk. Monitor results recorded by older releases carry no information.
• Inspect array-shaped streaming delta.content, and bring reasoning text into output-guardrail scope behind a new opt-in (#1083, closes #1080). The streaming extractor skipped an array-shaped delta.content that its non-streaming twin read. A backend survey found no supported backend emits that shape today, so this closes an inspection asymmetry as defense in depth rather than fixing a live leak, but a future backend or proxy that streams content parts is now inspected and masked instead of bypassing the output stage. Reasoning and extended-thinking text was invisible to the output stage by accident rather than by decision, and is now inspectable through guardrails.inspect_reasoning (bool, default false), covering reasoning_content on both the streaming and non-streaming OpenAI-shaped paths and thinking_delta and thinking blocks on the native Anthropic path. It is off by default because reasoning roughly doubles the text volume providers see, which costs latency and, for network-backed providers, money, and it deliberately covers the non-streaming path too, since enabling it for streaming alone would recreate the very asymmetry the fix removes. The scope is an explicit TextScope threaded through every extractor, rewriter, and transform applier on both paths, and a table-driven test asserts extractability and rewritability agree across every chunk shape under both scopes, so the two sides cannot drift apart again. Configuration-assistant schema 1.28.0 to 1.29.0.
• Redact the verdict payload from guardrail debug logs (#1081, closes #1077). GuardrailService::evaluate logged the aggregated verdict with ?aggregated, Debug-formatting GuardrailVerdict and printing Transform::new_content (and, for a block, Block::reason) in full, bypassing the audit module's deliberately text-free design. Both debug! sites now log a redacted() view that prints only the verdict's shape: variant name, category, and score, never the payload. With the built-in PII provider the transformed text is already redacted, so this is log hygiene and defense in depth; the exposure it closes is a custom or self-hosted provider that puts raw content into either field.
• Guard test_legacy_routing_section_is_ignored_on_load with ENV_TEST_MUTEX (#1085, closes #1084). It was the only test in its module that read environment-sensitive config without holding the lock, so it raced the dozen or more siblings that set CONTINUUM_BACKEND_URLS and failed intermittently in CI once the guardrail work reshuffled parallel test scheduling. Test-only, and latent since the test was introduced in #1027.

Dependencies¶

• Bump the minor-and-patch group with 6 updates (#1073): tokio 1.53.0 to 1.53.1, tokio-util 0.7.18 to 0.7.19, clap 4.6.3 to 4.6.4, tokio-stream 0.1.18 to 0.1.19, aws-config 1.9.0 to 1.10.1, and libc 0.2.186 to 0.2.189.
• Bump actions/setup-python from 6 to 7 in the CI workflows (#1072).

v1.15.5 - 2026-07-24¶

Restore stateless multi-turn tool calling on /v1/responses for clients that replay the model's own output items, and stop startup from gating listener binding on unbounded backend probes.

Added¶

• Add health_checks.block_startup (bool, default true) so listeners can bind before backends are probed (#1066, closes #1065). At the default the startup sequence is byte-for-byte identical to before: connection pre-warm and the initial health-check round run inline and gate listener binding. Set to false, the listeners bind first and both steps run in background tokio::spawn tasks that log completion, so /health and /v1/models answer immediately while per-backend health converges through the admin and model endpoints. This matters when an orchestrator polls /health under a readiness cap (Backend.AI GO uses 30s) and a single unreachable or stalling backend pushes the pre-bind window past it. /health is a router-liveness probe that never reported backend health, so it stays meaningful during background warm-up. Documented in the English and Korean health-and-caching, configuration, and deployment guides, the example config, and configuration-assistant guidance (schema 1.26.0 to 1.27.0).

Fixed¶

• Keep replayed Responses input items convertible, so stateless multi-turn tool calling (store: false) works again on conversion backends (#1064, fixes #1063). A v1.15.0 regression (#1051) treated plain id and status as native-only metadata alongside the beta phase/agent/caller fields, so the message, function_call, and function_call_output items that every OpenAI-compatible client replays verbatim (including the model's own item id) became opaque passthrough values, and convert_to_anthropic and convert_to_gemini rejected the whole request with Request field 'native Responses input item' ... would lose it. The preserve-exactly heuristic now triggers only on the genuinely lossy beta fields, and only bounded standard id strings and recognized status values count as droppable replay metadata. Oversized, malformed, or unknown metadata stays byte-preserved, native-gated, and accounted for by request-size validation, so conversion still fails closed.
• Serialize replayed reasoning items byte-faithfully (#1064). encrypted_content, content, and status on a reasoning input item now skip serialization when absent, so a forwarded item no longer gains explicit nulls that OpenAI rejects with Unknown parameter: 'input[N].status'.
• Apply the configured health_checks.timeout as a per-request timeout on the startup health-check path (#1066). HealthChecker::with_shared_client reuses the router's shared reqwest client, whose global timeout is the 600s streaming total, so a backend that accepted a connection and then stalled could hold a health-check round for up to 10 minutes. The timeout now bounds every check on both the shared-client and dedicated-client constructor paths; the Duration is read out of the config lock and the guard dropped before the request, so no lock is held across the await.
• Bound the generic connection pre-warm HEAD request at 10s (#1066), matching the cap the Anthropic branch already used, instead of inheriting the shared client's 600s ceiling.

v1.15.4 - 2026-07-23¶

Accept the Anthropic-native x-api-key header in standard blocking-mode authentication, and key the hub policy hash, guardrail bypass allowlists, and per-key stats off the same presented credential.

Fixed¶

• Accept x-api-key in standard blocking mode (#1062, closes #1061). With api_keys configured in blocking mode, the standard (non-AppProxy) path of the dynamic auth middleware only read Authorization: Bearer, so Anthropic-native clients (the Anthropic SDKs, Claude Code with only ANTHROPIC_API_KEY set) presenting a valid key in x-api-key got 401. A shared extract_presented_api_key helper (Bearer preferred, x-api-key fallback, whitespace trimmed, raw credential never logged) now feeds the middleware in both blocking and permissive mode, the AppProxy ROUTER key extraction delegates to it, and the Anthropic messages, count-tokens, and models handlers derive the authenticated user from the middleware-attached auth context instead of re-requiring x-api-key. Removing the handlers' early return also means an authenticated request with a malformed anthropic-version header now gets 400 instead of silently passing.
• Key every credential consumer off the presented credential the auth middleware validated (#1062). The control-plane hub key hash now trims a whitespace-padded Bearer credential exactly like the auth path, so a padded Authorization header can no longer authenticate while evading hub rate, budget, and tier enforcement and usage attribution. The guardrail bypass allowlists (non-streaming, streaming, and Responses-only paths) and per-key stats attribution read the same presented credential, so x-api-key-only clients get correct bypass and stats, and a stray non-authenticated x-api-key that matches a bypass key cannot trigger a bypass when the request authenticated under a different Bearer key.

v1.15.3 - 2026-07-22¶

Enforce Qwen3Guard verdicts in the self-hosted guardrail classifier, isolate the AppProxy feature graph from the Admin API, fix empty tool-call arguments on the Chat-to-Responses bridge, and refresh five dependencies including the rmcp 2.x migration.

Added¶

• Add a native Qwen3Guard template to the self-hosted classifier so its structured plaintext verdicts are enforced instead of silently allowed by an incompatible parser (#1060, closes #1059). The template parses bounded, line-oriented Safety: and Categories: output for Safe, Unsafe, and Controversial results, maps Qwen's official taxonomy with case-insensitive selection and mapped thresholds (categorizing injection positives as jailbreaks), and adds a configurable controversial_action: flag | block | allow with a non-blocking flag default. Malformed output follows the effective fail-open/fail-closed policy. The classifier options, Qwen parsing, and tests are split into bounded modules under 500 lines each, and the configuration is documented in the English and Korean guardrail guides, the example config, and configuration-assistant guidance.

Changed¶

• Make the AppProxy feature graphs compile in isolation without transitively activating the Admin API (#1058, closes #1053). The process-wide config-mutation lock moves into infrastructure::config and CIDR-aware IP allow-list evaluation into a feature-independent HTTP middleware helper, both with compatibility re-exports, and appproxy-common no longer enables admin. An isolated --no-default-features AppProxy build therefore neither compiles nor mounts the Admin route tree, while official release builds still receive Admin through the default full feature set, so their behavior is unchanged. Isolated appproxy-common, appproxy-router, and control-plane,appproxy-router checks are added to CI.

Fixed¶

• Preserve function-call arguments when a Responses stream provides the complete JSON only on response.output_item.added or done events, so Chat Completions tool calls no longer reach clients with an empty arguments string (#1057, closes #1056). The converter buffers arguments seen on output_item.added per output index, tracks deltas already forwarded to each tool call, and reconciles the terminal events by emitting only the longest prefix-compatible missing suffix.

Dependencies¶

• Bump rmcp from 1.8 to 2.2 and migrate the MCP resource listing to the non-annotated Resource/ResourceTemplate builders, since RawResource, RawResourceTemplate, AnnotateAble, and no_annotation() were removed in the 2.x API. Bump tower-http to 0.7 (cors layer), object_store to 0.14 (s3-cache), aws-smithy-eventstream to 0.61 (bedrock-sigv4), and tokio-tungstenite to 0.30 (control-plane policy stream).

v1.15.2 - 2026-07-21¶

Ship the Windows release binary with the MSVC C runtime statically linked so it launches on a clean Windows install, and refresh the dependency tree.

Fixed¶

• Statically link the MSVC C runtime in the x86_64-pc-windows-msvc release build so the shipped continuum-router.exe no longer requires the Visual C++ Redistributable (#1055). Every Windows release since v0.23.1 linked the CRT dynamically, so on a Windows machine without the VC++ 2015-2022 Redistributable the binary failed to start with a missing VCRUNTIME140.dll error. The release workflow now builds the Windows target with -C target-feature=+crt-static, and because that job builds with an explicit --target, the flag applies only to the final binary and not to host build scripts or proc-macros. macOS and Linux builds are unchanged.

Dependencies¶

• Bump a group of 12 minor and patch dependencies (#1052): tokio 1.52.3 to 1.53.0, redis 1.3.0 to 1.4.1, uuid 1.23.5 to 1.24.0, serde 1.0.228 to 1.0.229, serde_json 1.0.150 to 1.0.151, thiserror 2.0.18 to 2.0.19, async-trait 0.1.89 to 0.1.91, clap 4.6.1 to 4.6.2, futures-util 0.3.32 to 0.3.33, fastrand 2.4.1 to 2.5.0, regex 1.13.0 to 1.13.1, and toml 1.1.2 to 1.1.3.

v1.15.1 - 2026-07-20¶

Align the GPT-5 family across the OpenAI Chat Completions and Responses APIs. This release registers the GPT-5.6 models and GPT-5 Pro, validates reasoning_effort against the exact per-model matrix without silent downgrades, and preserves compatible request fields across the Chat-to-Responses bridge instead of dropping them.

Added¶

• Register the GPT-5.6 Sol, Terra, and Luna models with family aliases, wired into OpenAI model normalization and reasoning-effort handling (#1051). Model metadata covers the new generation, and the family aliases resolve to the concrete models.
• Add GPT-5 Pro as a Responses-only model so /v1/chat/completions transparently bridges to /v1/responses (#1051). GPT-5 Pro is Responses-only upstream, but the router keeps Chat Completions compatible through its existing Chat-to-Responses bridge.
• Expand the Responses API schema for reasoning context and mode, text verbosity, prompt caching, programmatic tool calling, multi-agent items, native replay metadata, safety identifiers, context management, and tool-call limits (#1051).

Changed¶

• Validate OpenAI reasoning_effort against the exact GPT-5 generation and model matrix on both streaming and non-streaming Chat Completions requests (#1051). The original gpt-5 family accepts minimal, GPT-5.1 starts at none/low, GPT-5.2 through GPT-5.5 add xhigh, GPT-5.6 adds max, and gpt-5-pro accepts only high. The router never rewrites auto, xhigh, or max to another effort, and OpenAI models no longer pass through the former Gemini-normalization interference.
• Preserve compatible Chat fields across the Chat-to-Responses bridge, including flat verbosity, metadata, safety identifiers, and prompt-cache controls, rejecting invalid values instead of dropping them (#1051). Native OpenAI persisted-response state and beta headers are kept intact, while local session state is resolved only for conversion strategies and native previous_response_id values are forwarded unchanged. Prompt-cache TTL validation accepts the documented 30m value.

Fixed¶

• Fail closed with unsupported_request_parameter when native-only Responses controls would otherwise be silently dropped by non-native conversion paths (#1051). Native Responses requests carrying GPT-5.6-only controls bypass response caching and are rejected on conversion-only backends rather than degraded silently.

Breaking Changes¶

• OpenAI Chat Completions now rejects unsupported model and effort pairs with 400 invalid_request_error instead of silently rewriting the effort (#1051). A client that previously relied on the router adjusting an out-of-range reasoning_effort receives an explicit error and must send a value the target model supports.

v1.15.0 - 2026-07-20¶

A configuration truth pass. A manual code audit of the v1.14.0 tree found configuration knobs and endpoints that the schema accepted and the manuals documented but that no production code path honored. This release wires each one into the runtime so the documented behavior actually takes effect, adds the environment overrides and interpolation the manuals already described, and finishes with a documentation alignment audit and a post-merge hardening pass.

Added¶

• Interpolate ${VAR} environment references in every string-valued config field, not just the previously hardcoded secret/URL/path set (#1039, closes #1032). The loader walks the parsed YAML/TOML tree and resolves ${VAR}, ${VAR:-default}, and a $$ literal-dollar escape before typed deserialization, with variable-name validation and bounds against oversized values and excessive substitutions. .env files load before parsing so their values feed interpolation, an unset required reference aborts startup and hot reload with a path-aware error, config validate reports unset references as warnings, config show --resolved and config diff interpolate (diff still redacts secret-named fields), and config show plus the MCP surface keep references literal and read no environment values.
• Add a validated model_aggregation config section wired into the runtime (#1042, closes #1022). The section maps to the aggregation service, cache TTLs, fetcher and entry limits, deduplication strategy, force-refresh gate, and background refresh task, so allow_force_refresh and background_refresh are operator-controllable instead of pinned to defaults. Reload class is restart, because the cache and services are built at startup. Schema count moves to 35 with the addition.
• Read the CONTINUUM_SELECTION_STRATEGY environment override in the config loader (#1037, closes #1023). It was documented but never read; the loader now parses all six strategies with the same spelling rules as --selection-strategy, an unrecognized value fails the config load with a clear error instead of being silently ignored, and precedence is --selection-strategy > CONTINUUM_SELECTION_STRATEGY > file value > RoundRobin default.
• Read the four CONTINUUM_FILES_AUTH_* Files API auth environment overrides in the config loader (#1046, closes #1033). CONTINUUM_FILES_AUTH_METHOD, CONTINUUM_FILES_AUTH_SCOPE, CONTINUUM_FILES_ENFORCE_OWNERSHIP, and CONTINUUM_FILES_ADMIN_ACCESS_ALL were documented but inert; they now apply to files.auth, construct a default FilesConfig when the file has no files: section, and the two booleans accept only literal lowercase true/false, rejecting 1, 0, yes, no, and mixed case with a clear error.
• Emit the X-Fallback-Attempts response header the router already advertised (#1035, closes #1024). The fallback path discarded the attempt count before building the response; the header now reports the total model attempts including the primary, appears only when a fallback backend served the response (minimum value 2), and is gated by the per-model notify_on_fallback setting. Streaming responses flush headers before the body, so mid-stream recovery stays observable through the streaming_fallback_* metrics instead.

Changed¶

• Align the English and Korean manuals, CLI --help, generated configuration guidance, Debian packaging text, Docker examples, and the manpage with the implemented router surface (#1034). Historical, aspirational, and nonexistent-feature claims were removed from user-facing docs while real endpoint, feature-gate, authentication, fallback, circuit-breaker, environment-expansion, model-aggregation, and hot-reload boundaries were documented, and Admin and MCP hot-reload metadata and regression tests were aligned with actual wiring. This audit surfaced the follow-up findings that the rest of this release closes.
• Bump the Debian Build-Depends Rust floor from 1.75 to 1.96 to match the lablup PPA, and reconcile the packaging docs (#1021, closes #1020). Corrects the false edition-2024 MSRV claim (the crate is edition 2021), the wrong Noble codename mapping, and the distro guidance to state that stock Ubuntu series do not ship Rust 1.96 and the lablup PPA must be enabled.

Fixed¶

• Drive the circuit-breaker state machine from ordinary proxy traffic (#1044, closes #1028). The breaker was implemented, configurable, hot-reloadable, and Admin-visible, but no LLM proxy request drove or consulted it, so enabling it did not protect the routing path. Selection now excludes open circuits, a pre-dispatch gate admits or rejects, and terminal outcomes are recorded across Chat, Completions, Responses, Embeddings, Images, Rerank, and every fallback hop plus standard streaming. Upstream 5xx and configured failure_status_codes open the circuit, timeouts honor timeout_as_failure, transport errors are unconditional failures, client-side 4xx are not recorded, and admission rejections become a retryable error so selection and fallback advance to another backend before returning the standard 503.
• Apply retry.* and timeouts.* hot-reload revisions to request execution (#1045, closes #1029). Both were classified hot-reloadable, but the request path read only the startup snapshots, so a reload changed the config and reported success while effective behavior stayed at startup values. Each request surface now snapshots the live policy and timeout budget at entry, keeping one coherent policy per in-flight request; the reqwest client connection and overall timeouts remain restart-only because they are baked in at client construction.
• Apply live selection_strategy and prefix_routing.load_factor_epsilon changes to the running BackendPool (#1038, closes #1025). A watcher-driven strategy change was detected and advertised as applied, but HotReloadService never pushed it to the live pool, so routing kept the startup strategy. Both are now shared lock-free cells swapped atomically while preserving backend membership, per-backend stats, in-flight accounting, and hash-ring safety, and the Admin, MCP, and docs surfaces report them as immediate.
• Wire the accepted-but-inert prefix_routing.virtual_nodes and anthropic_cache_control_injection knobs into the runtime (#1041, closes #1026). virtual_nodes now sizes the consistent-hash ring (rebuilding the cache on change), anthropic_cache_control_injection gates automatic Anthropic cache-control injection as a fleet-wide master enable, both are hot-reloadable, and /admin/prefix-routing/stats reports the active pool values.
• Inject the shared API key store into Admin authentication so admin.auth.method: api_key works (#1036, closes #1030). The Admin routes were mounted with AdminAuthState::without_api_key_store(...), so every api_key request hit the empty-store branch and returned 500 Internal Server Error. Admin API-key auth now validates against the same store as ordinary /v1 auth and Admin key CRUD (both Authorization: Bearer and X-API-Key are accepted), enforces required_scope, and honors optional allowed_ips; disabling, rotating, expiring, or deleting a key stops it authenticating immediately with no restart.
• Make POST /admin/config/apply truthful (#1047, closes #1031). The endpoint was a placeholder: it recorded the unchanged current config as a new history version, never published through config_sender, and could report hot_reload_triggered: true without triggering anything. It now takes an optional full config candidate; with no candidate it is an explicit no-op (no history version, hot_reload_triggered: false), and with a candidate it validates, diffs against the running config, and publishes atomically under the modification lock only when it differs. hot_reload_triggered is true only after a successful publish of a changed config, updated_sections/requires_restart reflect the real diff, hot_reload: false previews the diff without applying, and repeated no-candidate calls never create empty history versions.
• Add a metrics::security stub so src/errors.rs builds under --features embed (#1048). The always-compiled error path calls crate::metrics::security::truncate_at_char_boundary, which lives behind the metrics feature; the embed stub module gained the submodule with the real UTF-8-boundary-safe truncation logic. The embed feature is not in the CI matrix, so the break (introduced by #1017) was caught only by running the off-feature build matrix locally.
• Scope the capacity integer-only heartbeat assertion to the capacity fields (#1040). The release feature-set CI job failed deterministically since 1.14.0 because the assertion checked the whole serialized heartbeat for .0 and tripped on the legitimate dot-zero in router_version "1.14.0"; it now verifies only that the three observed_*_capacity values serialize as JSON integers.
• Consolidate post-merge corrections for the configuration and routing changes above (#1050). Made circuit-breaker admission, half-open accounting, and hot reload coherent; snapshotted the retry policy and hardened hash-ring generations; exposed the complete Admin config catalog with truthful reload metadata; and covered every streaming transport while synchronizing configuration documentation.

Removed¶

• Remove the inert legacy routing config section (#1043, closes #1027). The schema accepted it and the docs described it as advanced routing overrides, but no production code read it, so setting routing validated while none of its values affected traffic. Live selection uses top-level selection_strategy, per-backend weight/models, and smart_routing. A config that still carries a top-level routing: key keeps loading (the unknown key is ignored) and the loader and config validate now emit a migration warning pointing at the live settings instead of silently doing nothing. Schema count returns to 34 and schema_version moves to 1.26.0.

v1.14.0 - 2026-07-19¶

Added¶

• Overhaul the embedded WebUI into a full admin console (epic #931). The single-file SPA is vendored (Alpine.js, uPlot, a prebuilt Tailwind stylesheet), drops every CDN dependency so it runs air-gapped under a strict default-src 'self' CSP, and is restructured into a per-page module registry (#948). A login flow with credential validation and session UX gates the console (#949), and a GET /admin/capabilities endpoint drives page-level feature gating so panels for disabled subsystems degrade instead of breaking (#950). New management pages cover guardrails (#951), cache operations (#952), smart routing (#953), usage analytics (#954), prompts (#955), files (#956), models and routing with catalog, aliases, and fallback chains (#957), backend deep-edit forms with an on-demand health probe (#958), schema-driven config forms with diff preview and hot-reload indicators (#960), and integrations status for control-plane, ACP, and AppProxy (#961). Config PATCH saves that would silently drop diff-shown removals are blocked (#962), API keys gain allowlists, annotations, and usage drill-down (#963), the dashboard adds trends, uptime, and circuit detail (#964), and a UX pass adds a command palette, keyboard shortcuts, responsive layout, and accessibility fixes (#965, #966, #1004).
• Add a typed operator request-parameter policy across every LLM API (epic #990). A protocol-independent policy engine (#996) applies operator-defined defaults, overrides, and min/max clamps for the seven canonical parameters (temperature, top_p, max_tokens, presence_penalty, frequency_penalty, top_k, min_p) with typed values and validated ranges, exposed through config (#1006) and enforced on the Chat Completions and Completions proxy path (#1007), the native Anthropic Messages path (#1008), and the Responses API (#1009). Mutations emit deterministic, value-free metadata, and an invalid policy is distinguished from an invalid request field. Runtime paths and the local-bound preservation guarantee were hardened after audit so a Hub tier limit can only tighten, never loosen, a locally configured bound (#1013, #1014, #1015).
• Enforce Continuum Hub tier request-parameter limits on the Router data plane (#1002, Hub #299 / PR #314 and #301 / PR #319). The additive version-0 wire contract carries integer-microunit bounds for seven canonical parameters, exact alias-resolved model scopes, a canonical public tier digest, and typed capability/active/rejection evidence. A full envelope is validated and converted before one atomic last-known-good swap; invalid, stale, or digest-mismatched candidates never partially apply. Each authenticated Hub key freezes the tightest intersection of Hub and deployment-local limits once before cache identity, substitution, arbitrage, backend selection, retry, or fallback across Completions, Chat Completions, Responses, and Anthropic Messages. Authoritative clears, mixed-version Hubs, feature-off/default builds, and Hub outages preserve their documented behavior, and logs, metrics, and heartbeat status remain value-free. The vendored protocol is pinned to Hub PR #319 merge 59387265e755bf02a9dbbb238da369e36d566d49.
• Apply Continuum Hub fleet request-parameter configuration through the outbound-only control-plane agent (#1001, Hub #302 / PR #317). The separate control_plane.config_sync.enabled opt-in advertises a masked, bounded structured request_params capability, fetches revision-bound public scalar content, verifies the canonical Hub-compatible digest and full effective config, layers Hub leaves below explicit local pins, and publishes one atomic hot-reload snapshot. Dry runs, rollback revisions, duplicate delivery, lost acknowledgments, reconnects, and restarts are idempotent through a private last-known-good state/outbox; transient Hub failures never become terminal rejections, in-flight requests keep their captured config, and snapshots, logs, and apply results remain value-free. The vendored protocol stays additive at version 0 and is pinned to Hub PR #317 head 771591f.
• Estimate per-backend observed RPM, input TPM, and output TPM for Continuum Hub inventory (#972). The control-plane agent promotes a rolling 60-second throughput observation only after a confirmed transient upstream 429, requires a full window with at least two successful completions, requires complete token accounting before emitting either token dimension, and retains a 15-minute expiring high-water mark. Cold-start, idle, lightly loaded, token-incomplete, quota-exhausted, and stale backends remain absent rather than reporting 0; observed capacity never reads or merges the operator-declared capacity stored by the hub. The same optional estimates are visible through /admin/stats/backends in control-plane builds.
• Add router-managed Gemini context caching (#929, closes #928). When gemini_context_cache.enabled is set, the router caches large, stable system-prompt prefixes sent to type: gemini backends as Google cachedContents resources and reuses them across requests, transparently to OpenAI-compatible clients, guaranteeing the cached-token discount and cutting prefill latency for workloads that resend a large fixed system prompt. It mirrors the existing Anthropic cache_control injection on both the compat and native Gemini surfaces, keyed by (backend, model, prefix digest) with single-flight creation, negative caching of API-rejected prefixes, LRU eviction, and TTL extension on hit. Default off: explicit caching bills storage per token-hour, and with the feature disabled it is a strict no-op with no map allocation, background task, or added per-request latency. A continuum_gemini_context_cache_* metric family reports requests, creates, failures, entries, evictions, and cached tokens.
• Execute cost-bounded, Hub-synced synthetic provider/model probes (#969, #1019). When the control-plane feature is compiled and control_plane.enabled, control_plane.policy.enabled, and an enabled Hub ProbePolicy with targets all hold, the agent runs one non-streaming probe call per listed (provider, model) target on the policy cadence through the internal Backend::execute_probe_chat_completion seam (normal transformation and effective auth, structured outcomes, no substitution, fallback, cache, or retries, and no UsageEvent), bounded by a per-router monthly budget ceiling persisted in an owner-only sidecar next to control_plane.state_file (0 is unlimited; positive ceilings reserve before dispatch and fail closed without price or state). Results post to POST /api/agent/v1/probes with stable report_id/probe_id redelivery and a non-fatal rate-limited 404 path for older hubs. No config field is added, and every gate off means zero probe behavior, cost, or wire traffic.
• Report per-backend served models (#920), per-backend auth and quota rejection counters (#925), and token-supply and saturation telemetry (#968) to Continuum Hub inventory, and add an opt-in policy.unmatched_key reject strict mode that fails closed on an unrecognized key instead of falling open (#923). All behind the control-plane feature with PROTOCOL_VERSION unchanged.
• Add a GET /admin/capabilities endpoint that reports compile-time Cargo feature flags and runtime-enabled subsystems in one read-only response, gated by the existing admin auth and audit middleware (#950).

Changed¶

• Bump 12 minor and patch dependencies in one group (#976), including regex 1.12.4 to 1.13.0, aws-config 1.8.18 to 1.9.0, http-body 1.0.1 to 1.1.0, rust-embed 8.11.0 to 8.12.0, plus uuid, bytes, lru, and socket2.

Fixed¶

• Wire live Prometheus metric sources and reconcile the legacy collectors for the router observability epic (#973; #982, #986, #993). Metric families that were registered but never written now have production writers, legacy lazy_static counters are synced from the modern metric sources, and the English and Korean metrics references document the exported metric and label names.
• Report observed per-backend capacity through the control-plane inventory (#972, #983), reading the single in-flight gauge rather than a parallel counter.
• Hide internal backends across every user-facing API surface so a backend:-referenced internal model never leaks into model listings or the routing candidate set (#985).
• Harden the epic #990 response cache identity so cache keys cannot collide across distinct effective requests (#1018).
• Surface upstream error bodies on the Unix-socket streaming path (#1017, closes #1016). Non-2xx backend responses on the plain, thinking, and Anthropic Unix-socket streams now drain the error body under an 8 KiB cap and a two-second budget bounded by the remaining request timeout, reaching error-body parity with the TCP streaming path instead of discarding the backend's explanation.
• Account for Unix-socket usage in streaming responses so control-plane usage records are not dropped on that transport (#927).
• Harden control-plane observability and strict auth (#926), reconcile a renamed backend name during health sync (#921), stamp an unattributed sentinel on fail-open usage attribution (#922), and resolve the usage provider from the live config per event rather than a stale snapshot (#919).
• Harden epic #973 observability state after the merged implementation audit. Capacity samples are now kept in timestamp order so concurrent, out-of-order completions cannot retain stale throughput or surface future observations; backend removal and same-name endpoint replacement unconditionally reconcile name-keyed capacity and credential-health state even when health checks are disabled; and the registered aggregate model-usage, token, and final-timeout counters now have production writers. The English and Korean metrics references now document the exported metric and label names.
• Dispatch live HTTP traffic through the configured selection_strategy (#975). No HTTP request path ever called the pool selector: chat completions, streaming chat, the Responses API (non-streaming, streaming, and compact), Anthropic Messages and count_tokens, and image generation each picked the FIRST healthy candidate in model-lookup order, which left RoundRobin, WeightedRoundRobin, Random, LeastLatency, ConsistentHash, and PrefixAwareHash all inert for live traffic. All of these paths now share one selection seam: each keeps its own pre-filters and error semantics (model lookup, internal-backend filter, per-key allow-list, retry-state exclusion, and the model-not-found / 403 / all-unhealthy distinctions are unchanged), the seam health-filters the surviving candidates so an unhealthy backend can never be selected, and the final pick goes through the new candidate-restricted pool entry point BackendPool::select_from_candidates, which shares the strategy core with select_backend_with_context and keys the consistent-hash ring cache by the participating backend names so per-model candidate subsets each hash against a correct ring. Prefix-aware routing is live end to end: with prefix_routing.enabled, the prefix key is extracted from the request body (OpenAI messages[] and Anthropic top-level system shapes) and drives PrefixAwareHash CHWBL placement over the live in-flight gauge from #971, and the KvOverlapScorer is registered at startup when a KV cache index is also configured. Traffic-distribution behavior change: deployments that configured a non-default selection_strategy now actually get it enforced, and default deployments change from "always the first candidate backend" to real round-robin across the healthy candidates for each model.
• Wire per-backend in-flight tracking into the production request path so the CHWBL load cap is no longer degenerate (#971). BackendStats.in_flight_requests previously had no production writer, so the bounded-load half of the PrefixAwareHash selector computed its cap over a permanent zero and could never bind. The backend pool now owns an always-compiled RAII in-flight tracker written at every dispatch seam (chat proxy, streaming chat, Responses API, Anthropic Messages and count_tokens, image generation, image edit), with streaming guards riding the response body until the last frame or a client disconnect. The cap formula now counts the request being placed, ceil((total_in_flight + 1) * (1 + epsilon) / backend_count), and prefix_routing.load_factor_epsilon (default 0.25) finally reaches the pool, so the documented knob takes effect. The gauge is the single source of truth: the admin prefix-routing/stats endpoint and the control-plane inventory's per-backend active_requests now read it, replacing the parallel control-plane gauge from #968, and both report real load instead of zeros.
• Note on scope: this fixes the gauge the CHWBL selector reads, not which code paths invoke that selector. The router's HTTP request paths still dispatch to the first healthy backend serving the model, so selection_strategy (CHWBL included) does not yet influence live traffic distribution and no request will be routed differently after this change. Wiring the pool selector into the request funnel landed in #975 (see the entry above).
• Prevent the Unix socket integration tests from hanging and restore them in CI (#984, #992, #995).

v1.13.1 - 2026-07-10¶

Added¶

• Amazon Bedrock multi-provider support via the Converse API (#615). A new endpoint_type: converse on type: bedrock backends targets POST /model/{modelId}/converse[-stream] and reaches any chat model on Bedrock (Claude, Nova, Llama, Mistral, Cohere, AI21 Jamba, DeepSeek) through one unified wire format: OpenAI-shape requests are transformed into Converse content blocks (system-prompt extraction, toolConfig/toolUse/toolResult, inferenceConfig sampling parameters), signed with SigV4 through the shared #614 plumbing, and the binary ConverseStream event stream is decoded back into OpenAI SSE with tool-call and reasoning deltas. A hard-coded per-model capability matrix (chat, streaming, tools, vision, documents, guardrails, prompt caching, reasoning) rejects unsupported-capability requests with a 4xx naming the capability before signing, vendor-specific options pass through verbatim via extra_body / additionalModelRequestFields, remote image URLs are fetched (SSRF-validated, size-capped) and inlined as base64, and Converse-native bodies are detected and forwarded without double transformation. Gated behind the existing bedrock-sigv4 Cargo feature.
• Backend-level internal visibility flag (#908, #911). Setting internal: true on a backend keeps its infrastructure-only models (a guardrail classifier, an internal summarizer, an embeddings model) out of user-facing surfaces: internal backend names are stripped from the OpenAI /v1/models, /v1/models/extended, single-model, and Anthropic /anthropic/v1/models listings, and removed from the routing candidate set before the per-key allow-list so a user request for an internal-only model returns model-not-found (404) rather than a misleading Forbidden. Router-internal backend: references and admin surfaces still resolve internal backends. The field defaults to false and is omitted on serialize, so existing configs are unchanged.
• Accept the retiring hub key hash during a control-plane key-rotation overlap (#906). KeyEntry gains additive previous_key_hash and previous_retires_at_ms fields so a router admits both the current and the previous hash during the overlap window, retiring the previous hash at the local timestamp or when the hub omits it on the next sync, whichever comes first; a current hash never shadows the previous one and revocation still rejects both. Behind the control-plane feature; older envelopes parse unchanged and PROTOCOL_VERSION stays 0.
• Stamp streaming time-to-first-token on control-plane usage records and keep heartbeat inventory fresh under load (#912). UsageRecord gains an additive ttft_ms, measured from the same request-start instant used for latency_ms up to the first streamed byte across the chat, thinking, Anthropic, and Gemini SSE loops, clamped to latency_ms, and omitted rather than fabricated for non-streaming, local-cache-replay, and batch responses. The heartbeat loop subtracts each beat's build and send time from the interval so a slow beat under load does not drift the cadence past the hub's load-sample interval, and every beat still carries a full router inventory. Behind the control-plane feature; PROTOCOL_VERSION stays 0.

Changed¶

• Official release binaries now ship with the control-plane and appproxy-router features compiled in (#905). The Release workflow builds all six targets with --features control-plane,appproxy-router, and the Debian packages and Docker images repackage those binaries, so every distribution channel carries them. Both features stay out of the crate default/full set, so a plain cargo build from source is byte-for-byte unaffected; for packaged binaries the opt-in moves to runtime configuration (control_plane.enabled, default false, and the appproxy_router section, absent by default), so existing deployments see no behavior change. The legacy appproxy/appproxy-legacy worker remains a source-build opt-in.

Fixed¶

• Resolve clippy lint drift that red-failed the default-feature cargo clippy -- -D warnings CI step on every branch (#907). Two mechanical, clippy-suggested rewrites in the Anthropic Gemini transform and the Bedrock reasoning transform, with no behavior change.

v1.13.0 - 2026-07-09¶

Added¶

• Continuum Hub control-plane enforcement and optimization, promoting the opt-in agent from outbound telemetry (v1.12.0) to a full policy layer. Everything is gated behind the control-plane feature and fails open when no policy is loaded, so a default build has zero added dependencies, zero runtime cost, and is byte-for-byte unchanged.
• Enforce a hub-synced key table, tier rate limits (rpm plus cache-aware input/output TPM), monthly budgets, and model allowlists through a post-auth middleware that emits x-continuum-ratelimit-* headers, fed by an additive KeyUsageSnapshot protocol type and poll/WebSocket policy sync into an ArcSwap store (#877).
• Gate the exact response cache on tier cache_enabled plus the org optimization policy, enforce the x-continuum-cache and x-continuum-batch per-request override headers within tier bounds (off always wins, on never exceeds the tier), emit x-continuum-cache hit/miss/bypass outcome headers, and meter local cache hits as metadata-only events that charge no token budget (#878).
• Stamp a CacheHitType (exact/prefix/semantic, plus an open-ended unknown) on usage records alongside the already-metered cache_hit, cached_input_tokens, batched, and provider_batch_id fields; additive with a serde default and PROTOCOL_VERSION unchanged (#879).
• Dispatch batch-eligible traffic through OpenAI-style and Anthropic Message Batches endpoints on a separate batch rate pool (batch_rpm plus an in-flight batch_queue_depth gauge) that never consumes the synchronous rpm/tpm limits, report each provider job's lifecycle to the hub via at-least-once, 404-tolerant BatchStatusUpdate messages, and expose a full key-scoped /v1/batches surface (create, retrieve, list, cancel, results) behind a per-key ownership gate with an age-based reclaim backstop; inert unless control_plane.batch is enabled (#880, #895).
• Execute a prompt-prefix response cache on a keyspace disjoint from the exact-match cache for OpenAI chat and Anthropic Messages non-streaming, keeping the prompt-determining fields and dropping the sampling-bound tail, guarded to store only naturally-completed responses and serve only when the stored entry fits the new request's max_tokens (#895).
• Select the cheapest equivalent provider via price arbitrage (x-continuum-arbitrage, tier arbitrage_enabled, tri-state equivalence_classes) only after health, circuit-breaker, latency, tier, and per-request header gates pass, constraining dispatch to the selected backend so a later selection pass cannot override the policy decision (#896, closes #891).
• Compress request context router-side for managed requests (x-continuum-compression, TierLimits.compression_enabled, CompressionPolicy), deterministically shrinking chat history before responses-only routing, streaming, cache lookup/storage, and dispatch while preserving leading system/developer and recent turns, and reporting compressed/uncompressed_input_tokens without exposing content to the hub (#898, closes #892).
• Defer eligible /v1/batches submissions until the next matching UTC routing window (PolicyEnvelope.routing_windows tri-state, weekday masks, tier scoping, end-exclusive and wrap-around-across-midnight boundaries) with batch admission checked at dispatch time so no pool slot is held while a request sleeps, and add integer-only LoadSummary request/error-rate/p50/p99 counters to router inventory heartbeats plus advisory OptimizationPolicy.load thresholds (#900, closes #893).
• Substitute the requested model per hub policy on the control-plane path, keeping substitution and arbitrage metadata mutually exclusive (#894).
• Consume hub-delivered, org-scoped provider credentials (PolicyEnvelope.provider_credentials, hub M3 vault) in place of the locally configured api_key, applying rotations without a restart or dropping in-flight requests; secrets stay in memory only, are redacted from every Debug rendering, and are resolved value-only at every provider-dispatch send site (proxy, streaming, images, Responses, batch, native Anthropic, count_tokens, embeddings) so auth-header presence and OAuth/SigV4/Bedrock gating are unchanged (#901).
• Extend hub-policy enforcement and usage attribution to the native Anthropic Messages path (streaming and non-streaming, across the native, OpenAI-bridge, Responses-bridge, and Bedrock-runtime sub-paths) and streaming /v1/responses, accepting the Anthropic-native x-api-key credential and charging cache-aware token usage under the hub key_id instead of anonymous (#882, #889).

Fixed¶

• Harden tiered KV cache event routing: wire configured KV event sources into the index at startup, preserve existing affinity scores when an event only changes the storage tier, expose the full offload/reload/purge event set through metrics and admin status, constrain disaggregated fast-decode execution to the decode backend selected from the KV index, honor per-client backend allow-lists during orchestration, and return a service-unavailable error instead of a false successful prefill/decode header when the two-phase worker protocol is unavailable (#903).
• Harden the epic 864 guardrail hot-reload behavior across the config validator and the guardrail service (#864).
• Thread the model-substitution argument through the prefix cache-hit metering call sites so the tree compiles under --features control-plane after #894 and #895 merged cleanly but on divergent bases (#897).

Documentation¶

• Redesign the AppProxy worker-mode reference (English and Korean) around the new ROUTER frontend: a single binding address per worker, cluster-level model abstraction, two-level hierarchical routing, event+pull propagation of mappings and hash-only key custody, and a BEP-1053 cross-reference (#748).
• Backfill v1.11.0 and v1.12.0 into the docs/en and docs/ko changelog mirrors, which had lagged two releases.
• Correct the config-assistant sync note for the relaxed drift guard and note the CHANGELOG en/ko mirror requirement.

Dependencies¶

• Bump the minor-and-patch group with 2 updates (#883): rand 0.10.1 to 0.10.2 and arc-swap 1.9.1 to 1.9.2.

CI¶

• Relax the skill-doc version drift guard from an exact CARGO_PKG_VERSION match to a compatibility-window check, so a routine in-range version bump no longer needs a hand edit while a window-crossing major bump still fails the build for re-review.

v1.12.0 - 2026-07-05¶

Added¶

• Configuration assistant for authoring and validating config.yaml (epic #530).
• A config CLI subcommand with validate, generate, diff, and show --resolved verbs, plus the configuration-assistant skill and config templates (#829).
• An mcp-serve stdio MCP server mode (behind the mcp feature, included in default/full) that answers claude mcp add continuum-router-config -- continuum-router mcp-serve out of the box (#830).
• An IDE rule generator (scripts/generate-ide-rules.sh) and documentation for the assistant (#831), and a @lablup/continuum-router-mcp npm wrapper package (#832).
• Verification and CI drift guards that fail the build when a schema change skips the assistant doc/template update (#833), plus follow-up hardening (#834).
• Continuum Hub control-plane agent behind the opt-in control-plane feature (#875, closes #874). An outbound-only task that enrolls a router against a hub, heartbeats liveness plus a router inventory, and pushes batched, metadata-only usage records. Not in default/full, so a default build has zero added dependencies and zero runtime cost; the sole new dependency is the vendored serde-only continuum-protocol crate, compiled only under the feature.
• Backend.AI AppProxy ROUTER worker mode (epic). ROUTER wire types, events, and worker config (#815); a ROUTER event overlay with per-node applied-ack (#816); reconciliation of circuits and mappings into backends (#817); a coordinator client with router-config pull (#818); a model-level API-key gate via allowed_models (#819); and the ROUTER worker lifecycle, config wiring, and feature flags (#820).
• Claude Sonnet 5 support (#857).
• Native streaming Gemini Responses conversion via streamGenerateContent (#856).
• Two guardrail providers: reference a served backend as the guardrail model with a backend: prefix (#871), and a custom_classifier provider that returns a prompt-defined JSON verdict (#870).
• Configurable web-search provider base URL for internal mirrors (#836).
• A cli-less library build behind the embed feature for in-process iOS embedding (#850), exposing continuum_router::serve_embedded(config, addr).

Changed¶

• Consolidate the two backend connection pools into a single trait-based pool (#851, refs #812) and rename the legacy Backend struct to PooledBackend (#843, refs #511).
• Unify the dual streaming paths: extract SSE pipeline concerns into composable stream middleware (#839), route Gemini streaming through the Backend trait (#840), collapse the duplicated Anthropic streaming transform (#841), and apply the OpenAI reasoning normalization on the streaming path (#842).
• Extract a shared LLM-backend transport core for guardrails (#869), and add a native-Anthropic system gate with an alloc-free family lookup (#861).
• Relocate the current AppProxy worker to legacy (appproxy v3) and extract a shared common core (#813).

Fixed¶

• Preserve usage attribution metadata on the control-plane usage push (#876).
• Validate /v1/responses streaming requests before file_id resolution so a malformed request is rejected with the right error instead of failing during resolution (#873).
• Honor the per-key allow-list on the /v1/responses streaming paths (#863).
• Normalize Bedrock model prefixes for Anthropic capability gating (#859).
• Reject a Brave web_search base_url that carries a query string (#852).
• Harden the native Gemini Responses audio and image conversion (#853, follow-up to #780) and wire the native Gemini Responses conversion for input_audio (#835).
• Harden the epic 505 streaming prompt parity (#846) and the epic 530 configuration assistant (#834).
• Harden AppProxy ROUTER request gating (#823).
• Anchor the daily-series prune and read tests to the wall clock so they no longer flake (#814).

Documentation¶

• Document the SSE streaming pipeline overhead with a reproducible benchmark (#848) and audit the divergences between the two streaming paths (#837).
• Add backend transform consistency integration tests (#838).

Dependencies¶

• Bump the minor-and-patch group across 1 directory with 3 updates (#845) and with 2 updates (#802).
• Bump actions/cache from 5 to 6 (#844) and actions/checkout from 6 to 7 (#801).

v1.11.0 - 2026-06-21¶

Added¶

• Content-safety guardrails subsystem (epic #644), config-gated through an Option<GuardrailsConfig> that defaults to off, so an absent guardrails key leaves the feature inert and existing configs are unaffected.
• Foundation: the object-safe, async Guardrail trait with input/output/streaming check stages, a GuardrailVerdict (Allow/Block/Transform/Flag) with most-severe-wins severity ordering, a borrowing GuardrailContext, and a GuardrailsConfig schema (global monitor/enforce mode, providers with credentials by env, per-route overrides, per-API-key bypass, per-guardrail timeout_ms and on_error fail-open/fail-closed, block behavior, streaming mode, and exact and regex allow/deny lists) with a validator that rejects invalid combinations such as enforce with no providers, thresholds outside [0,1], non-compiling regex, and zero timeouts (#779, closes #644). The validator is wired into the real config load path so an invalid guardrails block is rejected at load time (#781).
• Service: a GuardrailService pipeline that runs the configured guardrails with per-guardrail timeouts and fail-open/fail-closed handling, hot-reload, and a block-response builder (#782).
• Providers behind a factory: OpenAI Moderation (#783), a self-hosted classifier (#785), PII detection and redaction (#786), and AWS Bedrock and Azure cloud guardrails (#787).
• Gating: input gating wired into the chat, Anthropic, and responses dispatch paths (#788), output gating on the non-streaming response paths (#789), streaming output gating with buffer/chunked/passthru modes (#790), and per-route policy threaded through the dispatch handlers so a guardrails.routes[<model>] override (mode, enable/disable, provider subset, thresholds) takes effect end to end, keyed by the request model id (#796).
• Controls and observability: admin runtime controls for guardrail policy (#784), Prometheus metrics and per-decision audit logging (#792), an end-to-end integration suite for input/output gating (#791), and a guardrails guide with architecture and metrics coverage plus site nav (#793).
• Per-model and daily usage-series dimensions on the admin stats API (#799, closes #798). GET /admin/stats/api-keys/{id}/models and GET /admin/stats/users/{user_id}/models return a per-model token and request breakdown, and .../{id}/series and .../{user_id}/series return an ascending daily time series of total_requests, prompt_tokens, completion_tokens, and total_tokens keyed by the UTC completion date (from/to accept Unix-millis or RFC 3339; interval defaults to day and any other value returns 400). Four cardinality-bounded DashMaps on the StatsCollector (keyed by composite keys joined with a U+001F separator that cannot occur in ids, models, or dates) back the dimensions, a series_retention_days option (default 30, serde-defaulted) prunes daily buckets older than the cutoff, and the four maps are added to the persisted snapshot as #[serde(default)] so snapshots written before this change still load.

Fixed¶

• Harden the daily-series pruning lifecycle (#800, refs #799). Restore now counts every separator-bearing composite key, including the anonymous daily and model buckets that the runtime insert path treats as non-reserved, so pruning an expired bucket after a restart no longer decrements a slot that was never reserved, underflows the AtomicUsize, and silently folds new real buckets into the overflow row. A dedicated hourly series pruner is spawned when stats persistence is off (where the snapshot task that drove pruning never runs), so the default config physically drops expired daily buckets instead of accumulating them up to the cardinality ceiling.
• Record admin stats for /v1/responses traffic (#794), correcting the stats success and failure boundary for conversion failures and strengthening regression coverage for the registered per-key and per-user admin stats endpoints.
• Handle OpenAI input_audio content blocks in the native Gemini and Anthropic transforms (#778). The Gemini Responses converter now maps audio to an inline_data part with a mime_type derived from the format (wav to audio/wav, mp3 and other to audio/mp3), and routes a data: URL through the existing inline-data path, instead of discarding it. The Anthropic Messages API has no audio content type, so the native transform now returns a ValidationError surfaced as HTTP 400 at both the non-streaming and streaming entry points instead of silently dropping the block or forwarding it verbatim.

Dependencies¶

• Bump the minor-and-patch group with 5 updates (#769): uuid 1.23.2 to 1.23.3, regex 1.12.3 to 1.12.4, redis 1.2.2 to 1.2.3, aws-smithy-eventstream 0.60.20 to 0.60.21, and aws-smithy-types 1.4.9 to 1.5.0.

v1.10.2 - 2026-06-15¶

Added¶

• Kimi K2.7-Code and GLM-5.2 model metadata (#775). Kimi K2.7-Code is a 1T-parameter MoE (32B active) with a MoonViT vision encoder and 256K context that runs only in thinking mode and returns reasoning in the native reasoning_content field, so it carries no <think> marker config. GLM-5.2 is the GLM-5 line coding flagship with a 1M context window, 131K max output, and two thinking-effort levels (High and Max); it carries the standard <think>/</think> marker config like the rest of the GLM family. GLM-5.2 standalone API pricing was not published at launch, so its input/output rates are estimated from the GLM-5/GLM-5.1 tier.

Fixed¶

• Normalize the vLLM/OpenAI-compatible reasoning field to the canonical reasoning_content on /v1/chat/completions (#776, closes #774). Newer vLLM renamed its reasoning output field from reasoning_content to reasoning (streaming delta.reasoning, non-streaming message.reasoning), and the router relayed it unchanged, so clients reading reasoning_content silently dropped all reasoning text from self-hosted vLLM reasoning models. The rename runs across every OpenAI-compatible relay path (the streaming default and thinking transformers, the unix-socket relay, the mid-stream fallback relay, and the non-streaming proxy body), only when reasoning_content is absent so an upstream already using the canonical name is never overwritten, and is scoped away from the Gemini and Anthropic handlers and the Responses API.

Documentation¶

• Note in the reasoning-effort architecture reference (English and Korean) that the router normalizes the upstream vLLM reasoning field to reasoning_content.

v1.10.1 - 2026-06-15¶

Added¶

• Per-API-key and per-user usage statistics REST API: four admin endpoints GET /admin/stats/api-keys, GET /admin/stats/api-keys/{id}, GET /admin/stats/users, and GET /admin/stats/users/{user_id}, mirroring the existing /admin/stats/models shape under the admin auth router (#772, closes #770). The StatsCollector gains per_api_key and per_user dimensions recorded from the same spawned task that updates the Prometheus llm_tokens_total counter, so the in-memory dimensions no longer depend on the metrics feature and are attributed even for failed and zero-token requests. The api_key_id is the derived, non-reversible id resolved once off the request hot path; unauthenticated requests bucket under "anonymous", and ids beyond the 1000-per-dimension cardinality cap fold into an "unknown" overflow bucket so usage is still counted in aggregate. Each GET /admin/stats/.../{id} returns 404 when the id has no recorded usage, and the optional window query param is echoed back but does not filter, since the aggregates are all-time atomic counters like /admin/stats/models. The snapshot/persist format adds both dimensions as #[serde(default)], so snapshots written before this change still load without a format-version bump.

Changed¶

• Ignore root-level .sh scripts via .gitignore so local helper scripts are not accidentally committed.

Documentation¶

• Document the admin API key management endpoints and the per-API-key/per-user usage statistics in the Admin REST API reference for both English and Korean (#773, closes #770). A new "API Key Management APIs" section covers the eight /admin/api-keys endpoints (create, list, get, update, delete, rotate, enable, disable) with request/response schemas reflecting the sk-***abcd masking and the full-value-once-on-create behavior, the ApiKeyConfig fields, the permissive vs blocking api_keys.mode semantics, and runtime-key persistence via persistence_file with hot-reload. The Statistics section gains the four new stats endpoints, the window echo, the anonymous and unknown buckets, the per-dimension cardinality cap, and the api_key_id-to-issued-key linkage.

v1.10.0 - 2026-06-12¶

Added¶

• Dynamic model enumeration for Codex (ChatGPT OAuth) backends (#753, closes #752). The Codex backend has no standard /v1/models; the router instead queries the plan-gated GET <base>/models?client_version=<ver> endpoint with the loaded OAuth token during model discovery, keeps only user-facing entries (visibility: list and an available_in_plans that is empty or matches the account's chatgpt_plan_type claim), and uses the result to populate /v1/models and routing. On any failure (network error, non-2xx, empty list) the configured models: list remains the fallback, and the behavior applies only to Codex OAuth backends.

Changed¶

• Extract a shared parse_responses_body in src/services/responses/sse.rs and a shared build_openai_chat_request_core in src/http/streaming/handler.rs, eliminating the duplicated /responses SSE-aggregation parse path and the duplicated streaming request-construction logic that existed across the Codex, passthrough, and Anthropic handler variants (#766, closes #762). No behavior change; the consolidation removes drift risk where a fix to one copy would not propagate to the others.

Fixed¶

• Route Codex (ChatGPT OAuth) chat completions to the backend's /responses endpoint (#755, closes #754). build_responses_url appended /v1/responses to the .../backend-api/codex root, and the upstream edge rejects that path with a 403 HTML page that the router surfaced as an opaque authentication error despite a valid OAuth token; the Codex root now routes through the shared URL composer that strips the internal /v1. With the path fixed, Codex rejects a bare-string input with HTTP 400 "Input must be a list", so sanitize_codex_responses_request coerces a single-message string input into a one-element item list.
• Accept an RFC3339 expires_at in the OAuth token store (#756, closes #751). The field was a plain u64, so a token store that wrote the field as a datetime string failed to deserialize, the backend silently lost its OAuth strategy, and requests went out unauthenticated, surfacing as an opaque 403 that hid the real cause. expires_at now deserializes leniently (integer or float epoch seconds, a numeric string, or an RFC3339 datetime with offset, all normalized to epoch seconds) and rejects anything else with an error naming the accepted formats; the canonical on-disk shape remains a JSON number, so existing stores and the save/load roundtrip are unchanged.
• Strip local-engine-only fields (chat_template_kwargs, thinking_budget_tokens, enable_thinking, preserve_thinking, top_k, min_p, repeat_penalty) from /v1/chat/completions requests bound for cloud OpenAI, which rejects unknown top-level keys with HTTP 400 "Unknown parameter" (#760, closes #758). The strip mirrors the existing cloud Gemini behavior through a shared field_filter module gated on api.openai.com, runs at every cloud-OpenAI chat send site (non-streaming, streaming, and per-hop in the fallback loops), never touches extra_body or reasoning_effort, and is a no-op for local OpenAI-compatible engines, preserving the backend-passthrough contract.
• Respect the configured models: selection for Codex dynamic enumeration and stamp the clean owned_by on enumerated models (#765, closes #763). When live enumeration succeeded, the operator-configured models: allowlist was bypassed, so /v1/models exposed every enumerated Codex model; the post-filter now applies uniformly, with a non-empty list intersecting the enumerated set down to the selected subset and an empty list exposing the full set. Enumerated Codex entries also carried the raw backend name as owned_by; the owner is now resolved from the backend type (openai), matching every other backend.
• Return a valid non-streaming response for Codex (ChatGPT OAuth) backends on /v1/chat/completions with stream:false (#764, closes #761). Codex's /responses endpoint only accepts stream:true + store:false; sending stream:false produced HTTP 400 "Stream must be set to true". sanitize_codex_responses_request now forces stream:true and store:false unconditionally. A second fix makes SSE detection in PassthroughService::execute_request robust to Codex responses that carry an SSE body without Content-Type: text/event-stream: a looks_like_sse sniffer checks the first non-empty line for data:/event:/: prefixes, and a failed JSON parse retries SSE aggregation once before surfacing the original parse error.
• Reconstruct the assistant message from response.output_text.delta events for Codex store:false responses (#768, closes #767). When Codex runs with store:false the terminal response.completed event carries an empty output array and the assistant text arrives only as incremental response.output_text.delta events; the SSE aggregation now accumulates those deltas and synthesizes a message item when the completed event carries no assistant text, so non-streaming clients (including the title/summary utility path) receive the text instead of an empty response.

Documentation¶

• Document the post-v1.9.1 Codex and transport changes across the English and Korean manuals: the Codex Chat-to-Responses request handling and lenient expires_at parsing in backends.md, the cloud-OpenAI field strip in backend-passthrough.md, and item_reference resolution on the Responses API in api.md, and convert the remaining Korean pages to the polite -습니다 register so every page reads in one voice.

v1.9.1 - 2026-06-11¶

Added¶

• GET /version endpoint that returns { "version": "<CARGO_PKG_VERSION>" }, registered unconditionally on the base router (not behind the appproxy or any other Cargo feature) so it is present in the standard release binary, and a version field on the existing GET /health response (#750, closes #749). Both endpoints stay outside the API-auth boundary, matching /health, so a downstream consumer can probe the running router version for feature-gating instead of failing open against a 404.

v1.9.0 - 2026-06-10¶

Added¶

• AppProxy worker mode behind an opt-in appproxy Cargo feature, letting Continuum Router run as a Backend.AI AppProxy inference worker driven by an AppProxy coordinator (epic #709). The feature is deliberately left out of full, so default builds are unaffected.
• Foundation: the typed AppProxyWorkerConfig section (coordinator URL, shared api_secret/jwt_secret, redis_url, wildcard frontend parameters, heartbeat/reconcile durations, and an events toggle, with both bearer secrets redacted in Debug and ${ENV_VAR} references resolved through the same path as backends[].api_key), the SerializableCircuit/RouteInfo wire types and ProxyProtocol/AppMode/FrontendMode enums (snake_case and kebab-case tolerant, unknown fields ignored), and the module scaffold (#716).
• Coordinator REST client CoordinatorClient with register, heartbeat, deregister, list_circuits, and get_circuit, each carrying X-BackendAI-Token and a fresh per-call X-BackendAI-RequestID, and an error type that separates retryable (connection, timeout) from fatal (HTTP 4xx) failures (#717).
• Circuit-to-backend translation and reconcile: circuit_to_backends builds one BackendConfig per replica (named appproxy-<circuit_id>-r<route_key>, traffic-ratio mapped to a 1..=1000 weight that never drops a route to 0, vLLM detection from runtime_variant), and apply_circuits injects the translated backends through the existing hot-reload config_sender, namespaced by the appproxy- prefix so statically configured and admin-API backends are preserved (#718).
• Worker lifecycle service and a /status endpoint: run_worker registers with backoff, performs an initial circuit pull, then runs a heartbeat loop (kept under the coordinator's 30s LOST timeout) and a pull-reconcile loop (the always-on backstop for missed events), auto-discovering each circuit's model from a replica's GET /v1/models and deregistering on shutdown. The shared in-memory AppProxyRegistry is indexed by subdomain and circuit id (#719).
• Host/subdomain ingress resolver that turns a manager-issued endpoint subdomain into a concrete circuit and pins the request model so the existing selection path serves that circuit's replicas, with HS256 circuit-bearer verification that checks the decoded id against the circuit id and rejects an alg:none downgrade, an optional aggregation_hosts field for cross-circuit model aggregation, and a pure fall-through when no wildcard_domain is configured (#720).
• Redis Pub/Sub circuit-event overlay that gives legacy-mode coordinators sub-second circuit updates: a subscriber loop with exponential-backoff reconnect, a base64 + msgpack envelope codec, handlers for circuit_created / circuit_removed / circuit_route_updated, and the ack envelope that prevents the coordinator's E10001 "Proxy worker not responding" error (#721).
• Claude Fable 5 (claude-fable-5) and Mythos 5 (claude-mythos-5) model support (#747). Both are 1M-context, 128K-max-output models priced at \(10/\)50 per MTok; Mythos 5 is the same underlying model with safety classifiers lifted, shipped only through the limited Project Glasswing release. A new is_mythos_class helper routes both ids through the Anthropic capability gates: adaptive thinking is required (legacy budget_tokens is rejected with HTTP 400 and normalized to adaptive), temperature/top_p/top_k are dropped, the max effort level is supported (xhigh maps to max), and mid-conversation system messages are preserved. Both reject an explicit thinking.type == "disabled", so explicit_thinking_for_model now returns an Option and the router omits the thinking parameter entirely instead of forwarding a value that would 400. opus_supports_max_effort is renamed to supports_max_effort because the max effort level is no longer Opus-only. The same handling applies to the OpenAI Responses API conversion path.
• Gemma 4 QAT model metadata for the five quantization-aware-training checkpoints (E2B, E4B, 12B Unified, 26B-A4B MoE, 31B dense) with load-bearing -it-qat aliases and resolution/drift-guard tests (#723, closes #722).
• Gemma 4 12B Unified model metadata (#705).

Changed¶

• Log request-body extractor rejections at warn, so a malformed or oversized body that Axum rejects before the handler runs is visible in the logs instead of failing silently (#707).

Fixed¶

• Stop the retry loop from hammering the same upstream on HTTP 429 by distinguishing transient rate limits from non-transient quota/credit exhaustion and by honoring the upstream Retry-After hint (#742, closes #740). Previously every 429 was retried up to max_attempts (default 3) with a fixed exponential backoff that ignored the provider's Retry-After, so for a model served by a single upstream the router re-hit the same exhausted endpoint, amplifying load and adding latency before an inevitable failure. The RouterError::RateLimited variant now carries a retryable flag: non-transient 429s (OpenAI insufficient_quota / billing_hard_limit_reached, and clear credit-depletion language such as "prepayment credits are depleted") are classified as non-retryable so the router fails fast after a single call and passes the provider status and body through, while transient signals (bare RESOURCE_EXHAUSTED/RPM throttling, rate_limit_exceeded, rate_limit_error) stay retryable. The classifier is deliberately narrow: the over-broad "billing" and "exceeded your current quota" markers that Google reuses verbatim for transient throttling were removed so a recoverable Google 429 is no longer flipped to fail-fast. A retried 429 uses the upstream Retry-After for its backoff (capped to max_delay) and fails fast without sleeping when the requested interval would exceed the remaining total-timeout budget. The hint is preserved end to end (parsed from Google's RetryInfo.retryDelay and the integer-seconds Retry-After header, then reflected in the client-facing Retry-After header). The budget probe now uses saturating_add and the parsed hint is clamped to 24 hours (MAX_RETRY_AFTER_SECS), closing a remotely triggerable panic where a hostile upstream's near-u64::MAX Retry-After overflowed the Duration addition and aborted the request task.
• Persist the accumulated /v1/responses response on the streaming conversion paths (Anthropic, Chat-Completions/Gemini fallback) so a follow-up request that references a streamed output item via {"type":"item_reference","id":"item_..."} (the default behavior of the OpenAI and Vercel AI SDKs) resolves instead of returning HTTP 400, even when step 1 used store:true (#746, closes #745). The completed response is stored before the first response.completed event reaches the client; error-terminated streams that never emit response.completed are not stored, matching the non-streaming error paths, and passthrough streaming is unchanged because the upstream owns storage there.
• Resolve item_reference input items before strategy dispatch so /v1/responses no longer returns HTTP 400 for Anthropic/Claude backends on a multi-step tool round-trip that submits an item_reference instead of an inline item (#743, refs #741). References are rewritten to inline FunctionCall/Message/FunctionCallOutput items (de-duplicated by call_id, first-wins), build_context_for_user reconstructs stored function-call output items as proper tool_use/tool_result pairs, the OpenAI/Azure passthrough path still forwards references unchanged, an unresolvable reference returns a descriptive 400 naming the id, and a 256-item cap (MAX_ITEM_REFERENCES) bounds the per-request session-store scan.
• Propagate server.workers to the Tokio runtime (#736, refs #734). The value was documented and shipped in config.yaml.example but had no effect, because main used an argument-less #[tokio::main] and the runtime always ran with num_cpus::get() worker threads. main is now synchronous: it peeks server.workers from the config file, builds a correctly sized multi-thread runtime through the existing RuntimeConfig::build_runtime path, and runs the async body on it, falling back to the CPU count when the value is unset or 0.
• Emit conformant function_call output items and argument events on /v1/responses streaming for non-passthrough providers (Anthropic and chat-completions-backed routes), preserving text output payloads and tracking interleaved parallel tool-call arguments by upstream index (#725).
• Reuse the shared ApiKeyStore in the Files API routes instead of constructing a separate store, so a runtime-managed API key is recognized consistently across the Files API and the rest of the router (#706).
• AppProxy: preserve sibling circuits on single-circuit events (#737, closes #731). A worker serving two or more circuits previously wiped every unaffected sibling's appproxy-* backends on any single-circuit event (leaving them 404/502 until the next pull-reconcile, up to 15s), because RegistryEntry carried no route info and the rebuilt set held only the delta circuit. The full circuit is now cached on each RegistryEntry and unchanged siblings are rebuilt from it, so only the delta circuit's backends change.
• AppProxy: reach the fallback chain from wildcard subdomain ingress (#738, closes #735). A registered circuit whose replicas are all down is no longer a dead end; after per-circuit authorization it is pinned to its canonical model and handed to the normal pipeline, where FallbackService takes over (the "deployment went down, traffic goes to a cross-provider model" behavior operators expect). The fall-through is scoped to registered-but-down circuits, a truly unknown subdomain stays a 404, and the open-to-public / bearer-token / IP-allow-list gates still run first.
• AppProxy: preserve event-known models during periodic reconcile (#739). When the Redis event overlay learned a circuit's model before any successful pull probe, a reconcile could evict the registered-but-down registry entry and break scoped fallback; reconcile now reuses the shared registry's known model before probing replicas.
• AppProxy: align WorkerRegisterResponse deserialization with the Backend.AI coordinator's actual response shape, which carries slots as an array plus available_slots as the count (#726).

Documentation¶

• Rewrite the Zensical user documentation as a current-state manual: drop development-log narration, the roadmap, and "coming soon" entries; correct configuration-reference drift against the actual config structs (nonexistent sections and keys, retry field names, the admin auth shape, the environment-variable tables, and the config discovery order); document previously missing shipped behavior (seven CLI flags, the auth login subcommand, Windows AF_UNIX support and SSE over Unix sockets, and the Windows/musl and .deb release artifacts); and bring the Korean docs to parity with English.
• Add the AppProxy worker mode design document (#708).
• Condense the README "Recent Updates" list to one concise line per release.

Dependencies¶

• Remove the validator derive dependency and its transitive proc-macro-error2, clearing the RUSTSEC-2026-0173 advisory that previously needed a temporary cargo-deny ignore while validator had no safe upgrade path (#733, #732).
• Update Rust package versions (#732).

v1.8.2 - 2026-06-02¶

Fixed¶

• Stop forwarding the client Accept-Encoding header on the /v1/responses path (#702). When a client sent Accept-Encoding: gzip, deflate, br, the responses-path header filter omitted accept-encoding from its block list and forwarded it to the upstream backend, which then negotiated gzip and returned compressed bytes. Because reqwest disables automatic decompression once any Accept-Encoding header is set manually (the explicit .header("Accept-Encoding", "identity") call only appended a second value rather than replacing the forwarded one), the SSE transform received raw gzip bytes, parsed them as text, and dropped the leading response.created, output_item.added, content_part.added, and output_text.delta events, leaving only tail fragments with empty item_id and text. "accept-encoding" is now in FILTERED_HEADERS for both the primary convert path (src/http/handlers/responses.rs) and the responses-native passthrough path (src/proxy/responses_only.rs), restoring parity with the chat-completions proxy (src/proxy/backend.rs) so the upstream only ever receives Accept-Encoding: identity.
• Stop double-wrapping Responses SSE lines so the non-GPT /v1/responses streaming conversion emits single-layer OpenAI-compatible SSE records on the converted Anthropic and Chat-Completions paths instead of nested ones (#701).

Dependencies¶

• Bump uuid 1.23.1 → 1.23.2, redis 1.2.1 → 1.2.2, socket2 0.6.3 → 0.6.4, and serial_test 3.4.0 → 3.5.0 (#699).

Tests¶

• Hardening regression coverage for the production StreamService conversion processors, asserting single-layer SSE output on the converted Anthropic and Chat-Completions paths (#701).
• Integration regression for the /v1/responses streaming path with an Anthropic backend and a gzip-requesting client, asserting the upstream request receives only Accept-Encoding: identity and the transformed Responses SSE stream retains the full event sequence with populated text and item ids (#702).

v1.8.1 - 2026-05-29¶

Added¶

• Claude Opus 4.8 recognition with a claude_family_version parser that replaces the hardcoded opus-4-7/opus-4-6 substring gates (#693, part of #687). The four Anthropic capability predicates now compare a parsed (major, minor) version: uses_adaptive_thinking_api ≥ (4,6), model_requires_adaptive_thinking / model_forbids_sampling_params ≥ (4,7), and opus_supports_max_effort = Opus and ≥ (4,6). The parser treats the first integer token as the major and the next version-like token (1 to 2 digits, value < 100) as the minor, so an 8-digit date suffix like 20250514 yields minor 0 and is never mistaken for a version, and new minor releases are recognized without per-version edits. Adds the claude-opus-4-8 metadata entry (1M context, 128K output, \(5/\)25 pricing, adaptive thinking, Jan 2026 cutoff) and registers claude-opus-4-8 / claude-opus-4-8-latest in the built-in supported models and config samples. Behavior for 4.5/4.6/4.7 and Sonnet variants is preserved.
• Anthropic fast mode behind a per-backend anthropic_fast_mode opt-in (default off) (#694, part of #687). is_fast_mode_eligible returns true only for Opus 4.6/4.7/4.8 and later Opus minors; merge_beta_header comma-joins and de-duplicates beta tokens while preserving any client-supplied anthropic-beta. On the native /anthropic/v1/messages path, resolve_fast_mode_beta injects the merged fast-mode-2026-02-01 beta header only when the request is speed: "fast", the model is eligible, the backend is native Anthropic (never Bedrock), and the opt-in is enabled; when fast mode does not apply, speed is stripped from the outgoing body so it cannot trigger a spurious upstream 400. The OpenAI-compatible path forwards speed: "fast" and injects the beta header only for eligible, opted-in, native Anthropic targets. The Anthropic -> OpenAI and Anthropic -> Google fallback parameter mappings remove speed so a fast-mode request that falls back to a non-Anthropic backend does not leak the native-only field. usage.speed is preserved on the response.
• Mid-conversation system messages for Claude Opus 4.8+ (#695, part of #687). A role:"system" entry inside the messages array (which earlier Claude families reject with HTTP 400) is now accepted, gated on a new supports_mid_conversation_system(model_id) predicate that reuses claude_family_version and matches family version ≥ (4,8). The native handler round-trips the entry unchanged to a native Anthropic backend; the cross-provider transforms map the System role onto the OpenAI system role and preserve it as user-role text for Gemini and Responses. The OpenAI-compatible transform emits mid-conversation system/developer messages (after the first user turn) as in-array role:"system" entries for supporting models, while leading system messages still fill the top-level system field. Non-supporting models (Opus 4.7 and below, all Sonnet/Haiku, Bedrock-prefixed ids, non-Claude ids) keep the historical flattening into the single top-level system.
• Refusal stop_details and the refusal stop reason propagated through the full Anthropic response pipeline (#696, part of #687). map_anthropic_finish_reason maps "refusal" to "content_filter"; the non-streaming transform and the streaming message_delta handler attach the stop_details object to the choice when stop_reason is "refusal", and omit the key (rather than forwarding a null) when upstream sends an explicit null.

Fixed¶

• Accept an input_image content part that references a Files API upload via file_id instead of an inline image_url on POST /v1/responses (#686, refs #681). Because the parent enums are #[serde(untagged)], a {"type":"input_image","file_id":"file-..."} part previously failed deserialization with a generic untagged-enum error and Axum returned HTTP 422. image_url is now optional with an added file_id, mirroring input_file; a shared resolve_local_file_to_data_url helper resolves a local file_id to an inline base64 image_url data URL through the same metadata, ownership, size, load, and base64 sequence, honoring ownership and the 10MB size limit. The OpenAI/Anthropic/Gemini converters handle the optional image_url, emitting the image when resolved and warning + skipping an unresolved file_id. validate_request walks message content and rejects an input_image with neither image_url nor file_id with a clear 400 before file resolution.
• Harden the Claude Opus 4.8 routing gates so fast-mode speed is only forwarded when the transport has confirmed native-Anthropic opt-in and beta-header injection, non-Opus Claude families keep mid-conversation system messages flattened, and OpenAI-compatible Anthropic responses preserve usage.speed (#698, refs #687).

Documentation¶

• Document Claude Opus 4.8 support in English and Korean (#697, closes #692, part of #687): add claude-opus-4-8-* to the adaptive-thinking model list and the sampling-params-deprecated warning in reasoning-effort.md; add the Claude Opus 4.8 model detail, an Anthropic Fast Mode section, a Mid-Conversation System Messages section, and the refusal stop_reason -> content_filter mapping to backends.md; and add a changelog entry covering model recognition, fast mode, mid-conversation system messages, and refusal stop_details.
• Document input_image file_id support in api.md (English and Korean), noting that exactly one of image_url or file_id is required and that file_id is resolved to an inline base64 data URL before reaching the backend under the same ownership and 10MB size limit as the input_file path (#686, refs #681).

Tests¶

• claude_family_version and supports_mid_conversation_system boundary tests (4.7 false, 4.8 true, Sonnet/Haiku/older false, Bedrock-prefixed and cross-region/ARN ids false), fast-mode eligibility, beta-header merge/dedup, speed/usage.speed (de)serialization, resolve_fast_mode_beta gating with client-beta merge, and speed non-leak versus preservation across fallback providers (#693, #694, #695).
• Refusal coverage: non-streaming and streaming refusal with and without stop_details, the explicit-null omission path, and regression tests for end_turn/max_tokens/stop_sequence/tool_use (#696).
• input_image file_id-only and image_url-only deserialization, the full failing request payload as a regression, FileResolver resolution honoring ownership and the size limit, converter output across all three backends, and the neither-field validation case (#686).

v1.8.0 - 2026-05-28¶

Added¶

• Per-API-key backend access control via an optional allowed_backends allow-list on client API keys (#677, closes #674). When the list is non-empty, requests authenticated with that key may only route to the named backends; an empty or absent list keeps the existing unrestricted behavior. The field is integrated end-to-end: config file and hot-reload, the runtime ApiKey and AuthContext, the backend-selection chokepoint and the Responses / Anthropic selection paths, cross-provider fallback, the Admin REST API (create/update/get/list), runtime-key persistence, and the models-listing endpoints.
• select_backend_with_retry and the Responses (StreamService), Anthropic-native, count_tokens, and image handlers filter candidates by the key's allow-list. When the model exists but the allow-list rejects every candidate, the request is rejected with a new RouterError::Forbidden variant mapped to 403 with error_type = "permission_error" (non-retryable), distinct from the 401 AuthError.
• /v1/models, /v1/models/extended, and /anthropic/v1/models are filtered to models served by at least one allowed backend when a restricted key is authenticated; GET /v1/models/{model} returns 404 for a model the key cannot reach. Unauthenticated or unrestricted callers see the full list.
• A new api_optional_auth_middleware is layered in permissive mode. It validates a presented bearer token on a best-effort basis and attaches AuthContext without ever rejecting, so per-key restrictions apply to authenticated callers while anonymous and invalid-token callers pass through unrestricted. The existing blocking-mode api_auth_middleware is unchanged; the two are never layered together.
• Config validation warns (does not hard-fail) when a key's allowed_backends references an unknown backend name, so a backend rename does not brick the router before operators update the keys.
• fallback.mid_stream_enabled config field (default true) so operators can keep cheap pre-stream backend re-selection on the initial connection while turning off the per-stream mid-stream buffering (#680, closes #676). Previously fallback.enabled was all-or-nothing: on meant both pre-stream and mid-stream fallback (with a per-stream StreamAccumulator buffering roughly 100 to 200 KB), off meant no fallback at all. Memory-constrained or high-concurrency hosts now have a middle ground.
• The streaming dispatch becomes a three-way decision factored into a pure decide_streaming_fallback_dispatch helper. With fallback.enabled true and a chain configured, mid_stream_enabled = true keeps the buffering path (handle_streaming_with_mid_stream_fallback), mid_stream_enabled = false routes to the revived handle_streaming_with_pre_stream_fallback (no StreamAccumulator or MidStreamFallbackContext allocation; mid-stream failures surface as a normal stream error), and otherwise the standard no-fallback path runs.
• The previously dead handle_streaming_with_pre_stream_fallback and advance_to_next_fallback are now live; the #[allow(dead_code)] markers are removed and the per-key allow-list is threaded through fallback re-selection so the newly-live path does not bypass per-API-key access control.

Changed¶

• Removed the streaming-local copies of transform_payload_for_openai and its requires_max_completion_tokens helper from src/http/streaming/handler.rs; both call sites now resolve to the canonical implementations in crate::proxy::utils (#679, closes #660). The two copies were byte-for-byte identical, creating a drift risk where a change to one would not be mirrored to the other. The streaming-local duplicate unit tests are removed since the canonical tests in src/proxy/utils.rs already cover the same contract.

Fixed¶

• Enforce the per-API-key backend allow-list in the default mid-stream fallback handler when it re-selects a backend after a mid-stream failure (#683). With fallback.mid_stream_enabled = true (the default), a restricted key whose fallback chain mapped to a disallowed backend was transparently switched to it on a mid-stream failure, an access-control bypass that the new mid_stream_enabled work in PR #680 had only closed on the pre-stream path. handle_streaming_with_mid_stream_fallback now takes an owned allowed_backends: Option<Vec<String>> (moved into its spawned streaming task), and a new resolve_allowed_backend_name_for_model helper wraps resolve_backend_name_for_model and applies the same allowed_backends.filter(|l| !l.is_empty()) + exact-name membership semantics used everywhere else in the module (try_get_healthy_backend_for_model, get_backend_for_model_streaming, get_healthy_backend_for_streaming). All three fallback re-selection sites resolve through it; a disallowed or unresolvable candidate returns None, so the existing warn + continue arm skips it and the chain index advances. When every remaining candidate is filtered out the loop terminates via the existing chain-exhausted paths and surfaces an error to the client. An empty or None allow-list is byte-for-byte the prior unrestricted behavior.
• Resolve an allowed fallback backend before rebuilding the fallback payload in mid-stream fallback, so a disallowed chain entry can no longer mutate current_payload before being skipped (#684).
• Anthropic-native x-api-key callers now supply the same allowed_backends policy as Authorization: Bearer-authenticated callers when no AuthContext is present, covering Messages, count_tokens, and models listing (#684). The previously documented limitation that the per-key allow-list was unenforced on the native Anthropic surface is now closed.
• Close a HIGH-severity authorization-bypass on POST /v1/responses/compact found in the post-merge security audit of #677. The endpoint was the one client-facing model-routing handler that never read AuthContext from request extensions, so a key scoped to backend set A could reach a disallowed passthrough backend B (OpenAI / Azure) through compaction whenever B served the requested model. compact_response now mirrors create_response exactly: it derives the per-key allow-list via allow_list_from_auth, filters the model's candidate backends against it before any health-check or passthrough, and returns a deterministic 403 permission_error when the model exists but the key cannot route to any backend serving it. The filter precedes backend forwarding, so the rejection happens without contacting an upstream.

Documentation¶

• Clarify in config.yaml.example, src/services/streaming/mid_stream_config.rs rustdoc, and docs/en/configuration/advanced.md that mid_stream_fallback.enabled does not disable mid-stream buffering or reduce memory: regardless of its value the StreamAccumulator is still constructed and still buffers up to roughly 100 KB per stream, and mid-stream fallback still activates on backend failure (#678, closes #675). The flag only selects continuation versus restart mode for the fallback request. The real kill-switch for the buffering and memory is fallback.enabled: false or omitting the model from fallback.fallback_chains. The Korean docs (docs/ko/configuration/advanced.md) had no corresponding Mid-Stream Fallback section so no Korean change is made for the clarification, but #680 separately added a translated "미드스트림 버퍼링 비활성화" subsection covering the new toggle.
• Document the per-API-key allowed_backends allow-list in the security docs (English and Korean) including the now-closed x-api-key Anthropic Messages limitation (#677, #684).

Tests¶

• Deterministic unit tests for resolve_allowed_backend_name_for_model: allow-list excludes the resolved backend returns None, includes it returns Some, empty list returns Some, None matches the unfiltered resolver, unresolvable model returns None (#683).
• Integration tests in tests/per_key_backend_access_test.rs driving /v1/responses/compact behind blocking auth: a restricted key requesting a model served only by a disallowed backend gets 403 permission_error (the security case), and a restricted key requesting an allowed backend's model passes the filter (#677).
• Unit test for the new decide_streaming_fallback_dispatch helper covering the three-way decision matrix between fallback.enabled, mid_stream_enabled, and chain presence (#680).

v1.7.1 - 2026-05-28¶

Fixed¶

• Anthropic web_search_20250305 server-tool emulation no longer masks provider failures as empty result sets. A missing API key, HTTP 401/403/429, timeout, or parse error now surfaces to the client as a web_search_tool_result_error content block with a mapped error_code ("too_many_requests" for HTTP 429, "unavailable" for all other failures). A genuinely empty-but-successful search continues to emit an empty content: [] array, so the two outcomes remain distinguishable. Both non-streaming and streaming SSE code paths are covered, with an explicit SSE event-ordering assertion guarding the streaming error path. Serper response-shape drift (a body that parses cleanly but omits the organic key) is now detected and logged with a provider-tagged warn! that records only the observed top-level keys, never the user query text. (#671, #672, #673)

Dependencies¶

• Bump lru 0.16 → 0.18, reqwest 0.13.3 → 0.13.4, and rusqlite 0.39 → 0.40 (libsqlite3-sys 0.37 → 0.38, unpinned now that CI runs Rust 1.95), plus a transitive lockfile refresh (aws-lc-rs 1.16.3 → 1.17.0, h2 0.4.13 → 0.4.14, http 1.4.0 → 1.4.1, and hyper/axum/reqwest knock-on revisions); cargo audit reports zero vulnerabilities across the dependency graph (#669, #670).

v1.7.0 - 2026-05-26¶

Added¶

• AWS Bedrock Claude backend Phase 1 (bedrock-mantle) over a Bearer token (#616, closes #613)
• New type: bedrock backend with serde aliases (aws-bedrock, bedrock-anthropic, AwsBedrock, ...). is_commercial() returns true and owned_by() returns Some("anthropic") so OpenAI-shaped clients see the expected model lineage.
• endpoint_type: mantle (default) speaks the native Anthropic Messages API at the region-templated https://bedrock-mantle.{region}.api.aws, routes to /anthropic/v1/messages, uses Authorization: Bearer, and omits anthropic-version (Bedrock returns HTTP 400 if present). An explicit url: field overrides the template for proxies and tests; empty or uppercase regions are rejected at load time.
• model_ids.rs recognizes plain (anthropic.<family>), geographic (us./eu./jp./au.), global (global.anthropic.<family>), and full-ARN identifiers, forwarded unchanged with no automatic alias mapping because the geo prefix carries real billing and residency consequences.

• The existing OpenAI/Anthropic body transforms and the Anthropic SSE stream transformer are reused unchanged, so per-model quirks (Opus 4.7 sampling-param ban, adaptive thinking) apply identically to Bedrock without duplication. endpoint_type: runtime is reserved here and implemented in Phase 2 below.

• Request-path rate limiting is now enforced, plus a Redis storage backend (#635, #632, closes #626)

• state.rs previously dropped the MiddlewareLayer returned by initialize_rate_limiting, so the rate_limiting.* config was a silent no-op. The layer now flows through ServiceHandles → ContinuumRouterBuilder::build_router → Router::layer and attaches to the assembled Axum app, so configured budgets actually return 429.
• All five rate_limiting dimensions are now optional: per_client, per_backend, and global join the already-optional per_api_key/per_model. Operators may omit any dimension and load with it disabled; the Default impl retains all three so existing deployments are unaffected (#632).

• New redis-cache-gated rate_limit_v2::redis_backend runs a token-bucket and a sliding-window Lua script, each as a single atomic EVAL, reusing the shared create_redis_pool helper with keys like cr:rl:per_client:10.0.0.1. On any Redis failure (pool unavailable, timeout, Lua error) the backend reports BackendUnavailable and the caller falls through to the in-process token bucket, degrading to per-replica enforcement rather than dropping requests.

• Cross-provider fallback is now wired into request dispatch and hot-reload (#631, #637, #665)

• The completed src/core/fallback/ module (~4,900 lines, 36 unit tests) was never called from the request path, so configured fallback.fallback_chains were a silent no-op. FallbackService now runs for chat_completions (non-web-search), completions, embeddings, rerank, sparse_embeddings, and image generation, with an execute_with_optional_fallback wrapper (no overhead when fallback is unconfigured), X-Fallback-* response headers, and a From<RouterError> to TriggerReason mapping that satisfies the executor bound (#631).

• fallback.fallback_chains and fallback.fallback_policy changes now apply at runtime through the hot-reload subscriber via FallbackService::update_config; toggling fallback.enabled remains restart-only and is documented as such in config.yaml.example (#637, #665).

• POST /v1/models/refresh force-refresh endpoint for interactive desktop use (#593, #664)

• Clears the ModelCache (all_models key) and synchronously re-aggregates from all configured backends before responding, returning the same {"object":"list","data":[...]} shape as GET /v1/models. Desktop clients (e.g. backend.ai-go "Refresh models" button) can use the response immediately without a second round-trip.
• Rate-limited per verified API key, with anonymous or invalid-token callers sharing one global anonymous bucket: 3 requests per 5-second burst window, 12 per minute. Callers that exceed the limit receive 429 Too Many Requests. The limit is intentionally tighter than the regular list endpoint because each call triggers an upstream fetch from every configured backend.
• Gated by a new model_aggregation.allow_force_refresh: bool config field (default true). Setting it to false makes the endpoint return 403 Forbidden, suitable for hardened deployments where clients must rely on TTL-based expiry.
• Each refresh logs the verified API key ID when available, or anonymous otherwise, at INFO level for audit correlation.
• config.yaml.example extended with desktop-embedded guidance: cache_ttl: 10, soft_ttl_ratio: 0.5, and allow_force_refresh: true for backends.ai-go style embedded proxy.

• New force_refresh(state) helper on ModelAggregationService encapsulates the clear-then-aggregate flow. allow_force_refresh() accessor exposes the config flag to handlers.

• AWS Bedrock Claude backend Phase 2: bedrock-runtime with SigV4 + AWS binary event-stream (#614)

• New endpoint_type: runtime value on type: bedrock backends targets https://bedrock-runtime.{region}.amazonaws.com/model/{modelId}/invoke[-with-response-stream]. The router signs each request with AWS Signature V4 (service: "bedrock"), wraps the OpenAI → Anthropic body with "anthropic_version": "bedrock-2023-05-31", strips the top-level "model" field (Bedrock takes the model ID from the URL path), and percent-encodes the model identifier into the path so versioned foundation IDs (anthropic.claude-3-5-sonnet-20240620-v1:0) and full ARNs round-trip cleanly.
• Streaming responses arrive in the AWS application/vnd.amazon.eventstream binary frame format. A new bedrock::event_stream::EventStreamDecoder reassembles frames that span multiple TCP reads, base64-decodes each chunk payload, and emits synthetic event: <type>\ndata: <json>\n\n SSE bytes for the existing AnthropicStreamTransformer to translate into OpenAI-shape SSE. Exception frames (ThrottlingException, ValidationException, ...) surface as synthetic event: error SSE chunks instead of being silently dropped.
• AWS credentials resolve in this order: inline auth.aws.access_key_id + auth.aws.secret_access_key (+ optional session_token), then a named profile via auth.aws.profile, then the standard AWS chain (env vars, shared config, IMDS, IRSA / EKS pod identity, ECS task role). The resolver is fronted by aws_credential_types::provider::SharedCredentialsProvider so temporary credentials refresh transparently between requests.
• New BackendAuthType::Sigv4 variant on BackendAuthConfig, plus an AwsAuthConfig sub-block under auth.aws. Both are wired through Debug redaction so static credentials never leak into logs. BackendAuthType accepts sigv4, aws_sigv4, and aws-sigv4 as YAML spellings.
• Health check for runtime probes POST /model/{probe_model}/invoke with a single-token body; HTTP 2xx, 400, 401, 403, and 429 all count as healthy because they prove the AWS surface is reachable and the operator can address auth/billing issues separately.
• All AWS SDK crates (aws-sigv4, aws-smithy-eventstream, aws-credential-types, aws-config) sit behind a new optional bedrock-sigv4 Cargo feature. Default builds do not pull them in; configuring endpoint_type: runtime without the feature returns a clear error pointing at the rebuild flag instead of failing at the AWS edge. The Phase 1 mantle path is unaffected and works with or without the feature.
• Proxy header policy in src/proxy/backend.rs and src/http/handlers/anthropic/handler.rs splits on endpoint_type: Bedrock-mantle keeps Authorization: Bearer, Bedrock-runtime suppresses the static-Bearer injection because the Backend trait implementation signs each request with SigV4.

• English and Korean docs in docs/{en,ko}/configuration/backends.md extended in place with the new endpoint_type: runtime configuration, build requirement, credential chain, IAM policy snippet, geo/global profile behaviour, and a streaming-pipeline overview.

• EXAONE 4.0 (vLLM) registration with a request-gated hybrid-model thinking transform (#640, refs #639)

• EXAONE 4.0 (e.g. EXAONE-4.0-32B-FP8-RNGD) is a hybrid reasoning model: in reasoning mode it streams chain-of-thought inline in content ended by a lone </think>; in non-reasoning mode it emits a plain answer with no </think>.
• The assume_reasoning_first (unterminated_start) transform is now gated on the request actually enabling thinking (chat_template_kwargs.enable_thinking or top-level enable_thinking; conservative default false) at both the HTTP and Unix-socket streaming decision points, so non-reasoning mode no longer emits the whole answer as reasoning_content with empty content. Standard-pattern models keyed off a real <think> marker are unaffected.

• Registers exaone-4.0-32b with the unterminated_start config; the served -RNGD name resolves via the hardware-suffix peel below.

• NPU/accelerator hardware-variant suffix normalization in model-id matching (#662)

• Adds a HARDWARE/ACCELERATOR category (rngd, warboy, atom, atommax, rebel) to is_recognized_format_token() so FuriosaAI (RNGD/WARBOY) and Rebellions (ATOM/ATOMMAX/REBEL) serving-target suffixes normalize to the canonical base metadata entry through the existing layered peel chain, without a per-model alias. layered_format_strip() lowercases before peeling, so runtime-emitted upper-case names resolve correctly.
• Exact-id and exact-alias phases run before the peel, so any model legitimately registered as *-atom or *-rebel wins via exact match first; a grep of shipped model-metadata.yaml confirms zero current collisions. EXAONE-4.0-32B-FP8-RNGD now normalizes to exaone-4.0-32b by peel (-rngd then -fp8).

Changed¶

• Narrow the fallback handler's LM Studio compatibility shim so only / and /v1/models return 200; all other unmatched routes now return 404. The JSON error body shape is preserved unchanged, so any consumer already reading the body continues to work (#628).

Fixed¶

• Strip seven non-OpenAI top-level fields (chat_template_kwargs, thinking_budget_tokens, enable_thinking, preserve_thinking, top_k, min_p, repeat_penalty) before forwarding to Gemini's /v1beta/openai/chat/completions endpoint, which returns HTTP 400 INVALID_ARGUMENT for unknown keys; the extra_body escape hatch is untouched and reasoning_effort stays (Google maps it to thinking_level). Also extend the 3.5-flash thinking-disable and is-thinking matchers, since 3-flash did not substring-match 3.5-flash (#642).
• Wire request-stats recording into every Anthropic handler code path (native HTTP/Unix, Bedrock mantle/runtime, OpenAI-compatible, Responses API) for both streaming and non-streaming, adding an AnthropicStreamUsageTracker that accumulates input/output tokens from raw passthrough SSE (#627, #634).
• Use the configured timeouts.request.streaming.chunk_interval instead of a hardcoded 60s for mid-stream inactivity, and emit bounded keep-alives so a silent backend now advances to the next fallback model via StreamOutcome::Failed rather than emitting keep-alive comments forever (#633).
• Accept partial model_overrides.<model>.streaming/standard blocks via new StreamingTimeoutOverride/StandardTimeoutOverride structs whose Option<String> fields merge over the base config, fixing YAML parse failures on the --generate-config output path (#630).
• Gate admin/metrics/metrics-persistence/webui imports and functions behind their Cargo features, and add a #[cfg(not(feature = "metrics"))] no-op metrics stub mirroring the public surface used by always-compiled callers, so feature-reduced builds compile cleanly (#629, #666, closes #636).
• Harden force-refresh rate limiting so anonymous and invalid-token callers share one global bucket, preventing spoofed Authorization/X-Forwarded-For/X-Real-IP headers from bypassing the budget.
• Metrics history query limiting, UTF-8-safe metric label truncation, and Bedrock runtime routing through the typed SigV4 implementation (follow-up to #608, #609, #613, #614).

Documentation¶

• Document the Bedrock backend (region selection, geographic vs global inference profiles, model ID format, credential chain, IAM policy snippet, and streaming pipeline) in docs/{en,ko}/configuration/backends.md; add a Force-Refresh Models section to docs/en/api.md; and extend config.yaml.example with desktop-embedded model-aggregation guidance and fallback hot-reload annotations.

Tests¶

• Negative and positive case coverage for transform_payload_for_openai (#661).
• Rate-limit middleware hot-reload tests with documented bucket-reset behaviour, plus router_wiring_tests that build a real ContinuumRouter and assert 429 fires when the burst is exhausted (#635, #638, #667).
• Verify web_search injection interacts correctly with the passthrough contract (#663).
• MLxcel streaming passthrough integration test (#659).
• Bedrock unit and integration coverage: serde aliases, URL templating, header policy, model-ID parsing (geo/global/ARN), runtime SigV4, and event-stream frame decoding driven against a wiremock server (#616, #614).

Dependencies¶

• Bump tokio 1.52.1 → 1.52.3, tower-http 0.6.8 → 0.6.11, dashmap 6.1.0 → 6.2.1, serde_json 1.0.149 → 1.0.150, aws-config 1.8.16 → 1.8.17, aws-sigv4 1.4.3 → 1.4.4, and aws-smithy-types 1.4.7 → 1.4.8 (#619, #658).

v1.6.3 - 2026-05-12¶

Added¶

• Per-API-key LLM token usage metrics (#608, #610)
• New Prometheus llm_tokens_total{api_key_id, model, backend, kind} counter that records actual prompt and completion token consumption per API key, model, backend, and token kind. The hot-path counter's label set is intentionally minimal — extra dimensions live on the companion info-metric below.
• Companion api_key_info{api_key_id, ...} info-metric exposes a configurable allowlist of per-API-key annotation labels (e.g. email, team, environment) so dashboards can group/filter the token counter via standard PromQL * on(api_key_id) group_left(...) joins without bloating the hot-path counter's label set.
• derive_api_key_id returns either the configured id (when the auth layer matched the request) or a SHA-256 first-12-hex prefix k_<hex> of the raw bearer token. The raw key is never used as a label. A dedicated ApiKeyCardinalityTracker (default cap: 1000 unique key IDs) prevents label-cardinality explosion.
• ApiKeyConfig and the in-memory ApiKey gain an annotations: HashMap<String, String> field. MetricsConfig gains annotation_labels: Vec<String> — the allowlist that materializes as labels on api_key_info. Reserved canonical annotation keys are documented (email, uuid, owner, team, environment); operators may add custom keys.
• Streaming and non-streaming paths record at the existing usage parse site through a new StreamObservabilityContext field on StreamTransformConfig, threaded through handle_anthropic_streaming / handle_gemini_streaming / handle_successful_backend_response so OpenAI-compat / Anthropic / Gemini / thinking-pattern streaming response builders all emit the counter without duplicate parsing. The router already injects stream_options.include_usage=true for OpenAI-compat backends, so streaming metrics work uniformly regardless of client opt-in.
• api_key_info is initialized once at startup from metrics.annotation_labels; label names are frozen at registration (Prometheus does not allow renaming labels). Annotation values hot-reload through the existing config-watch path via ApiKeyStore::refresh_info_metric, called from load_from_config, add_key, and remove_key_by_id so admin operations stay in sync.
• All label values flow through CardinalityManager / sanitize_label_value. Annotation values use a slightly less strict sanitize_annotation_value that preserves @, +, and : so emails and namespaced identifiers round-trip cleanly.
• Persistent local metrics log backed by SQLite with configurable retention (#609, #611)
• New MetricsStore async trait + bundled rusqlite v1 implementation under src/metrics/persistence/ with WAL mode, prepared-statement cache, and PRAGMA user_version schema versioning (store.rs / sqlite.rs / snapshot.rs / snapshot_task.rs). Histograms and summaries are fanned into row-per-sample form.
• Counters and gauges are NEVER restored on startup — the persistent log is a separate read path so the live /metrics endpoint keeps Prometheus monotonic-counter semantics. Historical samples are read through a new GET /admin/metrics/history?metric=...&from=...&to=... surface (src/admin_metrics_history.rs) that returns 404 when persistence is disabled at runtime and 503 when the feature is not compiled in. No PromQL in v1.
• Hot-reload pipeline in src/server/serve.rs translates config changes into PersistenceCommand::{SetSnapshotInterval, SetRetentionDays, SetCompaction} messages, atomically rebuilding the ticker and prune cutoff without dropping in-flight snapshots.
• Compaction schedule honors a minute hour * * * cron subset to avoid pulling in a full cron crate for what is effectively a daily timer.
• Defaults to enabled: true; switch off via metrics.persistence.enabled: false. The redb and duckdb variants are reserved keywords in the YAML schema and return NotImplemented at startup until they get implementations.
• Disk usage: measured ~119 bytes/sample on a synthetic 100-series × 10-snapshot workload (see tests/metrics_persistence_test::disk_usage_smoke_check_under_synthetic_load). Documented formula in docs/en/persistent-metrics.md and config.yaml.example.

Fixed¶

• Coerce token-usage label values to &str in with_label_values so the release build no longer fails type inference. Mixing &String label variables with a &str literal ("prompt" / "completion") made the compiler pick &[&String] and reject the literal — regression introduced in #610.

Documentation¶

• Korean translations for the two metrics features (#612)
• docs/ko/metrics.md: new ### API 키별 LLM 토큰 사용량 section covering llm_tokens_total, api_key_id derivation, annotation_labels allowlist, api_key_info info-metric, PromQL examples, Grafana panel, and verification steps.
• docs/ko/persistent-metrics.md: new page translating docs/en/persistent-metrics.md (SQLite-backed snapshot semantics, configuration fields, disk-usage formula, /admin/metrics/history surface, schema layout, operational notes).
• docs/ko/admin-api.md: insert ## 지속 메트릭 로그 API section between Stats and Response Cache, plus a matching TOC entry.
• zensical.ko.toml: add 지속 메트릭 로그 nav entry under 운영 so the page is reachable from the Korean sidebar.
• New docs/en/metrics.md ### Per-API-Key LLM Token Usage section covering metric definition, api_key_id derivation rules, annotation config schema, cardinality and hot-reload semantics, example PromQL (tokens-per-email, top-10 keys, per-team rollup), a Grafana panel example, and verification steps. config.yaml.example gains a documented metrics.annotation_labels block and an annotations: example under each API-key entry.

Tests¶

• Per-API-key token-usage unit coverage: derive_api_key_id priority (configured id wins; otherwise hash; otherwise anonymous), determinism, hash format ^k_[0-9a-f]{12}$, annotation-label normalization, info-gauge one-time-init, refresh atomicity, cardinality bounds, email-preserving annotation sanitizer; streaming-transformer write-through; middleware annotation-snapshot exposure; integration coverage in tests/metrics_integration_test.rs (4-label counter with both kinds, anonymous fallback, hash regex). (#610)
• Persistent-metrics SQLite store unit coverage (insert, query by time range, retention deletion, idempotent open, unknown-kind round-trip) and integration coverage in tests/metrics_persistence_test.rs (snapshot task lands rows in SQLite, retention prunes only old samples, retention hot-reload preserves in-flight snapshots, disk-usage smoke check). (#611)

v1.6.2 - 2026-05-10¶

Fixed¶

• /v1/responses and /v1/chat/completions now accept the OpenAI reasoning-API developer role (#603, #605, #606)
• Add MessageRole::Developer with a serde lowercase rename so "developer" deserializes as a first-class variant. The previous failure surfaced as a misleading did not match any variant of untagged enum ResponseInput rather than naming the unknown role; the implicit-message deserialization error now names the offending role string and lists the valid roles.
• Per-backend translation: pass through as developer for OpenAI-compatible servers; merge into the Anthropic top-level system parameter (concatenated with \n\n when both system and developer text are present, fixing a pre-existing overwrite bug); merge into Gemini system_instruction; map to system for Ollama (older builds reject developer).
• Chat Completions → Responses converter recognizes developer as instruction-bearing: the first occurrence becomes top-level instructions; subsequent occurrences remain as input items with their original role preserved on the wire.
• Treat developer and system equivalently in cross-cutting string-based recognition sites: prefix-cache key extraction, cross-provider fallback translation, the OpenAI-to-Anthropic transform's system-content extraction, the global-prompt injector's existing-system-message lookup, and the smart-routing classifier / LLM prompt builder.

Documentation¶

• Migrate the docs site from MkDocs to Zensical and restore brand styling (#602)
• Remove mkdocs.yml and mkdocs.ko.yml in favor of native zensical.toml and zensical.ko.toml, both rooted under the [project] namespace per Zensical's TOML schema; per-extension options live inside [project.markdown_extensions] as a dict (Zensical's config loader ignores any separate mdx_configs table).
• Replace docs/en/shared and docs/ko/shared symlinks with rsync -a --delete docs/shared/ docs/{en,ko}/shared/ invoked before each build, since Zensical does not follow symlinks for asset directories.
• Register the lablup brand color via Zensical's documented primary = "custom" mechanism plus a [data-md-color-scheme="default"][data-md-color-primary="custom"] selector in docs/shared/stylesheets/extra.css that defines the orange CSS variables.
• Mermaid is registered as a pymdownx.superfences custom fence rather than relying on the now-incompatible mermaid2 plugin; favicon falls back to logo.png when missing.
• Restore Zensical render output for icons, diagrams, and brand color (#604)
• Re-enable pymdownx.emoji with the zensical.extensions.emoji twemoji index/generator (replaces the removed materialx) so :material-*: icon syntax stops rendering as literal text.
• Reimplement the <!-- diagram: PATH --> ... <!-- /diagram --> ASCII-replacement as a Python-Markdown extension (docs/hooks/diagram_extension.py); the prior MkDocs on_page_content hook does not run because Zensical exposes no MkDocs hook lifecycle. Add docs/__init__.py and prefix builds with PYTHONPATH=. so the extension is importable from Zensical's console-script entry point.
• Set --md-primary-bg-color on the custom palette and override .md-header / .md-tabs so the orange brand band paints on top of Zensical's modern layout.
• Move the nav table above the first [project.X] sub-table in both TOMLs so it stops being silently parsed under [[project.extra.social]] (alphabetical fallback was producing an unsorted top menu and wrong prev/next footer neighbors).

Tests¶

• Regression coverage for system/developer concatenation in the Anthropic transform on both streaming and non-streaming paths, plus per-backend converter mapping for the developer role across all five backends and the Chat Completions → Responses converter's developer-then-system ordering (#605, #606).

Dependencies¶

• Bump redis 1.2.0 → 1.2.1 (#598).

v1.6.1 - 2026-05-07¶

Fixed¶

• Claude Opus 4.7 (claude-opus-4-7) now routes correctly through the Anthropic backend (#599, #600, #601)
• Extended the adaptive thinking API gate (uses_adaptive_thinking_api) to include 4.7-series model IDs. Claude Opus 4.7 requires thinking.type == "adaptive" + output_config.effort; sending the legacy budget_tokens shape produces HTTP 400.
• Added model_requires_adaptive_thinking and model_forbids_sampling_params predicates for 4.7-series request-shape rules: explicit manual thinking is normalized to adaptive thinking and temperature, top_p, and top_k are dropped unconditionally before forwarding.
• Extended opus_supports_max_effort to include Opus 4.7 so xhigh reasoning effort maps to output_config.effort = "max" on Opus 4.7.
• Added claude-opus-4-7 and claude-opus-4-7-latest to the built-in supported-models list and to model-metadata.yaml; the speculative claude-sonnet-4-7 entry is intentionally not advertised until Anthropic publishes it (defensive request-shape matching is retained for user-supplied configurations).

Documentation¶

• Update reasoning-effort docs (EN + KO) and backends.md to cover the Claude 4.7 family adaptive-thinking requirement and unconditional sampling-parameter deprecation (#600).

Tests¶

• Responses API regression coverage for Opus 4.7 adaptive thinking and unconditional sampling-parameter stripping; both transform paths (Chat Completions and Responses) for the 4.7 family with negative regression on Opus 4.6 / Sonnet 4.6 / Haiku 4.5 / Haiku 3.5 (#600, #601).

v1.6.0 - 2026-05-04¶

Added¶

• ChatGPT subscription / Codex backend authentication via OAuth device flow (#551, #592)
• continuum-router auth login --backend <name> runs the OpenAI Codex three-step headless device-code flow: POST /api/accounts/deviceauth/usercode to mint a one-time user_code, POST /api/accounts/deviceauth/token polling, and a PKCE exchange at /oauth/token. Standards-compliant RFC 8628 device flow remains available for any future provider that implements it; the new OpenAICodexDeviceFlowClient is selected automatically for provider: openai.
• Tokens are wrapped in SecretString, written to the configured token_store with mode 0600 on Unix using an O_CREAT|O_EXCL open + atomic rename; a random tempfile suffix prevents concurrent saves from colliding, and a partial write is unlinked on failure so secret material does not linger on disk.
• Access-token expiry is parsed from the JWT exp claim (with a 1-hour fallback for non-JWT tokens) and clamped to a useful minimum so a degenerate expires_in from the provider cannot trigger a refresh storm.
• Proactive refresh fires 60 s before expiry, single-flighted with a tokio::sync::Mutex. A 401 from the upstream backend triggers exactly one forced refresh and a single retry; the previous refresh token is preserved race-free when the provider omits refresh_token from a refresh response.
• The strategy reports an identity_fingerprint() (backend name, client_id, token_store) so that hot-reload rebuilds the strategy when any of those rotate, instead of silently keeping the prior in-memory state.
• The CLI strips C0/C1 control characters from verification_uri_complete and user_code before printing, so a hostile provider response cannot inject ANSI escapes that rewrite the terminal.
• Every device-flow and runtime request to auth.openai.com / chatgpt.com/backend-api/codex carries originator: codex_cli_rs (configurable via auth.oauth.originator) and a codex_cli_rs/<version> User-Agent (configurable via auth.oauth.user_agent), matching the official Codex CLI so Cloudflare admits the traffic instead of returning a 403 JS challenge.
• auth.type: oauth is accepted in YAML alongside the legacy o_auth snake_case rendering. client_id and scope default to the public Codex CLI values; only token_store is required for the ChatGPT-subscription case.
• Anthropic Messages and Chat Completions surfaces both transparently route to the ChatGPT Codex backend (#592)
• Any backend whose auth.type is oauth and whose provider uses the Codex flow (currently openai) is forced through the Responses API for every request, regardless of per-model responses_only metadata. chatgpt.com/backend-api/codex exposes /responses only — no /chat/completions — so chat-shaped models (e.g. gpt-5.5, alias-mapped claude-haiku-4-5) and unknown model IDs all dispatch through /v1/responses → …/backend-api/codex/responses. Non-OAuth OpenAI backends continue to honor the per-model responses_only flag.
• New core::url_utils::compose_backend_url centralizes backend URL composition for the three OpenAI-compatible roots (/v1, /openai, /backend-api/codex). Replaces ad-hoc ends_with("/v1") || ends_with("/openai") checks across proxy/backend.rs, http/handlers/responses.rs, http/streaming/handler.rs, services/responses/stream_service.rs, and the Anthropic handler so the /backend-api/codex rule applies uniformly.
• The proxy hot path (proxy/backend.rs, proxy/responses_only.rs, proxy/image_gen.rs, proxy/image_edit.rs) now flows through a backend-name-keyed AuthStrategyRegistry exposed on AppState via src/proxy/oauth_helper.rs. The helper looks up the strategy, calls refresh_if_needed() before sending, replaces the static-bearer header with one derived from the strategy, and force-refreshes + retries once on a 401. Static api_key auth continues to work unchanged when no strategy is registered.
• The Anthropic-compatible handler (src/http/handlers/anthropic/handler.rs) consults the same registry. Client-supplied Authorization: sk-ant-… and x-api-key headers are dropped when the backend has an OAuth strategy, instead of being forwarded to OpenAI as the bearer.
• Model fetcher detects OAuth-authed backends and falls back to the configured models list rather than probing /v1/models, since chatgpt.com/backend-api/codex does not expose a models endpoint.
• Codex-compatible Responses API extensions (#536, #537)
• POST /v1/responses/compact endpoint for context compaction — passthrough to OpenAI / Azure OpenAI native /v1/responses/compact; other backend types return 501.
• store field on ResponsesRequest (defaults to true) controls upstream session persistence; Codex sends store: false for ephemeral requests.
• output_text content part type alongside input_text so converters can differentiate assistant vs. user content in input items. All converters (OpenAI, Anthropic, Gemini) handle the new variant.

Documentation¶

• Sync Codex / Responses-API extensions across the root CHANGELOG.md and the Korean docs (ko/configuration/backends.md, ko/configuration/advanced.md, ko/api.md, ko/architecture.md); resolve all zensical build warnings on both EN and KO builds and preserve unicode in toc anchor slugs via pymdownx.slugs.slugify (#596).
• Clean up AI-slop patterns across English and Korean mkdocs sources — replace em dashes in prose, remove filler/slop words, rewrite trailing participial clauses and inflated verbs, collapse colon+bullet AI-style intros, and replace closing summary slop with concrete next-action links (#597).

CI/CD¶

• Bump apple-actions/import-codesign-certs from 6 to 7 (#590).

Dependencies¶

• Bump tokio 1.51.0 → 1.52.1, axum 0.8.8 → 0.8.9, reqwest 0.13.2 → 0.13.3, clap 4.6.0 → 4.6.1, fastrand 2.4.0 → 2.4.1, uuid 1.23.0 → 1.23.1, rand 0.10.0 → 0.10.1, and lru 0.16.3 → 0.16.4 (#595).

v1.5.6 - 2026-04-29¶

Fixed¶

• /v1/chat/completions returned HTTP 502 responses_parse_failed for responses_only reasoning models (gpt-5.4-pro, gpt-5.5-pro). OpenAI's /v1/responses payload for these models contains output items shaped like { "id": "rs_...", "type": "reasoning", "summary": [] }, but OutputItem::Reasoning required content and status, so serde rejected the payload with missing field 'content'. The Anthropic Messages surface bypassed the strict variant on a different conversion path, masking the bug until directly tested. content and status are now optional on OutputItem::Reasoning; reasoning items are dropped before reaching Chat Completions clients (per existing project policy), so body shape is irrelevant beyond successful deserialization. (#594)

Changed¶

• Realign gemini-3.1-pro-preview as the canonical metadata id for the Gemini 3.1 Pro family in model-metadata.yaml, with gemini-3.1-pro (and existing -latest / -customtools forms) demoted to aliases. Matches what generativelanguage.googleapis.com actually serves today — the canonical gemini-3.1-pro form returns 404 from upstream — and avoids implying GA availability that does not exist yet. The metadata cache still resolves both forms to the same entry. Note: alias-to-canonical rewriting on the upstream-bound payload is out of scope for this release; clients calling with the gemini-3.1-pro alias will still hit upstream 404 until that work lands. (#594)
• Sample config.yaml registers the newly-available pro / 5.5 family models so the responses_only dispatch path can be exercised end-to-end against real upstreams (gpt-5.4-pro, gpt-5.2-pro, gpt-5.5, gpt-5.5-pro, claude-opus-4-7, gemini-3.1-pro, gemini-3.1-pro-preview); duplicate claude-haiku-4-5 entry removed.

v1.5.5 - 2026-04-27¶

Added¶

• Transparent Responses-API routing for OpenAI Pro models (epic #581)
• New responses_only: true capability flag in model-metadata.yaml and the built-in OpenAI registry marks gpt-5.2-pro, gpt-5.4-pro, and gpt-5.5-pro as served only on /v1/responses upstream (#574, #582)
• /v1/chat/completions requests for responses_only models are dispatched to the upstream /v1/responses endpoint and translated back into a strict-mode chat.completion (or chat.completion.chunk for streaming) envelope, transparent to the client. Stream usage is gated by stream_options.include_usage, and per-model latency / success counters are recorded for the responses_only path (#578, #584)
• /anthropic/v1/messages requests for responses_only models are converted to the Responses API shape, dispatched to /v1/responses, and translated back into Anthropic Messages JSON (or the Anthropic SSE event sequence for streaming) — tool-call round-trips, web-search emulation, and Unix-socket transports all branch on the flag (#575, #577, #583, #585, #586)
• Anthropic Messages <-> Responses request transformer covers system → instructions, tools, tool_choice (including disable_parallel_tool_use → parallel_tool_calls: false), max_tokens → max_output_tokens, reasoning effort derivation, and multi-turn tool round-trips; the response transformer preserves thinking/text/tool_use ordering and stop-reason fidelity (#575, #583)
• SSE streaming bridge (AnthropicResponsesStreamTranslator) maps Responses API events to Anthropic Messages events while preserving Anthropic's strict event-ordering invariants (single message_start, paired content_block_start/content_block_stop, terminal message_stop); handles mid-stream error / response.failed / response.cancelled, response.incomplete → stop_reason: max_tokens, deferred input tokens, and graceful early-close synthesis (#576, #585)
• Only OpenAI and Azure OpenAI backends serve /v1/responses; pairing a responses_only model with another backend type produces a 400 invalid_request_error before any upstream call (rejection fires on both /v1/chat/completions and /anthropic/v1/messages surfaces) (#577, #589)
• The first dispatch per (backend, model) pair logs at info level so operators can confirm Responses-API routing without enabling debug logs
• Anthropic Messages → Responses requests explicitly send store: false to avoid upstream side-effects (#589)
• 22 deterministic, in-process integration tests covering the {Anthropic, Chat} × {gpt-5.4-pro, gpt-5.2-pro} × {non-streaming, streaming} × {plain, tool-call, reasoning} matrix, mid-stream backend-failure negatives on both surfaces, and an upstream byte-fragmentation regression guard (#579, #588)
• Documented in docs/en/configuration/advanced.md (Responses-API-only Models section split into Models-marked-out-of-the-box, Marking-a-new-model, Dispatch-behavior, and Backend-type-constraint subsections), docs/en/architecture.md (Responses-API Routing data-flow diagram), and the docs/en/api.md Chat Completions and Anthropic Messages surface notes with a Transparent-Responses-API-routing subsection (#580, #587)

Fixed¶

• Chat Completions responses-only routing now rejects incompatible-only backend configs before upstream dispatch and chooses a compatible OpenAI/Azure Responses backend when available (#589)
• Chat assistant tool_calls[] are preserved as Responses function_call input items for stateless tool-result turns over /v1/chat/completions (#589)

v1.5.4 - 2026-04-25¶

Changed¶

• Refresh model-metadata.yaml for late-April 2026 frontier model releases (#572, #573)
• Add GPT-5.5 ($5/$30 per 1M, 1M context, knowledge cutoff 2025-12, omnimodal, leads Terminal-Bench 2.0 at 82.7%) and GPT-5.5 Pro ($30/$180 per 1M, Responses API only, deep reasoning) — released 2026-04-23
• Add DeepSeek V4 Pro (1.6T total / 49B active MoE, 1M context, 384K max output, three reasoning effort modes) and DeepSeek V4 Flash (284B total / 13B active MoE, 1M context, 384K max output) with deepseek-chat and deepseek-reasoner retained as deprecated aliases per official API docs — released 2026-04-24
• Add gpt-image-2 (token-billed instead of per-image: text $5/$30, image $8/$30 per 1M tokens; 1K/2K/4K resolution tiers; ~99% text accuracy in any language; built-in reasoning before generation; context-aware multi-turn editing; gpt-image-2-latest alias) — released 2026-04-21
• Add Claude Opus 4.7 ($5/$25 per 1M, 1M context, 128K max output, knowledge cutoff 2026-01, high-resolution image support up to 2576px / 3.75MP, new tokenizer with ~1.0–1.35× token usage vs prior models, new xhigh effort level) — released 2026-04-16
• Promote Gemini 3.1 series from preview to GA, retaining -preview suffix as alias for fallback compatibility (#573)
• gemini-3.1-pro-preview → gemini-3.1-pro (with gemini-3.1-pro-preview, gemini-3.1-pro-preview-customtools, and gemini-3.1-pro-latest aliases)
• gemini-3.1-flash-image-preview → gemini-3.1-flash-image (with gemini-3.1-flash-image-preview, nano-banana-2, and gemini-3.1-flash-image-latest aliases)
• gemini-3.1-flash-lite-preview → gemini-3.1-flash-lite (with gemini-3.1-flash-lite-preview and gemini-3.1-flash-lite-latest aliases)
• Updated gemini-3-flash-preview deprecation note to point to the new GA gemini-3.1-pro id

v1.5.3 - 2026-04-23¶

Added¶

• HuggingFace repo-prefix stripping as a new matching phase (phase 5) in src/models/pattern_matching.rs (#555)
• try_strip_hf_repo_prefix() validates a vendor/repo (or org/team/repo) prefix against a MAX_PREFIX_SEGMENTS = 3 bound, rejects empty segments (/repo, vendor/, vendor//repo), and rejects any ASCII whitespace before returning the residual
• Phase 5 re-enters phases 1-4 on the stripped residual with a structurally-enforced recursion depth of exactly 1 (the re-entry call clears the allow_prefix_strip gate), so prefix stripping composes with the existing layered suffix peel in a single lookup — the motivating case unsloth/Qwen3.6-35B-A3B-GGUF now resolves to qwen3.6-35b-a3b without any hand-registered alias
• Phase 5 runs before the wildcard phase; the blast-radius audit confirmed no *-bearing alias in model-metadata.yaml contains /, so the ordering change is behavior-neutral for existing routing
• Phase numbering in tracing output realigned to match the documented phase chain (previous code emitted phase = 7 for the namespace fallback while comments called it phase 6)
• 12 new unit tests covering standard HF form, composition with suffix peel, case-sensitive vendor, registered-alias precedence, unresolvable residual, three-segment form, segment-cap rejection, no-slash input, whitespace rejection, empty segments, re-entry bounding, and alias-phase precedence
• 9 new integration tests in tests/format_suffix_normalization_test.rs exercising the full RouterConfig / BackendConfig public API through phase 5
• Pipeline doc updated in docs/en/configuration/advanced.md (and Korean counterpart) with a new "HuggingFace repo-prefix stripping (phase 5)" section covering the composition semantics, security bounds, and out-of-scope list (hyphen prefixes, HF API discovery)

Changed¶

• Replaced the previous phase-6 namespace fallback with the new phase-5 HuggingFace prefix-strip layer. The previous phase was case-sensitive and did not compose with suffix peel; the new phase applies stricter input validation (segment cap, empty-segment rejection, whitespace rejection) but composes with phase 4's case-insensitive peel through the bounded re-entry. Pathological inputs above MAX_PREFIX_SEGMENTS (3) — such as provider/deep/nested/model — are now rejected by phase 5 rather than silently matched via recursive rsplit_once fallback (#555)
• Aliases currently classified as vendor-prefix in the #560 audit (e.g., Qwen/Qwen3.6-35B-A3B, MiniMaxAI/MiniMax-M2.5) are now peel-coverable-adjacent post-#555: phase 2 still wins on the explicit alias, but phase 5 + phase 4 together reach the same metadata. Retroactive removal is deferred to a follow-up audit per #555 design section 7

Fixed¶

• POST /anthropic/v1/messages now works when the selected backend is configured with a unix:// URL (#567)
• Native Anthropic backends and OpenAI-compatible backends both work over Unix sockets, for both non-streaming and streaming requests
• Socket paths containing spaces (e.g. macOS ~/Library/Application Support/...) are handled correctly
• Auth header selection (x-api-key for Anthropic backends, Authorization: Bearer for OpenAI-compatible backends) is correct on the Unix socket path
• anthropic-version header is added automatically for Anthropic backends on the Unix socket path, matching the HTTP path behavior

v1.5.2 - 2026-04-21¶

Added¶

• Regression tests locking down the transport-layer passthrough contract for llama.cpp and MLxcel backends (#562)
• New tests/llamacpp_passthrough_test.rs and tests/mlxcel_passthrough_test.rs covering all four passthrough call sites: direct backend execute_chat_completion, factory-built backend (BackendFactory -> LlamaCppBackend), proxy/backend.rs HTTP handler, and the streaming handler
• New test_mlxcel_factory_backend_passthrough_nonstandard_fields asserts that BackendFactory -> LlamaCppBackend::execute_chat_completion preserves non-standard fields byte-for-byte at transport time
• Anthropic input test (tests/anthropic_input_test.rs) extended with explicit passthrough coverage
• docs/en/architecture/backend-passthrough.md and its Korean counterpart docs/ko/architecture/backend-passthrough.md documenting the passthrough contract, the four guarded call sites, and the list of router-side transforms that run before transport (global_prompts, transform_payload_for_openai for o1/o3/gpt-5*, web_search injection) (#562, #563)
• docs/reports/alias-audit-2026-04.md classifying every alias in model-metadata.yaml into peel-redundant, peel-redundant-but-kept, and peel-independent categories, with an "aliases vs peel" policy section added to docs/en/configuration/advanced.md (and the Korean counterpart) explaining when to prefer each mechanism (#560)

Changed¶

• Narrowed the passthrough contract from an implied "byte-equivalent" global guarantee to a transport-layer scope — the router may still run global_prompts injection, o1/o3/gpt-5* payload transforms, and web_search tool injection before transport, but no provider-specific rewriting happens at the transport boundary (#563)
• Comment-only clarifications in src/http/streaming/handler.rs, src/infrastructure/backends/factory/backend_factory.rs, src/infrastructure/backends/llamacpp/backend.rs, and src/proxy/backend.rs
• Audited model-metadata.yaml aliases for peel-normalization redundancy: removed aliases that differ from the canonical ID only by suffixes already handled by the layered peel (-4bit, -q4_k_m, -fp8, -gguf, -mlx, -awq, etc.), while preserving aliases that encode canonical flavor variants (-qat, -instruct) or disambiguate parameter counts (#557)
• New tests/alias_audit_helper.rs and tests/format_suffix_normalization_test.rs enforce the peel-vs-alias boundary going forward

CI¶

• Target Ubuntu 26.04 LTS (Resolute) instead of 25.10 (Questing) in the Debian build workflow
• Fall back to createdAt when release publishedAt is null in debian/update-changelog.sh to prevent changelog regression when the latest release is still in draft

v1.5.1 - 2026-04-20¶

Added¶

• Built-in web_search tool for self-hosted LLM backends (#553)
• Router-level tool transparently injected into chat completion requests for vLLM, Ollama, llama.cpp, MLxcel, LM Studio, Continuum Router, and Generic backends
• Pluggable SearchProvider trait under src/services/search/ with SerperProvider implementation; Exa and Brave scaffolded behind the same trait
• Configurable inject_policy (auto/always/never) with per-backend overrides; commercial backends (OpenAI, Azure, Gemini, Anthropic) left untouched so their native web_search continues to flow through unchanged
• Bounded non-streaming tool-execution loop parses web_search tool calls, executes the provider, appends tool-role results, and re-invokes the backend up to max_tool_iterations rounds
• New BackendTypeConfig::is_self_hosted / is_commercial helpers covered by unit tests enforcing the commercial/self-hosted partition invariant
• API keys redacted in Debug output and never logged; hot-reload friendly WebSearchConfig with ${ENV} substitution
• Prometheus counters for tool calls, injections, and iteration-cap hits under src/metrics/web_search
• Layered quantization and format suffix normalization for model metadata lookup (#549)
• New layered_format_strip() in src/models/pattern_matching.rs iteratively peels allowlisted quantization/format/flavor tokens from the right side of a model ID, retrying exact-id/alias/date-suffix matches after each peel
• Token categories: BIT_WIDTH, GGUF_QUANT, FP_FORMAT, INT_FORMAT, LIBRARY, IMATRIX, UNSLOTH, CONTAINER, FLAVOR (all case-insensitive)
• Parameter-count suffixes preserved: -Nbit stripped as quantization; -Nb, -aNb, -eNb, -0.6b kept as parameter counts
• Canonical base IDs ending in allowlisted flavors (e.g. gemma-3-12b-qat) win via exact-id match before peel runs
• Normalization pipeline wired into find_matching_config, BackendConfig::get_model_metadata, RouterConfig::get_model_metadata, RouterConfig::get_thinking_pattern_config, resolve_model_tier (routing), and get_model_profile (admin)
• Model metadata for GLM 5.1, Qwen 3.6, and MiniMax M2.7 (#548)
• Teams release notification posted to Microsoft Teams via Power Automate webhook after build and Docker jobs

Changed¶

• Migrate documentation toolchain from MkDocs + Material for MkDocs to Zensical — reads mkdocs.yml natively and bundles required extensions

Fixed¶

• Security: Cap layered peel phase with MAX_MODEL_ID_LEN=256 and MAX_PEEL_ITERATIONS=8 to eliminate DoS via pathological model IDs (previously O(n²) allocation on inputs like -4bit-4bit-4bit-...)
• Security: Enforce 256-char model field length at /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/embeddings/sparse (parity with existing /v1/responses check)
• Consolidate 7-phase metadata matching pipeline into a single implementation (find_matching_config_slice) with thin adapters at each call site, eliminating drift between BackendConfig, Config::get_model_metadata, Config::get_thinking_pattern_config, and find_matching_config
• Replace cfg.to_ascii_lowercase() == peel with str::eq_ignore_ascii_case on the hot path (~4000 fewer per-request String allocations)
• Pin Pygments <2.20 to fix MkDocs build failure (superseded by Zensical migration)

CI¶

• Bump softprops/action-gh-release from 2 to 3 (#544)
• Bump actions/github-script from 8 to 9 (#545)
• Bump actions/upload-pages-artifact from 4 to 5 (#554)

Documentation¶

• Document suffix-order ambiguity (-qat-4bit vs -4bit-qat) and internal peel phase bounds in docs/en/configuration/advanced.md
• Add pattern_matching.rs to Model Aggregation Service module listing in docs/en/architecture.md with cross-reference to suffix normalization section
• New docs/en/web-search.md feature documentation; config.yaml.example extended with web_search section

v1.5.0 - 2026-04-11¶

Added¶

• Smart routing system with model tier & capability profile registry (#525, #531)
• Rule-based request classifier & smart routing policy engine (#526, #532)
• Load-aware dynamic tier adjustment (#527, #533)
• LLM-based request classifier with hybrid mode (#528, #534)
• Smart routing observability, admin API & documentation (#529, #535)
• Codex-compatible Responses API extensions (#536, #537)

Changed¶

• Upgrade core dependencies — axum 0.8, sha2 0.11, rand 0.10 (#523)
• Add Gemma 4 model family metadata (#538)

Fixed¶

• Complete smart routing integration gaps
• Increase DefaultTransformer PDF size limit from 20MB to 32MB (#542)

CI¶

• Bump actions/deploy-pages from 4 to 5 (#521)

Dependencies¶

• Bump the minor-and-patch dependency group with 4 updates (#539)

Documentation¶

• Add Codex-compatible Responses API gap analysis report

v1.4.5 - 2026-03-27¶

Fixed¶

• Return 400 error when file references are used without file service configured (#519)

Changed¶

• Add GLM-5-Turbo model metadata (#516)

Documentation¶

• Fix Korean anti-AI-slop violations in ko/ documentation
• Fix slop word and transition word in api.md

v1.4.4 - 2026-03-18¶

Fixed¶

• Fix Anthropic thinking failing for high/xhigh reasoning effort — budget_tokens (32768) exceeded default max_tokens (16384), causing API rejection (#514)
• Auto-adjust max_tokens to budget_tokens + 4096 when thinking is enabled and budget exceeds max

Changed¶

• Add GPT-5.4 model family: gpt-5.4, gpt-5.4-pro, gpt-5.4-mini, gpt-5.4-nano with 1M context window (#515)
• Update Gemini 3 series: add gemini-3.1-pro-preview, gemini-3-flash-preview, gemini-3.1-flash-lite-preview; mark gemini-3-pro-preview as deprecated
• Recognize Gemini 3 Flash and 3.1 Flash-Lite as thinking models for include_thoughts auto-injection
• Update Claude 4.6 models: context window to 1M (GA), fix Sonnet 4.6 max_output to 64K, correct knowledge cutoffs
• Update config examples and documentation with latest model names across 8 files

v1.4.3 - 2026-03-18¶

Fixed¶

• Fix Gemini thinking models (2.5 Pro, 3 Pro, etc.) not returning reasoning_content in streaming responses through the router (#513)
• Replaced transform_payload_for_gemini() with transform_request_gemini() across all three Gemini streaming paths to ensure include_thoughts: true auto-injection

v1.4.2 - 2026-03-17¶

Changed¶

• Change mid-stream fallback default to enabled for improved streaming reliability (#504)
• Breaking: Mid-stream fallback is now enabled by default; set mid_stream_fallback.enabled: false to restore previous behavior

Documentation¶

• Add failover latency tuning guide for optimizing fallback behavior

v1.4.1 - 2026-03-17¶

Added¶

• Mid-stream fallback for streaming inference (#497) — when a backend fails mid-stream during SSE streaming, the router transparently retries with a fallback backend

Changed¶

• Decouple pre-stream fallback from mid-stream fallback (#500) — each can now be independently enabled/disabled
• Bump dependency versions to latest major releases

Fixed¶

• Fix streaming config changes not detected in hot reload system (#503)
• Fix mid-stream connection errors leaking to client during fallback (#502)
• Remove unused config crate dependency

CI¶

• Bump dorny/paths-filter from 3 to 4 (#493)
• Bump actions/create-github-app-token from 2 to 3 (#494)

v1.4.0 - 2026-03-14¶

Added¶

• Prefix-aware routing: PrefixAwareHash selection strategy with Consistent Hash with Bounded Loads (CHWBL) (#455, #457, #461)
• Response caching: SHA256-based cache key computation with streaming response buffering and post-completion caching (#456, #459, #462)
• Multi-tier CacheStore: in-memory backend (#466), Redis/Valkey backend with connection pooling (#467), and S3-backed tiered L1/L2 cache (#483)
• KV cache index: shared data structure (#470), KV event consumer for vLLM backend streams (#471), prefix overlap scoring integrated into backend selection (#473), configuration/metrics/admin endpoints (#474)
• Tiered KV cache with storage-tier awareness (GPU hot / external warm) (#484)
• Disaggregated prefill/decode orchestration with external KV tensor transfer (#485)
• Anthropic cache_control breakpoint auto-injection (#460)
• Multimodal embedding support for Gemini Embedding 2 (#492)
• Shared cache configuration and operational metrics (#468)
• 30 new models added to model-metadata.yaml (#472)

Changed¶

• Rename VAST-specific identifiers to generic S3/external storage names (#490) — update configuration files if using VAST-specific field names

Fixed¶

• Make RequestExecutor transport-aware for Unix socket paths with spaces (#488)
• Replace relative source tree links with GitHub URLs in docs

CI¶

• Bump docker/setup-qemu-action from 3 to 4 (#428)
• Bump docker/metadata-action from 5 to 6 (#426)
• Bump docker/setup-buildx-action from 3 to 4 (#429)
• Bump docker/build-push-action from 6 to 7 (#430)
• Bump docker/login-action from 3 to 4 (#427)

Documentation¶

• Comprehensive KV cache feature documentation, benchmarks, and config examples (#477)
• VAST Data connection guide and integration examples (#486)
• Sync Korean documentation with English counterparts
• Split monolithic configuration.md into 6 smaller files

v1.3.0 - 2026-03-12¶

Added¶

• Agent Communication Protocol (ACP) support with JSON-RPC 2.0 protocol layer and stdio transport (#414, #420)
• ACP session management with protocol lifecycle, initialize/shutdown handshake (#415, #421)
• ACP-to-LLM inference pipeline with streaming support (#416, #422)
• ACP tool call reporting and permission delegation (#417, #423)
• MCP-over-ACP bridge for MCP server tunneling (#418, #424)
• ACP agent registry with metadata and configuration support (#419, #425)
• ACP integration tests for protocol lifecycle and session management

Fixed¶

• Resolve clippy field_reassign_with_default warnings in ACP integration tests

CI¶

• Bump actions/upload-artifact from 6 to 7 (#398)

Documentation¶

• ACP architecture documentation with MkDocs integration
• ACP practical usage guide with IDE integration examples
• KV cache integration plan for router-level caching strategies

v1.2.1 - 2026-03-07¶

Added¶

• MLxcel backend type support for MLX-based model serving (#412, #413) — fully API-compatible with llama-server, reusing the same backend implementation for health checks, model discovery, and proxying

v1.2.0 - 2026-03-06¶

Added¶

• Admin Statistics API with comprehensive request-level statistics collection and reporting (#409)
• Endpoints: GET /admin/stats, GET /admin/stats/models, GET /admin/stats/backends, POST /admin/stats/reset
• Time-windowed queries, token usage tracking, latency percentiles (p50, p95, p99)
• Statistics persistence with configurable snapshot path, interval, and staleness checks (#410, #411)
• Atomic writes, restore on startup, final snapshot on graceful shutdown

Documentation¶

• Add admin stats and persistence to configuration guide
• Add post-refactoring benchmark report for v1.1.0 (#407)

v1.1.1 - 2026-03-04¶

Added¶

• Embeddable library crate (Phase 1) — use continuum-router as a Rust dependency (#394)
• Type-safe config builders for programmatic library usage (#400)
• Cargo feature flags for optional library dependencies (#399)
• Persistent storage for runtime API keys (#405)
• New LLM model metadata entries (#403)

Fixed¶

• Fix Gemini-specific transforms incorrectly applied in Anthropic handler (#404)

v1.1.0 - 2026-03-01¶

Added¶

• Embedded WebUI for configuration management and API key administration (#388)
• Windows AF_UNIX socket support via socket2 crate (#390)
• Nano Banana 2 (Gemini Image Generation) support

Fixed¶

• Resolve compilation error in ClientAddr::is_unix for tuple variant matching
• Resolve Windows AF_UNIX socket accept failure and config validation
• Accept Windows absolute paths in Unix socket config validation (#393)
• Resolve Windows compilation errors in Unix socket tests and transport parsing (#392)

v1.0.0 - 2026-02-19¶

Added¶

• Continuum Router federation — router-to-router chaining as a new backend type (#385)
• LM Studio as a dedicated backend type (#381)
• Anthropic adaptive thinking effort parameter (output_config.effort) (#384)
• Adaptive thinking and auto reasoning effort level across backends (#378)
• Cohere/Jina-compatible rerank and sparse embedding endpoints (#374)
• BGE-M3 and multilingual embedding model support (#373)
• Claude Opus 4.6 model metadata
• Qwen3-Coder-Next, Qwen3-VL-30B/8B model metadata

Changed¶

• Handle SIGTERM for graceful shutdown on Unix systems (#370)
• Reduce per-backend filter and model metadata log verbosity during model refresh (#371, #375)

CI¶

• Replace Ubuntu 24.10 with 25.10 in deb build matrix (#376)

v0.36.1 - 2026-01-30¶

Fixed¶

• Trigger immediate health check after sync_backends during hot reload (#368) — new backends now available within 1-2 seconds instead of up to 30 seconds
• Sync health_check_info and use URL-based updates during hot reload (#369) — new backends properly receive API key authentication
• Accelerate health checks for recently added backends — 1-second check interval for 5 minutes after addition
• Trigger model cache refresh when backends transition to healthy state with 5-second debounce

v0.36.0 - 2026-01-27¶

Added¶

• Native Anthropic Messages API handler with endpoint routing (#355)
• Anthropic to OpenAI request/response transformation (#356, #357)
• Anthropic streaming response format (#358)
• Direct Anthropic to Gemini request/response transformation (#359)
• File_id source type and file resolution for Anthropic input (#360)
• Claude Code compatibility for Anthropic handler (#365)
• Tiered token counting for all backend types
• Parallel file reference resolution for improved performance
• Anthropic-version header format validation

Fixed¶

• Require HTTPS for image and document URLs to prevent SSRF
• Return generic error messages to clients instead of backend details
• Use authenticated user_id from API key for file ownership checks
• Use UUID v4 for secure message/tool ID generation
• Place tool messages before user text in Anthropic-to-OpenAI conversion
• Override stop_reason to tool_use when tool_use blocks are present
• Apply max_completion_tokens conversion for OpenAI-routed Anthropic requests
• Propagate file access denied and not found errors to client
• Call current_config() once per request for consistent behavior

Refactored¶

• Extract common SSE event type and data extraction logic
• Add parse_bytes method to SseParser for proper UTF-8 handling
• Remove unnecessary Arc wrapper in AnthropicFileResolver
• Box FileResolutionResult::Resolved to reduce enum size

v0.35.0 - 2026-01-23¶

Added¶

• Gemini 3 thoughtSignature support in function calling (#354)
• PDF support for OpenAI and Anthropic file transformers (#340)
• Text/plain support for AnthropicFileTransformer (#342)

Fixed¶

• Add PDF support to DefaultTransformer and file resolution (#343)
• Add tool message transformation to non-streaming Anthropic requests (#344)
• Reject non-image files in DefaultTransformer with clear error message (#338)
• Fix AI SDK incompatibility with Responses API streaming format (#335)

v0.34.0 - 2026-01-16¶

Added¶

• Automatic quality parameter conversion between DALL-E and GPT Image models (#330)

Changed¶

• Native Anthropic conversion for Responses API PDF file uploads (#332)

Fixed¶

• Gemini streaming tool_calls compatibility fixes (#333) — missing index field, tool_choice format preservation, unnecessary transformation removal

v0.33.0 - 2026-01-13¶

Added¶

• /v1/embeddings endpoint for embedding API support (#319)
• Resolve local file_id references in Responses API requests (#326)
• user_data and evals purpose values for Files API (#322)

Fixed¶

• Use flat tool format for Responses API function tools (#324)
• Improve Unix socket test stability for parallel execution (#328)

v0.32.0 - 2026-01-09¶

Added¶

• Reasoning effort documentation and improved xhigh fallback logging (#317)

Fixed¶

• Support implicit message type inference in Responses API InputItem (#316)

Refactored¶

• Optimize InputItem deserializer and add invalid role test

v0.31.5 - 2026-01-09¶

Added¶

• Responses API pass-through support for native OpenAI backends (#313) — smart routing based on backend type with direct forwarding to /v1/responses endpoint
• OpenAI Responses API file input types (#311) — support for input_text, input_file, input_image content parts with SSRF validation

Fixed¶

• Forward raw backend error responses in pass-through mode
• Address security and performance issues in Responses API pass-through

v0.31.4 - 2026-01-07¶

Fixed¶

• Use current_config() for hot reload support in proxy handlers (#310) — API key and configuration changes via hot reload now properly apply to new requests

v0.31.3 - 2026-01-06¶

Fixed¶

• Add Anthropic transformations to Unix socket transport (#308) — Unix socket transport now applies the same request/response transformations as HTTP transport
• Preserve stream parameter for non-streaming Anthropic requests (#306)

v0.31.2 - 2026-01-05¶

Added¶

• Non-streaming support for Anthropic backend requests
• Tool call and tool result transformation for Anthropic backend — enables multi-turn tool use conversations

v0.31.1 - 2026-01-04¶

Fixed¶

• Non-streaming Anthropic requests failing with wrong authentication header (#301) — now correctly uses x-api-key header instead of Authorization: Bearer

v0.31.0 - 2026-01-04¶

Added¶

• Unix socket server binding alongside TCP (#298) — supports unix: URI scheme, socket_mode configuration, auto-cleanup
• Reasoning parameter support for Responses API (#296) with nested format and low/medium/high/xhigh effort levels
• xhigh reasoning effort support for GPT-5.2 thinking models with auto-downgrade for unsupported models
• Configurable health check endpoints per backend type (#293) — custom endpoint, fallback endpoints, method, body, accept_status, and headers

Changed¶

• Comprehensive reasoning parameter normalization across backends (#294)

v0.30.0 - 2026-01-01¶

Added¶

• Wildcard patterns and date suffix handling in model aliases (#286) — automatic date suffix normalization, * pattern matching (prefix, suffix, infix), zero-config date handling

Fixed¶

• Apply default URL for Anthropic backend when not specified (#288)
• Replace owned_by placeholders with backend-type-specific values (#287)

Documentation¶

• Translate wildcard pattern and date suffix handling documentation to Korean (#289)

v0.29.0 - 2026-01-01¶

Added¶

• Accelerated health checks during backend warmup (#282) — 1s interval on HTTP 503, configurable via warmup_check_interval and max_warmup_duration
• --model-metadata CLI option for specifying model metadata file path at runtime (#281)

Fixed¶

• Replace OpenAI owned_by placeholder with 'openai' (#280)
• Prevent race condition in Admin API concurrent backend creation (#278)
• Fix missing processing steps in hot reload (#277)
• Cloud backends now show available: true in /v1/models/{model_id} (#272)

v0.28.0 - 2025-12-31¶

Added¶

• SSE streaming support for tool calls (#258)
• llama.cpp tool calling auto-detection via /props endpoint (#263)
• Extended /v1/models/{model_id} endpoint with rich metadata fields (#262)
• Tool result message transformation for multi-turn conversations (#265)
• Backend-specific owned_by placeholders for llamacpp, vllm, ollama, http (#267)

Changed¶

• Improved --help output formatting with title header and project attribution (#269)

Fixed¶

• Sync model metadata cache with ConfigManager (#270)

v0.27.0 - 2025-12-29¶

Added¶

• Complete Unix socket support for model discovery and SSE streaming (#248, #252, #253, #254, #256)
• SSE/streaming for Unix socket backends
• Backend type auto-detection for Unix sockets
• vLLM and llama.cpp model discovery via Unix sockets
• Tool call transformation across all backends (#244, #245, #246) — tool definitions, tool_choice, and tool call responses for Anthropic, Gemini, and llama.cpp

v0.26.0 - 2025-12-27¶

Added¶

• GET /v1/models/{model} endpoint for single model retrieval with real-time availability status (#236)

v0.25.0 - 2025-12-26¶

Added¶

• CORS (Cross-Origin Resource Sharing) support (#234) — configurable origins, wildcard patterns, custom schemes (e.g., tauri://localhost), preflight cache
• Unix Domain Socket backend support (#232) — unix:///path/to/socket scheme, lower latency than localhost TCP

v0.24.0 - 2025-12-26¶

Added¶

• llama.cpp backend support for local LLM inference (#230)
• Allow router to start without any backends configured (#226)

Changed¶

• Enable hot reload for backend additions/removals from config (#229)

v0.23.1 - 2025-12-25¶

CI¶

• Add Windows x86_64 build target to release workflow (#224)

v0.23.0 - 2025-12-23¶

Added¶

• GLM 4.7 model support with thinking capabilities (#222)
• GCP Service Account authentication support for Gemini (#208)
• Distributed tracing with correlation ID propagation (#207) — W3C Trace Context with traceparent header
• Thinking pattern metadata for models with implicit start tags (#218)
• Model metadata for NVIDIA Nemotron 3 Nano, Qwen Image Layered, and Kakao Kanana-2 (#202)
• ASCII diagram to image replacement system for MkDocs (#200)

Fixed¶

• Prevent cache stampede with singleflight, stale-while-revalidate, and background refresh (#220)
• Apply global_prompts changes via hot reload (#219)
• Invalidate model cache when backend config changes (#206)

CI¶

• Skip Rust tests in CI when only non-code files change (#204)
• Bump actions/github-script from 7 to 8 (#210)
• Bump apple-actions/import-codesign-certs from 3 to 6 (#212)
• Bump actions/cache from 4 to 5 (#211)
• Bump actions/checkout from 4 to 6 (#209)

v0.22.0 - 2025-12-19¶

Added¶

• Docker support with pre-built binary images — Debian (~50MB) and Alpine (~10MB) with multi-arch support (#198)
• Container health check CLI (--health-check) for orchestration (#198)
• Docker Compose quick start configuration
• Automated Docker image publishing to ghcr.io in release workflow
• MkDocs documentation website with Material theme (#183)
• Korean documentation translation (i18n) — complete localization of all 20 documentation files (#190)
• Security policy with vulnerability reporting process (#191)
• Dependency security auditing with cargo-deny and Dependabot (#192)

Changed¶

• Integrate orphaned architecture documentation into MkDocs site (#186)
• Rename documentation files to lowercase kebab-case for URL-friendly filenames

Fixed¶

• Fix health check response validation logic bug (operator precedence)
• Fix address parsing fallback silently hiding configuration errors
• Fix IPv6 address formatting in health check

v0.21.0 - 2025-12-19¶

Added¶

• Gemini 3 Flash Preview model support (#168)
• Default authentication mode for API endpoints (#173) — permissive (default) or blocking mode
• Backend error passthrough for 4xx responses (#177) — parse and forward original error messages from OpenAI, Anthropic, and Gemini

Fixed¶

• Handle UTF-8 multi-byte character corruption in streaming responses (#179)
• Strip response_format parameter for GPT Image models (#176)
• Allow auto-discovery for all backends except Anthropic (#172)
• Always return b64_json field for Gemini image generation responses (#181)

v0.20.0 - 2025-12-18¶

Added¶

• Image variations support for Gemini (nano-banana) models (#165)
• Image edit support for Gemini (nano-banana) models (#164)
• Enhanced /v1/images/generations with streaming and GPT Image features (#161)
• gpt-image-1.5 model support (#159)
• /v1/images/variations endpoint (#155)
• /v1/images/edits endpoint for image editing and inpainting (#156)
• External Markdown file support for system prompts with REST API management (#146)
• Automatic model discovery for backends without explicit model list (#142)
• Solar Open 100B model

Security¶

• API key redaction to prevent credential exposure in logs and error messages (#150)

Changed¶

• Optimized release binary size from 20MB to 6MB (70% reduction) (#144)

Refactored¶

• Split large files to keep each under 500 lines (#147, #148)

v0.19.0 - 2025-12-13¶

Added¶

• Runtime Configuration Management API (#139)
• Configuration query, modification, save/restore, and backend management APIs
• Sensitive information masking, JSON Schema generation, configuration history with rollback (up to 50 entries)
• Comprehensive Admin REST API reference documentation
• 33 integration tests for configuration API endpoints

Security¶

• Input validation with 1MB content limit and 32-level nesting depth
• Audit logging for sensitive data exports with 30+ sensitive field patterns

v0.18.0 - 2025-12-13¶

Added¶

• Per-API-key rate limiting (#137)
• API key management and configuration system
• Files API authentication and authorization (#131)
• Hot reload for runtime configuration updates (#130)

Fixed¶

• Add ConnectInfo extension for admin/metrics/files endpoints
• Address security vulnerabilities in API key management

Refactored¶

• Extract CLI and app utilities into modular structure (#132)
• Split converter.rs into modular structure (#132)
• Split large source files into modular components

v0.17.0 - 2025-12-12¶

Added¶

• Anthropic backend file content transformation (#126)
• Gemini backend file content transformation (#127)

Fixed¶

• Streaming file uploads to prevent memory exhaustion (#128)

v0.16.0 - 2025-12-12¶

Added¶

• OpenAI-compatible Files API endpoints (#111)
• File resolution middleware for chat completions (#120)
• OpenAI backend file handling strategy (#121, #122)
• Persistent metadata storage for Files API (#125)
• GPT-5.2 model support (#124)
• Circuit breaker pattern for automatic backend failover
• Admin endpoint authentication and audit logging
• Configurable fallback models for unavailable model scenarios with cross-provider support

Fixed¶

• Sanitize fallback error headers and metric labels
• Use index-based lookup for fallback chain traversal
• Reduce lock contention in FallbackService with snapshot pattern

v0.15.0 - 2025-12-05¶

Added¶

• Nano Banana (Gemini Image Generation) API support (#102)
• Split /v1/models endpoint — standard lightweight vs extended metadata response (#101)

Changed¶

• Optimize LRU cache to use read lock for cache lookups (#105)

Fixed¶

• Replace .expect() panics with proper error propagation in HttpClientFactory (#104)

Refactored¶

• Extract streaming handler logic to dedicated StreamService (#106)
• Eliminate retry logic code duplication in proxy.rs (#103)

v0.14.2 - 2025-12-05¶

Added¶

• Log token usage (input/output tokens) on request completion (#92)

v0.14.1 - 2025-12-05¶

Fixed¶

• Optimize Anthropic backend TTFT with connection pooling and HTTP/2 (#90)
• Optimize Gemini backend TTFT with connection pooling and HTTP/2 (#88)
• Apply base name fallback matching to aliases in model metadata lookup (#84)

v0.14.0 - 2025-12-04¶

Added¶

• Router-wide global system prompt injection (#82)

CI¶

• Replace deprecated actions-rs/toolchain with dtolnay/rust-toolchain
• Add RUSTFLAGS for macOS ARM64 ring build
• Switch to rustls-tls for musl cross-compilation support

v0.13.0 - 2025-12-04¶

Added¶

• OpenAI /v1/responses API support with session management (#49)
• True SSE streaming for /v1/responses API
• Background cleanup task for expired sessions
• Override /v1/models response fields via model-metadata.yaml (#75)

Security¶

• SecretString for API key storage across all backends (#76)
• Session access control and input validation for Responses API

Changed¶

• Immediate mode for SseParser for reduced first-response latency

Refactored¶

• String allocation optimizations and error handling standardization

v0.12.0 - 2025-12-04¶

Fixed¶

• Handle exact hash matches in consistent hash binary search (#72)
• Replace panics with Option returns and implement stats aggregation (#71)
• Remove hardcoded auth requirement from /v1/models endpoint

Refactored¶

• Reorganize OpenAI model metadata by family (#74)
• Extract AnthropicStreamTransformer to dedicated module (#73)
• Split backends mod.rs into separate modules (#69)
• Extract embedded tests to separate files (#68)
• Create HttpClientFactory for centralized HTTP client creation (#67)
• Create UrlValidator module with SSRF prevention (#66)
• Extract RequestExecutor to shared common module (#65)
• Extract HeaderBuilder with auth strategies (#64)
• Extract AtomicStatistics to shared common module

v0.11.0 - 2025-12-03¶

Added¶

• Native Anthropic Claude API backend with extended thinking support
• OpenAI to Claude reasoning parameter conversion
• Flat reasoning_effort parameter for Anthropic
• Claude 4, 4.1, 4.5 model metadata

Fixed¶

• Improve health check and model fetching for Anthropic/Gemini backends
• Accept-Encoding fixes for streaming — use identity header and disable compression

v0.10.0 - 2025-12-03¶

Added¶

• Native Google Gemini API backend support
• OpenAI Images API support for image generation
• Authenticated health checks for OpenAI and API-key backends
• Built-in OpenAI model metadata for /v1/models response
• API key authentication for streaming requests
• Configurable image generation timeout
• Response_format validation for image generation API

Fixed¶

• Convert max_tokens to max_completion_tokens for newer OpenAI models
• Correct URL construction for all API endpoints
• Request body size limits to prevent DoS attacks

Security¶

• Remove sensitive data from debug logs

Refactored¶

• Unify request retry logic with RequestType enum

v0.9.0 - 2025-12-02¶

Added¶

• Enhanced rate limiting with token bucket algorithm
• Comprehensive Prometheus metrics and monitoring (#10)

Security¶

• Prevent IP spoofing via X-Forwarded-For manipulation
• Prevent header injection vulnerabilities
• Eliminate race condition in token refill
• Protect API keys with SHA-256 hashing
• Prevent memory exhaustion via unbounded bucket growth
• Comprehensive authentication for metrics endpoint
• Cardinality limits and label sanitization to prevent metric explosion DoS

Fixed¶

• Implement singleton pattern for metrics to prevent memory leaks
• Improve error handling to prevent panic conditions
• Resolve environment variable race condition in config test
• Fix integration test failures in metrics

v0.8.0 - 2025-09-09¶

Added¶

• Model ID alias support for metadata sharing (#27)

Fixed¶

• Return empty list instead of 503 when all backends are unhealthy (#28)

v0.7.1 - 2025-09-08¶

Fixed¶

• Improve config path validation for home directory and executable paths (#26)

v0.7.0 - 2025-09-07¶

Added¶

• Rich metadata support for /v1/models endpoint (#23, #25)
• Enhanced configuration management (#9, #22)
• Advanced load balancing strategies (Weighted, Least-Latency, Consistent-Hash) with enhanced error handling (#21)

Fixed¶

• Use streaming timeout configuration from config.yaml instead of hardcoded 25s limit

v0.6.0 - 2025-09-03¶

Fixed¶

• Use timeout configuration from config.yaml instead of hardcoded values (#19)

Documentation¶

• Comprehensive timeout configuration and model documentation updates

v0.5.0 - 2025-09-02¶

Added¶

• Optional retry configuration with sensible defaults
• Comprehensive integration tests and performance optimizations
• Complete service layer implementation
• Middleware architecture and enhanced backend abstraction

Fixed¶

• Handle streaming requests without model field gracefully
• Resolve floating-point precision and timing issues in tests
• Resolve test failures and deadlocks in object pool and SSE parser
• Resolve initial health check race condition

Refactored¶

• Split oversized modules into layered architecture
• Extract complex types into type aliases for better readability

v0.4.0 - 2025-08-25¶

Added¶

• Model-based routing with health monitoring

Fixed¶

• Improve health check integration and SSE parsing

v0.3.0 - 2025-08-25¶

Added¶

• SSE streaming support for real-time chat completions (#5)
• Model aggregation from multiple endpoints (#4)

v0.2.0 - 2025-08-25¶

Added¶

• Multiple backends support with round-robin load balancing (#1)

v0.1.0 - 2025-08-24¶

Added¶

• Initial release with OpenAI-compatible endpoints and proxy functionality