Error Handling¶

Continuum Router returns OpenAI-compatible JSON errors at the HTTP boundary and uses retries, active health filtering, and model fallback to contain transient backend failures.

Error response format¶

Most router-generated failures use this shape:

{
  "error": {
    "message": "The service is temporarily unavailable. Please try again later.",
    "type": "service_unavailable",
    "param": null,
    "code": "service_unavailable"
  }
}

All four members are always present, matching the OpenAI Error schema: message and type are strings, and param and code are a string or null rather than being omitted.

code is a stable machine-readable identifier, never the HTTP status. The status stays on the HTTP status line, where it always was. param names the offending request field when the upstream provider supplies one. A rate-limited response may also include Retry-After.

When an upstream provider supplies its own string code, that code passes through unchanged, so a client can still branch on values such as insufficient_quota or context_length_exceeded. The table below lists the codes the router itself authors.

Router-authored error codes¶

Streaming failures that happen before any bytes are sent use an HTTP error response. Once an SSE response has started, the stream reports an error event and terminates; the status code can no longer be changed. On the Chat Completions wire the error event carries the same error object as the HTTP body. On the Responses wire it is the flat ResponseErrorEvent, with code, message, param, and sequence_number beside type.

Status codes¶

Provider error bodies are normalized when possible. For unrecognized provider 4xx responses, the router keeps only a bounded message excerpt and does not expose arbitrary internal data.

Retries¶

The top-level retry section controls retries to the selected backend:

retry:
  max_attempts: 3
  initial_delay: 100ms
  max_delay: 10s
  backoff_multiplier: 2.0
  jitter: true
  retryable_status_codes: [429, 502, 503, 504]
  retryable_errors: [ConnectionError, TimeoutError]
  timeout: 30s

These are also the default values. The router applies exponential backoff with optional jitter. A provider 429 is retried only when it is classified as a transient rate limit; quota, credit, and billing exhaustion fail fast. Upstream Retry-After or Google retry-delay hints are honored after validation and are capped at 24 hours.

Retries repeat an attempt against a backend. They are distinct from fallback, which can select another model or provider after an eligible failure.

Circuit breaker¶

Continuum Router includes a configurable three-state circuit-breaker implementation and Admin controls:

circuit_breaker:
  enabled: true
  failure_threshold: 5
  failure_rate_threshold: 0.5
  minimum_requests: 10
  timeout: 60s
  half_open_max_requests: 3
  half_open_success_threshold: 2
  failure_status_codes: [500, 502, 503, 504]
  timeout_as_failure: true
  sliding_window_size: 60s
  sliding_window_type: count_based
  backends:
    openai:
      failure_threshold: 10
      timeout: 120s

sliding_window_type accepts count_based or time_based. Entries below circuit_breaker.backends override only the supported per-backend fields.

The regular LLM proxy path records request outcomes in this state machine and consults it during backend selection: an open backend circuit is excluded from selection, and when every compatible circuit is open the router returns a 503 service-unavailable error and participates in configured fallback. It complements active health checks, retries, and fallback rather than replacing them. See Circuit Breaker for the outcome-recording and admission details.

When Admin API authentication permits access, operators can inspect or control breaker state through:

See Admin API for authentication and complete endpoint details.

Model fallback¶

Fallback chains choose alternative models after configured failure conditions:

fallback:
  enabled: true
  mid_stream_enabled: true
  fallback_chains:
    gpt-5.4:
      - gpt-5.4-mini
      - claude-sonnet-4-6
  fallback_policy:
    trigger_conditions:
      error_codes: [429, 500, 502, 503, 504]
      timeout: true
      connection_error: true
      model_not_found: true
      backend_unhealthy: true
    max_fallback_attempts: 3
    fallback_timeout_multiplier: 1.5
    preserve_parameters: true
  model_settings:
    gpt-5.4:
      fallback_enabled: true
      notify_on_fallback: true

With notify_on_fallback: true (the default), successful fallback responses can contain:

• X-Fallback-Used
• X-Original-Model
• X-Fallback-Model
• X-Fallback-Reason
• X-Fallback-Attempts — total model attempts including the primary; always at least 2 when a fallback served the response

Set notify_on_fallback: false for a model to suppress these headers while still serving the fallback.

The chain runs on every ingress that resolves a backend from the requested model: the OpenAI-shaped routes through the shared dispatch funnel, and /anthropic/v1/messages and /v1/responses through their own per-attempt selection and native dispatch, on both the non-streaming arm and the streaming arm before the first SSE byte. A streaming attempt whose handshake succeeded is committed and never hops. When the chain is exhausted or the failure is not a trigger, the error is the last attempt's own, in that ingress's dialect: an Anthropic {"type":"error",...} body on /anthropic/v1/messages, an OpenAI {"error":{...}} body on /v1/responses. /v1/responses/compact and count_tokens do not run a chain. See Model fallback for the per-ingress rules.

Cross-provider fallback keeps the request payload canonical and only swaps the model name; wire-format conversion for the newly selected backend happens at dispatch, not at the hop.

fallback_timeout_multiplier (1.0–5.0, default 1.5) scales each hop's own request timeout: the primary attempt keeps the timeout the configuration resolved for it, and hop n runs with base * multiplier^(n-1), clamped to the timeouts.limits ceiling for that timeout class. The base is whatever that attempt would have used without a chain, so a longer per-model budget stays a longer budget on the hop. It does not scale timeouts.connection, the wait for a fallback dial permit, a within-hop retry, or a mid-stream hop after the client already holds the stream. It is a different knob from timeouts.streaming_fallback_budget_multiplier, which sizes the single wall-clock budget shared by every attempt of a streaming request; the two compose, because a streaming hop is scaled first and then capped at what is left of that budget.

mid_stream_enabled: true permits mid-stream recovery where the streaming path supports it. It may buffer output, so disable it when immediate token delivery is more important than recovery after a stream has begun.

Timeouts and request size¶

Configure router and model deadlines in the timeouts section and backend request limits in the documented server/backend sections. Avoid making retry timeouts longer than the caller's total deadline; the caller can otherwise disconnect while retry work continues.

Use the generated configuration as the canonical field reference:

continuum-router --generate-config > config.yaml
continuum-router config validate config.yaml

Observability and diagnosis¶

Use implemented interfaces rather than undocumented debug endpoints:

# Public process/backend health
curl http://localhost:8080/health

# Admin health and circuit state (add configured admin credentials)
curl http://localhost:8080/admin/health
curl http://localhost:8080/admin/circuit/all

# Validate configuration without starting the server
continuum-router config validate /etc/continuum-router/config.yaml

Prometheus metrics are exposed at the configured metrics path when the metrics feature and runtime metrics configuration are enabled. Logs use tracing; select verbosity with logging.level, CONTINUUM_LOG_LEVEL, or RUST_LOG as appropriate.

The router does not provide /admin/errors/* or /admin/debug/error endpoints. Inspect structured logs, metrics, /admin/health, and /admin/circuit/* instead.

Client recommendations¶

1. Set a caller-side deadline longer than the expected router/backend timeout.
2. Retry only idempotent operations unless your application has deduplication.
3. Use exponential backoff with jitter and honor Retry-After.
4. Do not retry authentication, permission, validation, or quota-exhaustion errors without changing the request or credentials.
5. Record a correlation/request identifier from your own client logs and avoid logging API keys or resolved configuration output.