Error Handling¶

Continuum Router returns OpenAI-compatible JSON errors at the HTTP boundary and uses retries, active health filtering, and model fallback to contain transient backend failures.

Error response format¶

Most router-generated failures use this shape:

{
  "error": {
    "message": "The service is temporarily unavailable. Please try again later.",
    "type": "service_unavailable",
    "code": 503
  }
}

param is included when the upstream provider supplies it. A rate-limited response may also include Retry-After.

Streaming failures that happen before any bytes are sent use an HTTP error response. Once an SSE response has started, the stream reports an error event and terminates; the status code can no longer be changed.

Status codes¶

Provider error bodies are normalized when possible. For unrecognized provider 4xx responses, the router keeps only a bounded message excerpt and does not expose arbitrary internal data.

Retries¶

The top-level retry section controls retries to the selected backend:

retry:
  max_attempts: 3
  initial_delay: 100ms
  max_delay: 10s
  backoff_multiplier: 2.0
  jitter: true
  retryable_status_codes: [429, 502, 503, 504]
  retryable_errors: [ConnectionError, TimeoutError]
  timeout: 30s

These are also the default values. The router applies exponential backoff with optional jitter. A provider 429 is retried only when it is classified as a transient rate limit; quota, credit, and billing exhaustion fail fast. Upstream Retry-After or Google retry-delay hints are honored after validation and are capped at 24 hours.

Retries repeat an attempt against a backend. They are distinct from fallback, which can select another model or provider after an eligible failure.

Circuit breaker¶

Continuum Router includes a configurable three-state circuit-breaker implementation and Admin controls:

circuit_breaker:
  enabled: true
  failure_threshold: 5
  failure_rate_threshold: 0.5
  minimum_requests: 10
  timeout: 60s
  half_open_max_requests: 3
  half_open_success_threshold: 2
  failure_status_codes: [500, 502, 503, 504]
  timeout_as_failure: true
  sliding_window_size: 60s
  sliding_window_type: count_based
  backends:
    openai:
      failure_threshold: 10
      timeout: 120s

sliding_window_type accepts count_based or time_based. Entries below circuit_breaker.backends override only the supported per-backend fields.

The regular LLM proxy path records request outcomes in this state machine and consults it during backend selection: an open backend circuit is excluded from selection, and when every compatible circuit is open the router returns a 503 service-unavailable error and participates in configured fallback. It complements active health checks, retries, and fallback rather than replacing them. See Circuit Breaker for the outcome-recording and admission details.

When Admin API authentication permits access, operators can inspect or control breaker state through:

See Admin API for authentication and complete endpoint details.

Model fallback¶

Fallback chains choose alternative models after configured failure conditions:

fallback:
  enabled: true
  mid_stream_enabled: true
  fallback_chains:
    gpt-5.4:
      - gpt-5.4-mini
      - claude-sonnet-4-6
  fallback_policy:
    trigger_conditions:
      error_codes: [429, 500, 502, 503, 504]
      timeout: true
      connection_error: true
      model_not_found: true
      backend_unhealthy: true
    max_fallback_attempts: 3
    fallback_timeout_multiplier: 1.5
    preserve_parameters: true
  model_settings:
    gpt-5.4:
      fallback_enabled: true
      notify_on_fallback: true

With notify_on_fallback: true (the default), successful fallback responses can contain:

• X-Fallback-Used
• X-Original-Model
• X-Fallback-Model
• X-Fallback-Reason
• X-Fallback-Attempts — total model attempts including the primary; always at least 2 when a fallback served the response

Set notify_on_fallback: false for a model to suppress these headers while still serving the fallback.

Cross-provider fallback translates the supported request parameters; provider-only parameters that cannot be represented may be removed.

mid_stream_enabled: true permits mid-stream recovery where the streaming path supports it. It may buffer output, so disable it when immediate token delivery is more important than recovery after a stream has begun.

Timeouts and request size¶

Configure router and model deadlines in the timeouts section and backend request limits in the documented server/backend sections. Avoid making retry timeouts longer than the caller's total deadline; the caller can otherwise disconnect while retry work continues.

Use the generated configuration as the canonical field reference:

continuum-router --generate-config > config.yaml
continuum-router config validate config.yaml

Observability and diagnosis¶

Use implemented interfaces rather than undocumented debug endpoints:

# Public process/backend health
curl http://localhost:8080/health

# Admin health and circuit state (add configured admin credentials)
curl http://localhost:8080/admin/health
curl http://localhost:8080/admin/circuit/all

# Validate configuration without starting the server
continuum-router config validate /etc/continuum-router/config.yaml

Prometheus metrics are exposed at the configured metrics path when the metrics feature and runtime metrics configuration are enabled. Logs use tracing; select verbosity with logging.level, CONTINUUM_LOG_LEVEL, or RUST_LOG as appropriate.

The router does not provide /admin/errors/* or /admin/debug/error endpoints. Inspect structured logs, metrics, /admin/health, and /admin/circuit/* instead.

Client recommendations¶

1. Set a caller-side deadline longer than the expected router/backend timeout.
2. Retry only idempotent operations unless your application has deduplication.
3. Use exponential backoff with jitter and honor Retry-After.
4. Do not retry authentication, permission, validation, or quota-exhaustion errors without changing the request or credentials.
5. Record a correlation/request identifier from your own client logs and avoid logging API keys or resolved configuration output.