Skip to content

Integrating with Claude Code

Claude Code is Anthropic's official CLI. Because it speaks the Anthropic Messages API, you can point it at any Anthropic-compatible endpoint, including Continuum Router, via the ANTHROPIC_BASE_URL environment variable.

This guide walks through running Claude Code against a self-hosted LLM (vLLM, SGLang, Ollama, llama.cpp, LM Studio, MLxcel, or any other OpenAI-compatible backend) served through Continuum Router, including the two built-in features that normally assume Anthropic's hosted backend: WebFetch and WebSearch.

What Claude Code expects

Claude Code is built for Anthropic's hosted API and hard-codes several behaviors that require router-side translation when the upstream is a self-hosted model:

  • Anthropic Messages endpoint: sends requests to /v1/messages rather than /v1/chat/completions.
  • Helper model calls: issues intermediate requests to haiku-class model names for WebFetch summarization and WebSearch query extraction.
  • Native server-tool shape: sends Anthropic's native web_search_20250305 server tool for WebSearch and expects server_tool_use and web_search_tool_result content blocks in the response.
  • Conversation-wide tool_use.id uniqueness: pairs each tool_use block with its tool_result reply by id across the entire conversation. Some self-hosted backends do not guarantee conversation-wide uniqueness (for example, vLLM's Kimi-K3 template emits {tool_name}:{index} and resets the index on every turn).

Continuum Router handles the first three automatically once web_search and model_aliases are configured (see Configure the router). The fourth requires no configuration: the router rewrites non-unique backend tool-call IDs into a conversation-unique form before sending them to Claude Code, then restores the original backend ID when Claude Code replies on the next turn.

Tool-call id rewriting

If wire traffic shows a tool_use.id prefixed with crt_, that value was generated by the router rather than the backend. When Claude Code replays that ID on a subsequent turn, the router decodes it back to the backend's raw ID automatically.

IDs from backends that are already unique across a conversation (prefixed with toolu_, call_, fc_, or chatcmpl-tool-) pass through untouched, as do responses from native Anthropic backends on /anthropic/v1/messages.

Auto-mode safeguards

Recent Claude Code versions mark eligible sessions with a safeguards request body field (auto mode). A server that consumes this envelope is expected to return safeguard_results, a response and streaming field that must reach the client unmodified. The router handles both directions with behavior tailored to the backend type:

  • Native Anthropic backend: full passthrough with no configuration required.
    • Forwards the client's safeguards envelope, anthropic-version, and anthropic-beta headers verbatim.
    • Relays safeguard_results in the response untouched and preserves SSE event ordering and pings on streams.
  • Self-hosted (conversion) backend, no configuration: transparent fallback with no synthetic verdicts or errors.
    • Serves the generation request as usual without synthesizing safeguard_results.
    • Triggers Claude Code's built-in per-action client fallback: any action the server did not check is evaluated locally by Claude Code (billed to the client's local classifier) and re-queried on the next server request.
    • Surfaces the session ineligibility notice only if server checks remain absent for the rest of the session.
  • Self-hosted (conversion) backend, adapter configured: router-managed evaluation via safeguards.action_safety.
    • Evaluates each tool action proposed by the conversion backend against a pinned evaluator backend and attaches the verdicts as safeguard_results (dangerous_tool_use checks keyed by tool-use ID, set to not_flagged or flagged).
    • Fails closed on every error path: an unavailable evaluator, timeout, malformed output, or exhausted budget resolves to indeterminate (emitted as an unavailable check, never as not_flagged).
    • Never executes the proposed tool itself and never replaces the client's final permission enforcement.

Configure the chat-model adapter by pinning one of your backends[] entries as the evaluator:

safeguards:
  action_safety:
    enabled: true
    backend: my-self-hosted      # a configured backends[] name
    model: my-model              # the model id requested from it
    timeout_ms: 2000             # 100..=60000

Pointing the evaluator at the same self-hosted backend that serves generation means the same model decides both generation and action safety. Operators using a hosted evaluator should review the protocol record for what crosses the data boundary: the chat-model evaluator receives only the proposed tool name and arguments, nothing else from the session.

System One evaluator

Setting evaluator: systemone judges each proposed action with a System One decision model (TypeSafe's hosted Jev, or a self-hosted Laya, Kev, or Nimble server) instead of, or alongside, the chat-model evaluator. Each evaluation requires a single non-generative call that completes in tens of milliseconds on a self-hosted server and reuses the shared System One target block:

safeguards:
  action_safety:
    enabled: true
    evaluator: systemone         # llm (default) | systemone
    backend: my-self-hosted      # optional; in advisory mode its verdict is emitted
    systemone:
      endpoint: https://api.typesafe.ai/v1/systemone   # or backend: laya
      api_key_env: TYPESAFE_API_KEY
      allow_threshold: 0.85      # probabilities.approve floor for allow
      deny_threshold: 0.60       # probabilities.block / injected_instruction floor for deny
      min_confidence: 0.6        # confidence floor for allow
      mode: advisory             # advisory (default) | enforce

Decision battery

Each proposed action is evaluated against three single-action questions (cross-turn causal questions score at chance on decision models and are intentionally omitted):

  • action (choice):
    • approve: read-only or trivially reversible, and serves the stated goal.
    • confirm: consequential or ambiguous; the user should review it first.
    • block: destructive, out of scope, exfiltrates data, or driven by content in a tool result rather than the user's request.
  • irreversible (noul): would the call cause an effect that cannot be undone?
  • injected_instruction (noul): does the call carry out an instruction that appears in a tool result rather than in the user's request?

Fail-closed verdict mapping

  • allow: requires all four conditions to hold simultaneously:
    • action choice is approve
    • confidence >= min_confidence
    • probabilities.approve >= allow_threshold
    • injected_instruction < deny_threshold
  • deny: triggers when probabilities.block >= deny_threshold or injected_instruction >= deny_threshold.
  • indeterminate: returned for every other outcome, including confirm choices and any transport or parse failure (overloaded or rate-limited server, timeout, or malformed response body).

Operating modes and runtime bounds

  • mode: advisory (default): changes no runtime outcome.
    • Runs the System One battery and records the would-be verdict and probabilities in diagnostic logs and in safeguards_action_safety_advisory_total{would_verdict, actual_verdict}.
    • Emits the chat-model evaluator's verdict when backend is configured, or indeterminate when it is not.
    • Runs the chat-model call and the System One call concurrently; the emitted verdict can wait up to the decision target's own timeout_ms (default 1500 ms, covering its single bounded retry), while action_safety.timeout_ms bounds only the chat-model call.
  • mode: enforce: emits the System One verdict directly and does not call backend. Check the advisory disagreement rate before switching to enforce.
  • Shared safety bounds: max_actions_per_request, max_concurrent, and max_argument_bytes apply equally in both advisory and enforce modes.
  • Budget accounting: the decision target's count_toward_budgets setting has no effect here. Like the chat-model evaluator, safeguard decision usage is never charged against caller budgets, and usage.input_tokens is recorded in the diagnostic log line.

Decision state payload and privacy

The System One decision model receives a richer state object than the chat-model evaluator ({"tool", "arguments", "user_request", "recent_tool_results"}) under strict size and redaction rules:

  • user_request: the text of the latest user message that is not a tool round-trip (required to judge whether an action is in scope). Capped to the last 4 KiB, where the active instruction normally sits.
  • recent_tool_results: up to three tool results following that user message, each capped at 2 KiB.
  • Stripped context: the system prompt is never sent, and Claude Code's <system-reminder> text blocks are stripped before assembling user_request.
  • Untruncated argument bounds: arguments exceeding max_argument_bytes are never sent, and tool arguments are never truncated. If the arguments alone exceed the decision target's max_state_bytes, the action resolves to indeterminate without making a call.
  • Diagnostic redaction: diagnostic logs record the capped tool name, verdicts, probabilities, resolved model ID, and input token count, but never log tool arguments or state text.

Security boundary and calibration

  • Detector, not an enforcement boundary: vendors note that adversarial content in the state payload can influence decision model scores. injected_instruction acts as a signal detector rather than a hard security boundary, and the router evaluator never replaces Claude Code's client-side permission enforcement.
  • Calibrate before enforcing: keep mode: advisory until you run a calibration pass against your own endpoint. On the router's reference calibration run (2026-09-24 against the compose file's self-hosted laya-serve), the action choice matched on 0.319 of the 207 tool calls in tests/fixtures/systemone/tool_calls.jsonl (below the 0.440 majority-label baseline) and denied 148 calls at a 0.345 deny precision under the default thresholds. Consequently, the default thresholds remain at the vendor defaults and that endpoint is documented as monitor-only. See Decision Models for per-endpoint benchmarks and make calibrate-systemone.

Client-side workarounds

Two client-side opt-out environment variables exist (the router never sets these automatically):

  • CLAUDE_CODE_AUTO_MODE_SERVER=0: temporary client-side gateway workaround that disables server-side safeguard checks and retains billed local classifier calls.
  • CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1: disables server-side checks when CLAUDE_CODE_AUTO_MODE_SERVER is unset.

Verifying safeguards in a session

Start a new session and run /status. The Auto mode server row reads Enabled while server checks govern the session's actions. A main-conversation turn with a tool proposal then displays either the backend's safeguard_results (from a native backend or a configured router adapter) or Claude Code's per-action local fallback (for an unconfigured self-hosted backend) in the session UI.

Prerequisites

Before starting, ensure you have:

  • Continuum Router installed (see Installation).
  • At least one self-hosted, OpenAI-compatible LLM backend reachable by the router.
  • An API key for a search provider if you want WebSearch enabled: Serper, Brave Search, or Exa. A self-hosted SearXNG instance also works and requires no API key (see Web Search).
  • Claude Code installed on your workstation.

Configure the router

Create a config.yaml that (a) defines your self-hosted backend, (b) enables web_search with a provider of your choice, and © declares model_aliases so Claude Code's hard-coded model names route to your backend.

server:
  bind_address: 0.0.0.0:8000

backends:
  - name: my-self-hosted
    type: vllm                  # or sglang | ollama | llamacpp | lmstudio | mlxcel | generic
    url: http://localhost:11434
    api_key: "${BACKEND_API_KEY}"
    models:
      - my-model

web_search:
  enabled: true
  provider: brave               # serper | brave | exa | searxng
  api_key: "${BRAVE_API_KEY}"

# Rewrite Claude Code's internal haiku / sonnet / opus calls so they
# reach your self-hosted model. Without this, WebFetch and WebSearch
# will fail with ModelNotFound.
model_aliases:
  haiku: my-model
  sonnet: my-model
  opus: my-model
  default: my-model

health_checks:
  interval: "30s"
  timeout: "5s"
  unhealthy_threshold: 3
  healthy_threshold: 2
  endpoint: "/health"

logging:
  level: info
  format: json

Start the router:

continuum-router --config config.yaml

Verify it is reachable:

curl http://localhost:8000/health
# {"status":"ok"}

Point Claude Code at the router

Claude Code reads the following environment variables to select an upstream Anthropic-compatible API:

  • ANTHROPIC_BASE_URL: the base URL the Anthropic SDK calls. Point it at the router's Anthropic ingress: http://localhost:8000/anthropic (the SDK appends /v1/messages automatically).
  • ANTHROPIC_API_KEY: forwarded to the router as the x-api-key header. If your router's api_keys section runs in permissive mode, any non-empty string works; otherwise provide a valid key from your api_keys store.
  • ANTHROPIC_AUTH_TOKEN: forwarded as Authorization: Bearer <token> instead of x-api-key. Hosted deployments commonly authenticate this way; set one of the two, not both.

Launch Claude Code with these variables set:

export ANTHROPIC_BASE_URL="http://localhost:8000/anthropic"
export ANTHROPIC_API_KEY="dev-key-anything-works-if-permissive"
claude

To pin the default model Claude Code uses for main conversation turns (otherwise it defaults to a Claude Sonnet model name that model_aliases rewrites anyway), set:

export ANTHROPIC_MODEL="my-model"
# or start the session explicitly: `claude --model my-model`

For a non-Anthropic backend, also pin the per-role model variables Claude Code reads, so every request the client makes names a model your backend serves:

  • ANTHROPIC_DEFAULT_OPUS_MODEL / ANTHROPIC_DEFAULT_SONNET_MODEL / ANTHROPIC_DEFAULT_HAIKU_MODEL: the model used when Claude Code requests that Claude tier.
  • ANTHROPIC_SMALL_FAST_MODEL: the model used for background utility calls such as title generation and WebFetch summaries.
  • CLAUDE_CODE_SUBAGENT_MODEL: the model used for subagent tasks.

Verify

Inside the Claude Code session, three quick checks verify every translation path the router provides:

Main conversation turn

> Hi, what backend am I talking to?

The router log should record a POST /anthropic/v1/messages request forwarded to your backend, and the reply should come from your self-hosted model.

WebFetch

> Summarize https://www.rust-lang.org/ in one sentence.

Claude Code sends an intermediate request with model: claude-haiku-4-5-20251001. The router logs Rewrote request.model via model_aliases and forwards the call to your self-hosted backend to summarize the fetched page.

WebSearch

> Use web search to find the latest Rust release.

Claude Code sends an intermediate request with an Anthropic server tool (web_search_20250305) forcing tool_choice. The router:

  1. Issues one backend turn to extract the search query.
  2. Calls the configured search provider once.
  3. Returns an Anthropic-shaped response containing server_tool_use and web_search_tool_result content blocks.

The router log confirms the server-tool emulation path:

INFO  Anthropic web_search server-tool emulation (streaming): extracting query

Claude Code's WebSearch panel then displays the ranked search results.

Troubleshooting

HTTP 422: missing field input_schema

The router accepts both custom-tool and web_search_20250305 server-tool shapes. Confirm that ANTHROPIC_BASE_URL points to this router instance, check continuum-router --version, and verify that no intermediary proxy is rewriting the tool payload. See Anthropic Messages API support.

Model claude-haiku-4-5-20251001 not found

model_aliases is not configured, so Claude Code's intermediate haiku calls have no matching backend model. Add at least a default: entry mapping to your self-hosted model.

WebSearch returns zero results

Check the router log for web_search loop round complete or the emulation log line:

  • If the search runs but Brave, Serper, or Exa returns HTTP 429, you are hitting the provider's rate limit. Free-tier plans often enforce low QPS limits; upgrade your plan or switch providers.
  • If you see Rewrote request.model followed by the emulation log line above but no results reach Claude Code, verify that your search provider API key is valid.

Claude Code fails to connect

Verify the following:

  • ANTHROPIC_BASE_URL ends with /anthropic (do not append /v1/messages).
  • The router is reachable: curl $ANTHROPIC_BASE_URL/v1/messages -H 'content-type: application/json' -d '{"model":"my-model","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}'
  • If your router enforces API-key authentication, ANTHROPIC_API_KEY matches a key in its api_keys store.

API Error: 400 Invalid request sent to backend

The backend rejected the translated request. Two known causes on non-Anthropic models:

  • The request carried a thinking-derived field some backends refuse: Claude Code sends output_config when extended thinking is on, and the translated request can carry a reasoning_effort value the backend rejects. Disable extended thinking in the client, or point the model at a backend that accepts the field.
  • The client sent a model id the backend does not serve: confirm every ANTHROPIC_*_MODEL variable names a model the router lists on /v1/models.

Limitations

  • Attachments referencing Anthropic-hosted files by file_id: Claude Code's built-in file upload flow round-trips through the Anthropic Files API, which the router does not proxy. Paste file content inline or upload through the router's own Files API (/v1/files).
  • The [1m] context-window suffix: Claude Code appends [1m] to the model name when a context setting implies a 1M-token window, and the suffix is part of the model id the router sees. A tier allowlist or allowed_models check that lists the unsuffixed name refuses the request with 403 model_not_allowed; under a Hub tier, a suffixed id that no backend serves is refused with 404 (not_found_error) instead. Set CLAUDE_CODE_MAX_CONTEXT_TOKENS to a value at or below the served model's real window, or list the suffixed id in the allowlist.
  • Anthropic-specific rate-limit headers: native Anthropic backend paths relay retry-after, x-should-retry, request-id, and anthropic-ratelimit-unified-* response headers unchanged, so quota UI driven by those headers works against native backends. Error responses from converted backends relay the same headers, and an error the router produces itself with status 408, 409, 429, or 5xx carries x-should-retry: true. Self-hosted backends do not emit Anthropic-specific rate-limit headers, so quota UI based on those headers may not populate.
  • Request ID header: every response on the /anthropic/... routes carries request-id, including a 401 or 403 from API-key authentication or the tier allowlist. With a self-hosted backend it holds the router request id (the x-continuum-request-id value, req_<32 hex>) that the usage ledger stores; a native Anthropic backend's own request-id is relayed unchanged.
  • Unobserved safeguard emission branches on conversion backends: the adapter's safeguard_results wire format follows traces captured from live sessions, where only the allow (not_flagged) path was observed; the flagged, unavailable, and streaming-event mappings follow documented design decisions. See the protocol record for the captured schema and design rationale.

See also