Reasoning Effort Parameter¶

This document describes how Continuum Router handles the reasoning_effort parameter across different LLM backends, including the token budget conversions and supported effort levels.

Overview¶

The reasoning_effort parameter controls the amount of computational effort a model spends on reasoning before generating a response. Different backends implement this feature differently:

• OpenAI: Native reasoning_effort parameter for O-series and GPT-5.x thinking models
• Anthropic: Converted to thinking.budget_tokens for Claude models with extended thinking, or to {"type": "adaptive"} for auto effort
• Gemini: Native reasoning_effort for thinking-capable models via OpenAI-compatible endpoint
• Other backends: Pass-through (no transformation)

Parameter Formats¶

Continuum Router supports two input formats, both normalized internally:

Flat Format (Chat Completions API)¶

{
  "model": "o3-mini",
  "reasoning_effort": "high",
  "messages": [...]
}

Nested Format (Responses API)¶

{
  "model": "o3-mini",
  "reasoning": {
    "effort": "high"
  },
  "input": "..."
}

Both formats are automatically normalized to the flat reasoning_effort format before processing. If both are present, the flat format takes precedence.

GPT-5.6 Native Responses Controls¶

For a backend that exposes the native Responses API, Continuum Router preserves the newer GPT-5.6 request surface verbatim:

• reasoning.mode (standard or pro) and reasoning.context (auto, current_turn, or all_turns)
• text.verbosity, safety_identifier, and include
• prompt_cache_key, prompt_cache_options, and explicit prompt_cache_breakpoint markers on input_text, input_image, and input_file
• Programmatic Tool Calling tools and replay items (program, program_output, and caller-linked function outputs)
• Beta Multi-agent configuration and output items

{
  "model": "gpt-5.6-sol",
  "input": "Continue the investigation.",
  "previous_response_id": "resp_123",
  "reasoning": {
    "mode": "pro",
    "effort": "max",
    "context": "all_turns"
  },
  "text": {
    "verbosity": "high"
  },
  "safety_identifier": "usr_privacy_preserving_hash"
}

Native previous_response_id values are forwarded unchanged, allowing OpenAI to reuse persisted reasoning. Conversion backends still use the router's local session expansion. Requests containing native-only fields fail with unsupported_request_parameter when routing selects a conversion strategy instead of silently dropping fields. Stateful and native-only requests bypass the router's local response cache.

Multi-agent raw HTTP clients must send OpenAI-Beta: responses_multi_agent=v1; the router forwards this header while continuing to filter client credentials and transport headers.

Direct Token Budget Specification¶

For advanced use cases, you can specify the token budget directly instead of using reasoning_effort levels:

Anthropic: Direct thinking Parameter¶

{
  "model": "claude-sonnet-4-5-20250929",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 16000
  },
  "messages": [...]
}

For adaptive thinking (model decides when and how much to think):

{
  "model": "claude-opus-4-6",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Gemini: Direct thinking_budget via extra_body¶

{
  "model": "gemini-2.5-pro",
  "extra_body": {
    "google": {
      "thinking_config": {
        "thinking_budget": 10000,
        "include_thoughts": true
      }
    }
  },
  "messages": [...]
}

Fitting a derived reasoning effort¶

The Anthropic Messages ingress (POST /anthropic/v1/messages) derives a reasoning_effort from the client's thinking and output_config.effort fields when it routes a request to an OpenAI-wire backend. The derived level is fitted to what the selected target is known to accept before the request is sent, on the Chat Completions wire (reasoning_effort) and on the Responses wire (reasoning.effort). The dispatch-side conversion for a native Anthropic thinking request sent to /v1/chat/completions applies the same fit.

The vocabulary is resolved in the following order, and the first source that answers wins:

1. Backend declaration: backends[].model_configs[].metadata.reasoning_efforts. This is the operator's statement about the engine actually serving the model.
2. Model metadata: metadata.reasoning_efforts in model-metadata.yaml or a model-metadata.d/ drop-in.
3. Backend-type default:
   • openai and azure use the built-in OpenAI reasoning-family table. A model the table does not describe keeps the derived value.
   • generic, gemini, vllm, ollama, llamacpp, mlxcel, lmstudio, sglang, and tensorrtllm use low, medium, and high, with both high and xhigh sent as high.
4. Passthrough: no source describes the target, so the derived value is forwarded unchanged.

When the fit changes the value, the router logs it at debug level with vocabulary_source set to backend_model_config, model_metadata, or backend_type_default, together with backend, model, effort_field, and the fitted reasoning_effort. A request that reaches passthrough logs the same fields with vocabulary_source=passthrough.

The fit never rewrites a client-supplied reasoning_effort on /v1/chat/completions. It applies only to values the router derived on the Anthropic ingress and in the dispatch-side conversion.

Backend-Specific Behavior¶

OpenAI Backend¶

OpenAI models support the reasoning_effort parameter natively. The router passes the value directly to the OpenAI API.

Supported Effort Levels¶

Models Supporting reasoning_effort¶

O-series models (support low, medium, high):

• o1, o1-mini, o1-preview
• o3, o3-mini, o3-pro
• o4-mini

GPT-6 models:

• gpt-6-astra (low, medium, high, xhigh, and max). none returns a 400 from OpenAI, so this is the first OpenAI reasoning model with no way to turn reasoning off.
• gpt-6-sol and gpt-6-luna (none through max, default medium). Unlike Astra they accept none.

GPT-5.x thinking models (exact levels vary by generation):

• gpt-5.6, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna (none through max)
• gpt-5.2 through gpt-5.5 reasoning tiers (none, low, medium, high, xhigh)
• gpt-5.1 reasoning tiers (none, low, medium, high)
• gpt-5, gpt-5-mini, and gpt-5-nano (minimal, low, medium, high)
• gpt-5-pro (high only; Responses API only)
• gpt-5.2-pro, gpt-5.4-pro, and gpt-5.5-pro (medium, high, xhigh; Responses API only)

Function tools with reasoning effort¶

OpenAI refuses /v1/chat/completions on some models when the request carries function tools and the effective reasoning effort is anything other than none:

400 Function tools with reasoning_effort are not supported for gpt-5.6-sol in
    /v1/chat/completions. To use function tools, use /v1/responses or set
    reasoning_effort to 'none'.

The restriction is per model, not per family, and it is not the same rule in both affected generations:

The gpt-5.6 row differs because the upstream default effort for that family is medium: omitting the field does not mean "no reasoning" there, so the refusal fires on an omitted effort too.

GPT-6 Astra is stricter again: it rejects reasoning_effort: "none" on every request, with or without tools, so the second remedy OpenAI's own error text names does not exist for it. Every tool-carrying Chat Completions request for that model is bridged, and chat_completions_default_reasoning_effort: "medium" on its entry serves as a non-none placeholder since OpenAI does not publish the effort applied when the field is omitted.

GPT-6 Sol and Luna behave like the gpt-5.6 family: they accept none, OpenAI documents that Chat Completions takes function tools only at none, and their default effort is medium, so an omitted effort is bridged too.

Every cell above except the GPT-6 rows was measured against the live API on 2026-09-03; the GPT-6 rows are read from OpenAI's model documentation and refusal text. The split does not follow a name prefix in either direction: the whole GPT-5.4 generation is affected including its mini and nano tiers, while GPT-5.2 and earlier are not, so the rule is stored as data on each model entry rather than a family check.

The router bridges only the refused requests. When a model carries chat_completions_tools_require_none_reasoning, the request has a non-empty tools array, and the effective effort would be refused, the router dispatches to /v1/responses and translates the reply back into the Chat Completions shape (including streaming). The requested effort rides through as reasoning.effort, an omitted effort stays omitted, and the client receives a standard chat.completion or chat.completion.chunk response with its tool calls.

Every other request goes straight to /v1/chat/completions, including reasoning_effort: "none" with tools, any request without tools, and every unaffected model. Validation still runs first, so an effort outside the model's supported set returns the router's own 400 before reaching the bridge. The router does not rewrite the requested effort to none, which would silently change model behavior.

Two metadata keys drive this behavior on the model entry next to responses_only:

Both ship set on the affected entries (gpt-6-astra, gpt-6-sol, gpt-6-luna, gpt-5.6-sol and its gpt-5.6 alias, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano), in the built-in OpenAI registry and in model-metadata.yaml. Because they are ordinary metadata, a model-metadata.d/ drop-in can clear them the day OpenAI lifts the restriction, without waiting for a router release:

# model-metadata.d/90-lift-tools-reasoning-bridge.yaml
models:
  - id: gpt-5.6-sol
    metadata:
      chat_completions_tools_require_none_reasoning: false

Operators can watch the split on the responses_bridge_total metric, whose reason label separates tools_with_reasoning from an unconditionally bridged responses_only_model.

Models NOT Supporting reasoning_effort¶

The following models do not support reasoning parameters (parameter is stripped):

• GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4
• GPT-5.2-chat-latest, GPT-5.2-instant (non-thinking variants)
• GPT-3.5-turbo
• Embedding models, Image models

Anthropic Backend (Claude)¶

Anthropic Claude models use different mechanisms depending on the model generation:

• Claude Fable 5 and Mythos 5 (Mythos-class; Mythos 5 is the safeguards-lifted, limited-release variant of Fable 5): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort, matching Opus 4.8 (legacy budget_tokens rejected with HTTP 400; temperature/top_p/top_k dropped automatically; max effort supported). An explicit thinking.type == "disabled" is also rejected (HTTP 400), so the router omits the thinking parameter entirely instead of forwarding it.
• Claude Opus 5.5 (claude-opus-5-5): Thinking is always on using adaptive thinking with output_config.effort (low through max; Anthropic's default is medium, one level below Opus 5). Both explicit thinking.type == "disabled" and legacy budget_tokens are rejected (HTTP 400) at every effort, so the router omits disabled-thinking configs and normalizes enabled to adaptive. Forced tool use is removed as on Fable 5.1 (see below).
• Claude Sonnet 5.5 (claude-sonnet-5-5): Uses adaptive thinking with output_config.effort (low through max). xhigh maps to max. Explicit thinking.type == "disabled" and forced tool use (tool_choice any or tool) are rejected, so the router omits disabled-thinking configs and rejects a forced choice before dispatch.
• Claude 4.8+ models (Opus 4.8): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort to control thinking depth. The legacy thinking.type == "enabled" + budget_tokens shape is rejected (HTTP 400). Sampling parameters temperature, top_p, and top_k are dropped automatically. Effort default is high.
• Claude Haiku 5.5 (claude-haiku-5-5, no -latest alias): Uses adaptive thinking ({"type": "adaptive"}, on by default) with output_config.effort (low through max; Anthropic's default is medium). The router converts the legacy thinking.type == "enabled" + budget_tokens shape to adaptive and drops temperature, top_p, and top_k, which Haiku 5.5 rejects with HTTP 400. xhigh maps to max. An explicit thinking.type == "disabled" is accepted and forwarded.
• Claude Haiku 4.5 (claude-haiku-4-5-*): Uses thinking.budget_tokens for extended thinking, like Sonnet 4.x. It has no effort parameter.
• Claude Sonnet 5 (claude-sonnet-5-*, alias claude-sonnet-5-latest): Uses adaptive thinking ({"type": "adaptive"}) with output_config.effort. The legacy thinking.type == "enabled" + budget_tokens shape is rejected (HTTP 400), and temperature, top_p, and top_k are dropped automatically. xhigh downgrades to "high" (max is Opus/Mythos-class only). Effort default is high.
• Claude 4.6+ adaptive models (Opus 4.6, Sonnet 4.6, Opus 4.7): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort to control thinking depth. Claude Opus 4.7 requires this API; sending the legacy thinking.type == "enabled" + budget_tokens shape to Opus 4.7 produces HTTP 400.
• Pre-4.6 models (Opus 4.5, Sonnet 4, etc.): Use thinking.budget_tokens for extended thinking.

Conversion Table: Claude 4.6+ Models (Adaptive Thinking)¶

Claude 4.6+ adaptive models (claude-opus-4-6-*, claude-sonnet-4-6-*, claude-opus-4-7-*, claude-opus-4-8-*, claude-sonnet-5-*, claude-haiku-5-5, claude-fable-5-*, claude-mythos-5-*) use output_config.effort instead of budget_tokens:

Conversion Table: Pre-4.6 Claude Models (Budget Tokens)¶

Pre-4.6 models use thinking.budget_tokens:

Transformation Example: Claude 4.6+ (Adaptive Thinking + Effort)¶

Input (OpenAI format):

{
  "model": "claude-opus-4-6",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6",
  "thinking": {
    "type": "adaptive"
  },
  "output_config": {
    "effort": "high"
  },
  "messages": [...]
}

For xhigh on Opus 4.6 or Opus 4.7, output_config.effort is set to "max":

{
  "model": "claude-opus-4-6",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "max"},
  "messages": [...]
}

Transformation Example: Pre-4.6 Claude (Budget Tokens)¶

Input (OpenAI format):

{
  "model": "claude-sonnet-4-5-20250929",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-sonnet-4-5-20250929",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 32768
  },
  "messages": [...]
}

Adaptive Thinking (Auto)¶

When reasoning_effort is set to "auto", the router emits {"type": "adaptive"} without specifying output_config.effort. This lets Anthropic use its default effort level (currently high) and lets the model dynamically decide when and how much reasoning to apply.

Input (OpenAI format):

{
  "model": "claude-opus-4-6",
  "reasoning_effort": "auto",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Direct output_config Pass-Through¶

When a request already contains explicit thinking and output_config parameters, they are passed through directly without conversion:

{
  "model": "claude-opus-4-6",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "medium"},
  "messages": [...]
}

Models Supporting Extended Thinking¶

Extended thinking is supported by:

• Claude Fable 5 / Mythos 5 (adaptive thinking + effort; max supported; sampling params dropped; explicit disabled-thinking omitted): claude-fable-5-* (alias claude-fable-5-latest), claude-mythos-5-* (alias claude-mythos-5-latest, limited release)
• Claude Sonnet 5.5 (adaptive thinking + effort; max supported; sampling params dropped; explicit disabled-thinking omitted; forced tool choice rejected): claude-sonnet-5-5
• Claude Opus 4.8 (adaptive thinking + effort; sampling params deprecated; effort default high): claude-opus-4-8-*, alias claude-opus-4-8-latest
• Claude Sonnet 5 (adaptive thinking + effort; sampling params dropped; xhigh downgraded to high; effort default high): claude-sonnet-5-*, alias claude-sonnet-5-latest
• Claude Opus 4.7 (adaptive thinking + effort; sampling params deprecated): claude-opus-4-7-*
• Claude Haiku 5.5 (adaptive thinking + effort; max supported; sampling params dropped; effort default medium): claude-haiku-5-5
• Claude 4.6 family (adaptive thinking + effort): claude-opus-4-6-*, claude-sonnet-4-6-*
• Claude Opus models (budget tokens): claude-opus-4-*, claude-opus-4-5-*
• Claude Sonnet 4 models (budget tokens): claude-sonnet-4-*, claude-sonnet-4-5-*
• Claude Haiku 4.5 (budget tokens): claude-haiku-4-5-*

Reverse Direction on Fallback Hops¶

Everything above describes the forward direction: an OpenAI reasoning_effort becoming a Claude thinking config. A cross-provider fallback hop can require the reverse. When a request carrying thinking or output_config falls back onto an OpenAI-wire backend, dispatch converts that config into a reasoning_effort value the selected target accepts before stripping the Anthropic-native fields, preserving the caller's reasoning intent across the hop.

During this reverse conversion, output_config.effort maps level for level (with max becoming xhigh where the target's vocabulary includes it), budget_tokens maps through the shared band table, adaptive thinking without an effort and disabled thinking emit nothing, and a client-supplied reasoning_effort always takes precedence. See Model fallback for the full band table, per-target vocabulary rules, and local-engine behavior.

The Anthropic Messages API ingress (/anthropic/v1/messages) applies the same bands through the same function on both its Chat Completions transform and the Responses API transform used for responses_only models. A budget_tokens value therefore resolves to a single reasoning level across the router, whether the request arrived via /anthropic/v1/messages or fell back onto an OpenAI-wire backend.

Gemini Backend¶

Gemini models support reasoning_effort natively through their OpenAI-compatible endpoint. The router validates and passes the value directly.

Supported Effort Levels¶

Which models accept none¶

none is carried through to Gemini unless the model's metadata says the provider refuses it. The refusal is a deny list, rejects_reasoning_effort_none in model-metadata.yaml, and it ships set on the models measured to refuse:

• gemini-3.6-flash
• gemini-3.5-flash-lite
• every Gemini Pro tier

Everything else, including a model released after the router build, is forwarded and answered by Gemini so that newly released Gemini minor versions retain none support without waiting for a router release.

Support does not follow a family prefix. Measured against the OpenAI-compatible endpoint on 2026-09-10, one reasoning_effort: "none" request per catalog id: 2.5 Flash, 2.5 Flash-Lite, 3 Flash, 3.1 Flash-Lite, 3.5 Flash, 3.7 Flash, and 3.8 Flash answered 200; gemini-3.5-flash-lite and gemini-3.6-flash answered 400 INVALID_ARGUMENT; the Pro tiers answered 400 Budget 0 is invalid. This model only works in thinking mode.; the 2.0 family answered 404, having been retired upstream.

The router's own 400 exists for the two Flash tiers, whose upstream error names neither the field nor the model. A Pro model's upstream message is already specific, so the flag there mainly saves the round trip. The check runs on the /v1/chat/completions ingress, before backend selection, so it covers streaming and non-streaming alike. An operator who disagrees with a row can set or clear it in a model-metadata.d/ drop-in without waiting for a release.

A fallback hop is fitted rather than gated. While the ingress check evaluates the client-requested model, a hop lands on a different model chosen by the operator's chain, whose metadata may carry the flag even when the original did not. On such a hop the router drops reasoning_effort: "none" and logs it, serving a reasoned reply from the fallback model rather than failing the request. Because the substitute reasons, a small max_tokens sized for a non-reasoning answer may truncate the output.

The native thinkingConfig.thinkingLevel: "MINIMAL" field is a different question with a different answer: Gemini 3.8 Flash refuses it while Gemini 3.6 Flash accepts it, the reverse of the two rows here. Google documents the minimal level rather than this mapping, so a documentation statement about minimal must not be used to edit the deny list. Measure the id.

Additional Features¶

For Gemini thinking models, the router automatically:

1. Sets include_thoughts: true to expose reasoning content
2. Sets default max_completion_tokens: 16384 if not specified
3. Validates effort level against model capabilities

All three apply on both routes in the same order, so a rejected effort returns the same 400 on streaming and non-streaming requests alike.

Which models get these injections is a table of families (THINKING_FAMILIES in src/infrastructure/backends/gemini/transform.rs), not a name substring. A version, date, or preview suffix reduces away so gemini-3.1-pro-preview and gemini-3.1-pro resolve to the same family, while a -lite segment does not reduce away because Flash and Flash-Lite are different models.

The thought summary itself needs the response side. Google returns it inline in message.content, wrapped in <thought>...</thought> and marked extra_content.google.thought: true, rather than in a field of its own. The router moves that text into message.reasoning_content and leaves content holding only the answer, so the markup never reaches the client and content is always present, empty rather than absent when the model produced nothing but thought.

Malformed function calls¶

Gemini intermittently ends a tool turn with finish_reason: "function_call_filter: MALFORMED_FUNCTION_CALL" (the native API spells it MALFORMED_FUNCTION_CALL): HTTP 200, an assistant message carrying neither content nor tool_calls, and completion_tokens: 0. The model emitted something meant to be a function call and the provider could not parse it. Measured on 2026-09-10, one call in two and one call in five on gemini-3.1-pro-preview with a single tool; every other call returned a proper tool_calls response.

The router refuses to forward that as a success. It becomes a retryable upstream failure (502, type: upstream_error, whose message names malformed_function_call), so the retry loop and any configured fallback.fallback_chains hop engage. A retry succeeded in every observed case, and answering stop instead would hide a turn that produced nothing and still cost prompt tokens. On a streaming request the failure is detected after the response is committed, so it arrives as an SSE error event rather than an HTTP status.

Two shape guarantees hold on every Gemini route as a result. finish_reason is always one of the five OpenAI values (stop, length, tool_calls, content_filter, function_call), because one shared table maps the provider's spellings and reduces anything unrecognized to stop. An assistant message always carries a content key, null only alongside tool_calls and empty otherwise.

Generic/HTTP Backend¶

Generic HTTP backends (used for Ollama, vLLM, LocalAI, LM Studio, etc.) pass requests through without transformation.

llama.cpp Backend¶

The llama.cpp backend uses pass-through behavior:

vLLM Backend¶

vLLM provides an OpenAI-compatible API but reasoning effort support depends on the model:

Summary Table¶

Response Format¶

When reasoning/thinking is enabled, the response includes the model's reasoning process:

OpenAI Format (with reasoning_content)¶

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "Let me think through this step by step..."
    },
    "finish_reason": "stop"
  }]
}

Claude Extended Thinking (transformed to OpenAI format)¶

The router transforms Claude's thinking blocks to the reasoning_content field:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "I need to consider multiple factors..."
    },
    "finish_reason": "stop"
  }]
}

Best Practices¶

1. Use appropriate effort levels: Higher effort improves reasoning depth but increases latency and token cost.
2. Check model support: Not all models support reasoning parameters.
3. Handle upper effort levels carefully: GPT-5.2 through GPT-5.6 support xhigh, while only GPT-5.6 and GPT-6 support max; the router applies documented fallbacks for older models.
4. Consider cost: Extended thinking consumes additional tokens (especially with Anthropic's budget_tokens).
5. Test with your backend: Generic backends have varying support levels.

Related Documentation¶

• API Reference - Chat Completions
• API Reference - Responses API
• Configuration Guide

Stop sequences on reasoning requests¶

A backend may match stop inside hidden reasoning and finish before producing visible content. For requests that will reason, the router removes stop sequences from the final upstream payload and logs the removal at debug level. The removed stop sequence is not enforced by the router. Visible output may continue past it, and normal token limits still apply.

The decision uses the selected backend and model on each attempt, after backend body extensions:

Gemini's includeThoughts: false hides thoughts; it does not disable reasoning. Native provider decisions use the controls actually emitted by the converter. Native Anthropic Messages without enabled/adaptive thinking preserve stops, including when Responses reasoning.effort: none omits thinking. The Gemini Responses converter does not currently forward reasoning.effort; that ignored field is not an effective off switch.

The same protection covers buffered and streaming OpenAI-compatible sends over HTTP or Unix sockets, converted Responses and Anthropic Messages requests, and native Gemini generation configuration. Fallback attempts re-evaluate the rule on a fresh request copy: a reasoning backend's removal does not erase a stop that a later non-reasoning backend can honor. Body extensions cannot reintroduce stops after this check.