Reasoning Effort Parameter¶

This document describes how Continuum Router handles the reasoning_effort parameter across different LLM backends, including the token budget conversions and supported effort levels.

Overview¶

The reasoning_effort parameter controls the amount of computational effort a model spends on reasoning before generating a response. Different backends implement this feature differently:

• OpenAI: Native reasoning_effort parameter for O-series and GPT-5.x thinking models
• Anthropic: Converted to thinking.budget_tokens for Claude models with extended thinking, or to {"type": "adaptive"} for auto effort
• Gemini: Native reasoning_effort for thinking-capable models via OpenAI-compatible endpoint
• Other backends: Pass-through (no transformation)

Parameter Formats¶

Continuum Router supports two input formats, both normalized internally:

Flat Format (Chat Completions API)¶

{
  "model": "o3-mini",
  "reasoning_effort": "high",
  "messages": [...]
}

Nested Format (Responses API)¶

{
  "model": "o3-mini",
  "reasoning": {
    "effort": "high"
  },
  "input": "..."
}

Both formats are automatically normalized to the flat reasoning_effort format before processing. If both are present, the flat format takes precedence.

GPT-5.6 Native Responses Controls¶

For a backend that exposes the native Responses API, Continuum Router preserves the newer GPT-5.6 request surface verbatim:

• reasoning.mode (standard or pro) and reasoning.context (auto, current_turn, or all_turns)
• text.verbosity, safety_identifier, and include
• prompt_cache_key, prompt_cache_options, and explicit prompt_cache_breakpoint markers on input_text, input_image, and input_file
• Programmatic Tool Calling tools and replay items (program, program_output, and caller-linked function outputs)
• Beta Multi-agent configuration and output items

{
  "model": "gpt-5.6-sol",
  "input": "Continue the investigation.",
  "previous_response_id": "resp_123",
  "reasoning": {
    "mode": "pro",
    "effort": "max",
    "context": "all_turns"
  },
  "text": {
    "verbosity": "high"
  },
  "safety_identifier": "usr_privacy_preserving_hash"
}

Native previous_response_id values are forwarded unchanged, allowing OpenAI to reuse persisted reasoning. Conversion backends still use the router's local session expansion. Requests containing native-only fields fail with unsupported_request_parameter when routing selects a conversion strategy instead of silently dropping fields. Stateful and native-only requests bypass the router's local response cache.

Multi-agent raw HTTP clients must send OpenAI-Beta: responses_multi_agent=v1; the router forwards this header while continuing to filter client credentials and transport headers.

Direct Token Budget Specification¶

For advanced use cases, you can specify the token budget directly instead of using reasoning_effort levels:

Anthropic: Direct thinking Parameter¶

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 16000
  },
  "messages": [...]
}

For adaptive thinking (model decides when and how much to think):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Gemini: Direct thinking_budget via extra_body¶

{
  "model": "gemini-2.5-pro",
  "extra_body": {
    "google": {
      "thinking_config": {
        "thinking_budget": 10000,
        "include_thoughts": true
      }
    }
  },
  "messages": [...]
}

Backend-Specific Behavior¶

OpenAI Backend¶

OpenAI models support the reasoning_effort parameter natively. The router passes the value directly to the OpenAI API.

Supported Effort Levels¶

Models Supporting reasoning_effort¶

O-series models (support low, medium, high):

• o1, o1-mini, o1-preview
• o3, o3-mini, o3-pro
• o4-mini

GPT-6 models:

• gpt-6-astra (low, medium, high, xhigh, and max). none returns a 400 from OpenAI, so this is the first OpenAI reasoning model with no way to turn reasoning off.

GPT-5.x thinking models (exact levels vary by generation):

• gpt-5.6, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna (none through max)
• gpt-5.2 through gpt-5.5 reasoning tiers (none, low, medium, high, xhigh)
• gpt-5.1 reasoning tiers (none, low, medium, high)
• gpt-5, gpt-5-mini, and gpt-5-nano (minimal, low, medium, high)
• gpt-5-pro (high only; Responses API only)
• gpt-5.2-pro, gpt-5.4-pro, and gpt-5.5-pro (medium, high, xhigh; Responses API only)

Function tools with reasoning effort¶

OpenAI refuses /v1/chat/completions on some models when the request carries function tools and the effective reasoning effort is anything other than none:

400 Function tools with reasoning_effort are not supported for gpt-5.6-sol in
    /v1/chat/completions. To use function tools, use /v1/responses or set
    reasoning_effort to 'none'.

The restriction is per model, not per family, and it is not the same rule in both affected generations:

The gpt-5.6 row differs because the upstream default effort for that family is medium: omitting the field does not mean "no reasoning" there, so the refusal fires on an omitted effort too.

GPT-6 Astra is stricter again: it rejects reasoning_effort: "none" on every request, with or without tools, so the second remedy OpenAI's own error text names does not exist for it. Every tool-carrying Chat Completions request for that model is bridged, and the chat_completions_default_reasoning_effort: "medium" on its entry is there only to be a value other than none, since OpenAI does not publish the effort it applies when the field is omitted.

Every cell above except the gpt-6-astra row was measured against the live API on 2026-09-03; that row is read from OpenAI's model documentation and its refusal text, and has not been measured here. The split does not follow a name prefix in either direction: the whole GPT-5.4 generation is affected including its mini and nano tiers, while GPT-5.2 and earlier are not, which is why the rule is data on each model entry rather than a family check.

The router bridges exactly the refused requests. When the resolved model carries chat_completions_tools_require_none_reasoning in its metadata, the request has a non-empty tools array, and the effective effort would be refused, the request is dispatched to /v1/responses (which accepts the combination) and the reply is translated back into the Chat Completions shape, streaming included. Everything else is preserved: the requested effort rides through as reasoning.effort, an omitted effort stays omitted, and the client sees an ordinary chat.completion or chat.completion.chunk stream with its tool calls.

Every other request keeps going straight to /v1/chat/completions, so nothing that works today changes cost or latency: reasoning_effort: "none" with tools, any request without tools, and every unaffected model. Validation still runs first, so an effort outside the model's supported set is the router's own 400 and never reaches the bridge. The router does not rewrite the requested effort to none, which would silently change model behavior.

Two metadata keys drive it, on the model entry next to responses_only:

Both ship set on the affected entries (gpt-6-astra, gpt-5.6-sol and its gpt-5.6 alias, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano), in the built-in OpenAI registry and in model-metadata.yaml. Because they are ordinary metadata, a model-metadata.d/ drop-in can clear them the day OpenAI lifts the restriction, without waiting for a router release:

# model-metadata.d/90-lift-tools-reasoning-bridge.yaml
models:
  - id: gpt-5.6-sol
    metadata:
      chat_completions_tools_require_none_reasoning: false

Operators can watch the split on the responses_bridge_total metric, whose reason label separates tools_with_reasoning from an unconditionally bridged responses_only_model.

Models NOT Supporting reasoning_effort¶

The following models do not support reasoning parameters (parameter is stripped):

• GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4
• GPT-5.2-chat-latest, GPT-5.2-instant (non-thinking variants)
• GPT-3.5-turbo
• Embedding models, Image models

Anthropic Backend (Claude)¶

Anthropic Claude models use different mechanisms depending on the model generation:

• Claude Fable 5 and Mythos 5 (Mythos-class; Mythos 5 is the safeguards-lifted, limited-release variant of Fable 5): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort, the same surface as Opus 4.8 (legacy budget_tokens rejected with HTTP 400; temperature/top_p/top_k dropped automatically; max effort supported). One additional constraint: an explicit thinking.type == "disabled" is rejected (HTTP 400), so the router omits the thinking parameter entirely instead of forwarding it.
• Claude 4.8+ models (Opus 4.8): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort to control thinking depth. The legacy thinking.type == "enabled" + budget_tokens shape is rejected (HTTP 400). Sampling parameters temperature, top_p, and top_k are not accepted and the router drops them automatically. Effort default is high.
• Claude Sonnet 5 (claude-sonnet-5-*, alias claude-sonnet-5-latest): Uses adaptive thinking ({"type": "adaptive"}) with output_config.effort. The legacy thinking.type == "enabled" + budget_tokens shape is rejected (HTTP 400). Sampling parameters temperature, top_p, and top_k are dropped automatically. xhigh downgrades to "high" (max is not supported; Sonnet 5 is not Opus/Mythos-class). Effort default is high.
• Claude 4.6+ adaptive models (Opus 4.6, Sonnet 4.6, Opus 4.7): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort to control thinking depth. Claude Opus 4.7 requires this API; sending the legacy thinking.type == "enabled" + budget_tokens shape to Opus 4.7 produces HTTP 400.
• Pre-4.6 models (Opus 4.5, Sonnet 4, etc.): Use thinking.budget_tokens for extended thinking

Conversion Table: Claude 4.6+ Models (Adaptive Thinking)¶

Claude 4.6+ adaptive models (claude-opus-4-6-*, claude-sonnet-4-6-*, claude-opus-4-7-*, claude-opus-4-8-*, claude-sonnet-5-*, claude-fable-5-*, claude-mythos-5-*) use output_config.effort instead of budget_tokens:

Conversion Table: Pre-4.6 Claude Models (Budget Tokens)¶

Pre-4.6 models use thinking.budget_tokens:

Transformation Example: Claude 4.6+ (Adaptive Thinking + Effort)¶

Input (OpenAI format):

{
  "model": "claude-opus-4-6-20260205",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "output_config": {
    "effort": "high"
  },
  "messages": [...]
}

For xhigh on Opus 4.6 or Opus 4.7, output_config.effort is set to "max":

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "max"},
  "messages": [...]
}

Transformation Example: Pre-4.6 Claude (Budget Tokens)¶

Input (OpenAI format):

{
  "model": "claude-sonnet-4-20250514",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 32768
  },
  "messages": [...]
}

Adaptive Thinking (Auto)¶

When reasoning_effort is set to "auto", the router emits {"type": "adaptive"} without specifying output_config.effort. This lets Anthropic use its default effort level (currently high) and lets the model dynamically decide when and how much reasoning to apply.

Input (OpenAI format):

{
  "model": "claude-opus-4-6-20260205",
  "reasoning_effort": "auto",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Direct output_config Pass-Through¶

When a request already contains explicit thinking and output_config parameters, they are passed through directly without conversion:

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "medium"},
  "messages": [...]
}

Models Supporting Extended Thinking¶

Extended thinking is supported by:

• Claude Fable 5 / Mythos 5 (adaptive thinking + effort; max supported; sampling params dropped; explicit disabled-thinking omitted): claude-fable-5-* (alias claude-fable-5-latest), claude-mythos-5-* (alias claude-mythos-5-latest, limited release)
• Claude Opus 4.8 (adaptive thinking + effort; sampling params deprecated; effort default high): claude-opus-4-8-*, alias claude-opus-4-8-latest
• Claude Sonnet 5 (adaptive thinking + effort; sampling params dropped; xhigh downgraded to high; effort default high): claude-sonnet-5-*, alias claude-sonnet-5-latest
• Claude Opus 4.7 (adaptive thinking + effort; sampling params deprecated): claude-opus-4-7-*
• Claude 4.6 family (adaptive thinking + effort): claude-opus-4-6-*, claude-sonnet-4-6-*
• Claude Opus models (budget tokens): claude-opus-4-*, claude-opus-4-5-*
• Claude Sonnet 4 models (budget tokens): claude-sonnet-4-*, claude-sonnet-4-5-*

Reverse Direction on Fallback Hops¶

Everything above describes the forward direction: an OpenAI reasoning_effort becoming a Claude thinking config. A cross-provider fallback hop can need the reverse. When a request carrying thinking or output_config falls back onto an OpenAI-wire backend, dispatch converts that config into a reasoning_effort value the selected target accepts and only then removes the Anthropic-native fields, so the reasoning intent crosses the hop instead of being dropped. output_config.effort maps level for level with max becoming xhigh where the target's vocabulary has it, budget_tokens maps back through the same band table, adaptive thinking without an effort and disabled thinking both emit nothing, and a client-supplied reasoning_effort always wins. The full table, the per-target vocabulary rule, and the local-engine decision are in Model fallback.

The Anthropic Messages API ingress (/anthropic/v1/messages) applies the same bands through the same function, on its Chat Completions transform and on the Responses API transform it uses for responses_only models. A budget_tokens value therefore resolves to one reasoning level across the whole router, whether the request arrived as an Anthropic message or was rescued onto an OpenAI-wire backend by a fallback hop. The band table itself is in Model fallback.

Gemini Backend¶

Gemini models support reasoning_effort natively through their OpenAI-compatible endpoint. The router validates and passes the value directly.

Supported Effort Levels¶

Which models accept none¶

none is carried through to Gemini unless the model's metadata says the provider refuses it. The refusal is a deny list, rejects_reasoning_effort_none in model-metadata.yaml, and it ships set on the models measured to refuse:

• gemini-3.6-flash
• gemini-3.5-flash-lite
• every Gemini Pro tier

Everything else, including a model released after the router build, is forwarded and answered by Gemini. That default is deliberate. The predecessor was an allowlist of Flash families compiled into the router, so each new Gemini minor lost none support until someone measured it and cut a release, and Google shipped four Flash minors between May and September 2026.

Support does not follow a family prefix. Measured against the OpenAI-compatible endpoint on 2026-09-10, one reasoning_effort: "none" request per catalog id: 2.5 Flash, 2.5 Flash-Lite, 3 Flash, 3.1 Flash-Lite, 3.5 Flash, 3.7 Flash, and 3.8 Flash answered 200; gemini-3.5-flash-lite and gemini-3.6-flash answered 400 INVALID_ARGUMENT; the Pro tiers answered 400 Budget 0 is invalid. This model only works in thinking mode.; the 2.0 family answered 404, having been retired upstream.

The router's own 400 exists for the two Flash tiers, whose upstream error names neither the field nor the model. A Pro model's upstream message is already specific, so the flag there mainly saves the round trip. The check runs on the /v1/chat/completions ingress, before backend selection, so it covers streaming and non-streaming alike. An operator who disagrees with a row can set or clear it in a model-metadata.d/ drop-in without waiting for a release.

A fallback hop is fitted rather than gated. The ingress check judges the model the client asked for, while a hop lands on a different model chosen by the operator's chain, whose metadata can carry the flag when the original did not. On such a hop the router drops reasoning_effort: "none" and logs it, because a hop has already substituted the model: a client that accepted a different model over a failure is better served by that model reasoning than by an error. Before this, the hop forwarded none, the provider answered 400, the chain counted a failed hop, and the request came back as 500 fallback exhausted naming neither the model nor the field. One consequence is worth knowing: the substitute reasons, so a small max_tokens that fit a non-reasoning answer can now truncate one.

The native thinkingConfig.thinkingLevel: "MINIMAL" field is a different question with a different answer: Gemini 3.8 Flash refuses it while Gemini 3.6 Flash accepts it, the reverse of the two rows here. Google documents the minimal level rather than this mapping, so a documentation statement about minimal must not be used to edit the deny list. Measure the id.

Additional Features¶

For Gemini thinking models, the router automatically:

1. Sets include_thoughts: true to expose reasoning content
2. Sets default max_completion_tokens: 16384 if not specified
3. Validates effort level against model capabilities

All three apply on both routes. Until issue #1591 the transform ran only when stream: true, so a non-streaming client received no reasoning_content from a thinking model, got Gemini's low implicit output cap instead of the 16384 default, and had reasoning_effort: "xhigh" forwarded rather than downgraded, while the same request with stream: true got all three. The non-streaming route now runs the same two steps in the same order the streaming route does, so a rejected effort is the same 400 either way.

Which models get these injections is a table of families (THINKING_FAMILIES in src/infrastructure/backends/gemini/transform.rs), not a name substring. The substring list it replaced matched gemini-3.1-pro-preview but not gemini-3.1-pro, so the same model behaved differently depending on which of its two names the caller sent. A version, date, or preview suffix reduces away and a -lite segment does not, because Flash and Flash-Lite are different models.

The thought summary itself needs the response side. Google returns it inline in message.content, wrapped in <thought>...</thought> and marked extra_content.google.thought: true, rather than in a field of its own. The router moves that text into message.reasoning_content and leaves content holding only the answer, so the markup never reaches the client and content is always present, empty rather than absent when the model produced nothing but thought.

Malformed function calls¶

Gemini intermittently ends a tool turn with finish_reason: "function_call_filter: MALFORMED_FUNCTION_CALL" (the native API spells it MALFORMED_FUNCTION_CALL): HTTP 200, an assistant message carrying neither content nor tool_calls, and completion_tokens: 0. The model emitted something meant to be a function call and the provider could not parse it. Measured on 2026-09-10, one call in two and one call in five on gemini-3.1-pro-preview with a single tool; every other call returned a proper tool_calls response.

The router refuses to forward that as a success. It becomes a retryable upstream failure (502, type: upstream_error, whose message names malformed_function_call), so the retry loop and any configured fallback.fallback_chains hop engage. A retry succeeded in every observed case, and answering stop instead would hide a turn that produced nothing and still cost prompt tokens. On a streaming request the failure is detected after the response is committed, so it arrives as an SSE error event rather than an HTTP status.

Two shape guarantees hold on every Gemini route as a result. finish_reason is always one of the five OpenAI values (stop, length, tool_calls, content_filter, function_call), because one shared table maps the provider's spellings and reduces anything unrecognized to stop. An assistant message always carries a content key, null only alongside tool_calls and empty otherwise.

Generic/HTTP Backend¶

Generic HTTP backends (used for Ollama, vLLM, LocalAI, LM Studio, etc.) pass requests through without transformation.

llama.cpp Backend¶

The llama.cpp backend uses pass-through behavior:

vLLM Backend¶

vLLM provides an OpenAI-compatible API but reasoning effort support depends on the model:

Summary Table¶

Response Format¶

When reasoning/thinking is enabled, the response includes the model's reasoning process:

OpenAI Format (with reasoning_content)¶

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "Let me think through this step by step..."
    },
    "finish_reason": "stop"
  }]
}

Claude Extended Thinking (transformed to OpenAI format)¶

The router transforms Claude's thinking blocks to the reasoning_content field:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "I need to consider multiple factors..."
    },
    "finish_reason": "stop"
  }]
}

Best Practices¶

1. Use appropriate effort levels: Higher effort = better reasoning but slower and more expensive
2. Check model support: Not all models support reasoning parameters
3. Handle upper effort levels carefully: GPT-5.2–5.6 support xhigh, while only GPT-5.6 supports max; the router applies documented fallbacks for older models
4. Consider cost: Extended thinking consumes additional tokens (especially with Anthropic's budget_tokens)
5. Test with your backend: Generic backends may have varying support levels

Related Documentation¶

• API Reference - Chat Completions
• API Reference - Responses API
• Configuration Guide