Reasoning Effort Parameter¶

This document describes how Continuum Router handles the reasoning_effort parameter across different LLM backends, including the token budget conversions and supported effort levels.

Overview¶

The reasoning_effort parameter controls the amount of computational effort a model spends on reasoning before generating a response. Different backends implement this feature differently:

• OpenAI: Native reasoning_effort parameter for O-series and GPT-5.x thinking models
• Anthropic: Converted to thinking.budget_tokens for Claude models with extended thinking, or to {"type": "adaptive"} for auto effort
• Gemini: Native reasoning_effort for thinking-capable models via OpenAI-compatible endpoint
• Other backends: Pass-through (no transformation)

Parameter Formats¶

Continuum Router supports two input formats, both normalized internally:

Flat Format (Chat Completions API)¶

{
  "model": "o3-mini",
  "reasoning_effort": "high",
  "messages": [...]
}

Nested Format (Responses API)¶

{
  "model": "o3-mini",
  "reasoning": {
    "effort": "high"
  },
  "input": "..."
}

Both formats are automatically normalized to the flat reasoning_effort format before processing. If both are present, the flat format takes precedence.

Direct Token Budget Specification¶

For advanced use cases, you can specify the token budget directly instead of using reasoning_effort levels:

Anthropic: Direct thinking Parameter¶

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 16000
  },
  "messages": [...]
}

For adaptive thinking (model decides when and how much to think):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Gemini: Direct thinking_budget via extra_body¶

{
  "model": "gemini-2.5-pro",
  "extra_body": {
    "google": {
      "thinking_config": {
        "thinking_budget": 10000,
        "include_thoughts": true
      }
    }
  },
  "messages": [...]
}

Backend-Specific Behavior¶

OpenAI Backend¶

OpenAI models support the reasoning_effort parameter natively. The router passes the value directly to the OpenAI API.

Supported Effort Levels¶

Models Supporting reasoning_effort¶

O-series models (support low, medium, high):

• o1, o1-mini, o1-preview
• o3, o3-mini, o3-pro
• o4-mini

GPT-5.x thinking models (support low, medium, high, xhigh):

• gpt-5.4, gpt-5.4-pro, gpt-5.4-mini, gpt-5.4-nano
• gpt-5.2, gpt-5.2-thinking, gpt-5.2-pro

Models NOT Supporting reasoning_effort¶

The following models do not support reasoning parameters (parameter is stripped):

• GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4
• GPT-5.2-chat-latest, GPT-5.2-instant (non-thinking variants)
• GPT-5.1, GPT-5
• GPT-3.5-turbo
• Embedding models, Image models

Anthropic Backend (Claude)¶

Anthropic Claude models use different mechanisms depending on the model generation:

• Claude 4.6 models (Opus 4.6, Sonnet 4.6): Use adaptive thinking ({"type": "adaptive"}) with output_config.effort to control thinking depth
• Pre-4.6 models (Opus 4.5, Sonnet 4, etc.): Use thinking.budget_tokens for extended thinking

Conversion Table: Claude 4.6 Models (Adaptive Thinking)¶

Claude 4.6 models (claude-opus-4-6-*, claude-sonnet-4-6-*) use output_config.effort instead of budget_tokens:

Conversion Table: Pre-4.6 Claude Models (Budget Tokens)¶

Pre-4.6 models use thinking.budget_tokens:

Transformation Example: Claude 4.6 (Adaptive Thinking + Effort)¶

Input (OpenAI format):

{
  "model": "claude-opus-4-6-20260205",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "output_config": {
    "effort": "high"
  },
  "messages": [...]
}

For xhigh on Opus 4.6, output_config.effort is set to "max":

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "max"},
  "messages": [...]
}

Transformation Example: Pre-4.6 Claude (Budget Tokens)¶

Input (OpenAI format):

{
  "model": "claude-sonnet-4-20250514",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 32768
  },
  "messages": [...]
}

Adaptive Thinking (Auto)¶

When reasoning_effort is set to "auto", the router emits {"type": "adaptive"} without specifying output_config.effort. This lets Anthropic use its default effort level (currently high) and lets the model dynamically decide when and how much reasoning to apply.

Input (OpenAI format):

{
  "model": "claude-opus-4-6-20260205",
  "reasoning_effort": "auto",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {
    "type": "adaptive"
  },
  "messages": [...]
}

Direct output_config Pass-Through¶

When a request already contains explicit thinking and output_config parameters, they are passed through directly without conversion:

{
  "model": "claude-opus-4-6-20260205",
  "thinking": {"type": "adaptive"},
  "output_config": {"effort": "medium"},
  "messages": [...]
}

Models Supporting Extended Thinking¶

Extended thinking is supported by:

• Claude 4.6 family (adaptive thinking + effort): claude-opus-4-6-*, claude-sonnet-4-6-*
• Claude Opus models (budget tokens): claude-opus-4-*, claude-opus-4-5-*
• Claude Sonnet 4 models (budget tokens): claude-sonnet-4-*, claude-sonnet-4-5-*

Gemini Backend¶

Gemini models support reasoning_effort natively through their OpenAI-compatible endpoint. The router validates and passes the value directly.

Supported Effort Levels¶

Flash Models (support none)¶

• gemini-2.0-flash
• gemini-2.5-flash
• gemini-3-flash

Pro Models (do NOT support none)¶

• gemini-2.5-pro
• gemini-3-pro

Additional Features¶

For Gemini thinking models, the router automatically:

1. Sets include_thoughts: true to expose reasoning content
2. Sets default max_completion_tokens: 16384 if not specified
3. Validates effort level against model capabilities

Generic/HTTP Backend¶

Generic HTTP backends (used for Ollama, vLLM, LocalAI, LM Studio, etc.) pass requests through without transformation.

llama.cpp Backend¶

The llama.cpp backend uses pass-through behavior:

vLLM Backend¶

vLLM provides an OpenAI-compatible API but reasoning effort support depends on the model:

Summary Table¶

Response Format¶

When reasoning/thinking is enabled, the response includes the model's reasoning process:

OpenAI Format (with reasoning_content)¶

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "Let me think through this step by step..."
    },
    "finish_reason": "stop"
  }]
}

Claude Extended Thinking (transformed to OpenAI format)¶

The router transforms Claude's thinking blocks to the reasoning_content field:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "I need to consider multiple factors..."
    },
    "finish_reason": "stop"
  }]
}

Best Practices¶

1. Use appropriate effort levels: Higher effort = better reasoning but slower and more expensive
2. Check model support: Not all models support reasoning parameters
3. Handle xhigh carefully: Only GPT-5.2 family supports xhigh; use high for other models
4. Consider cost: Extended thinking consumes additional tokens (especially with Anthropic's budget_tokens)
5. Test with your backend: Generic backends may have varying support levels

Related Documentation¶

• API Reference - Chat Completions
• API Reference - Responses API
• Configuration Guide