Reasoning Effort Parameter¶

This document describes how Continuum Router handles the reasoning_effort parameter across different LLM backends, including the token budget conversions and supported effort levels.

Overview¶

The reasoning_effort parameter controls the amount of computational effort a model spends on reasoning before generating a response. Different backends implement this feature differently:

• OpenAI: Native reasoning_effort parameter for O-series and GPT-5.2 thinking models
• Anthropic: Converted to thinking.budget_tokens for Claude models with extended thinking
• Gemini: Native reasoning_effort for thinking-capable models via OpenAI-compatible endpoint
• Other backends: Pass-through (no transformation)

Parameter Formats¶

Continuum Router supports two input formats, both normalized internally:

Flat Format (Chat Completions API)¶

{
  "model": "o3-mini",
  "reasoning_effort": "high",
  "messages": [...]
}

Nested Format (Responses API)¶

{
  "model": "o3-mini",
  "reasoning": {
    "effort": "high"
  },
  "input": "..."
}

Both formats are automatically normalized to the flat reasoning_effort format before processing. If both are present, the flat format takes precedence.

Direct Token Budget Specification¶

For advanced use cases, you can specify the token budget directly instead of using reasoning_effort levels:

Anthropic: Direct thinking Parameter¶

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 16000
  },
  "messages": [...]
}

Gemini: Direct thinking_budget via extra_body¶

{
  "model": "gemini-2.5-pro",
  "extra_body": {
    "google": {
      "thinking_config": {
        "thinking_budget": 10000,
        "include_thoughts": true
      }
    }
  },
  "messages": [...]
}

Backend-Specific Behavior¶

OpenAI Backend¶

OpenAI models support the reasoning_effort parameter natively. The router passes the value directly to the OpenAI API.

Supported Effort Levels¶

Models Supporting reasoning_effort¶

O-series models (support low, medium, high):

• o1, o1-mini, o1-preview
• o3, o3-mini, o3-pro
• o4-mini

GPT-5.2 thinking models (support low, medium, high, xhigh):

• gpt-5.2, gpt-5.2-thinking, gpt-5.2-latest
• gpt-5.2-pro

Models NOT Supporting reasoning_effort¶

The following models do not support reasoning parameters (parameter is stripped):

• GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-4
• GPT-5.2-chat-latest, GPT-5.2-instant (non-thinking variants)
• GPT-5.1, GPT-5
• GPT-3.5-turbo
• Embedding models, Image models

Anthropic Backend (Claude)¶

Anthropic Claude models use a different mechanism called "extended thinking" with a thinking.budget_tokens parameter. The router automatically converts reasoning_effort to the appropriate token budget.

Conversion Table¶

Transformation Example¶

Input (OpenAI format):

{
  "model": "claude-sonnet-4-20250514",
  "reasoning_effort": "high",
  "messages": [...]
}

Transformed (Anthropic format):

{
  "model": "claude-sonnet-4-20250514",
  "thinking": {
    "type": "enabled",
    "budget_tokens": 32768
  },
  "messages": [...]
}

Models Supporting Extended Thinking¶

Extended thinking is supported by:

• Claude Opus models: claude-opus-4-*, claude-opus-4-5-*
• Claude Sonnet 4 models: claude-sonnet-4-*, claude-sonnet-4-5-*

Gemini Backend¶

Gemini models support reasoning_effort natively through their OpenAI-compatible endpoint. The router validates and passes the value directly.

Supported Effort Levels¶

Flash Models (support none)¶

• gemini-2.0-flash
• gemini-2.5-flash
• gemini-3-flash

Pro Models (do NOT support none)¶

• gemini-2.5-pro
• gemini-3-pro

Additional Features¶

For Gemini thinking models, the router automatically:

1. Sets include_thoughts: true to expose reasoning content
2. Sets default max_completion_tokens: 16384 if not specified
3. Validates effort level against model capabilities

Generic/HTTP Backend¶

Generic HTTP backends (used for Ollama, vLLM, LocalAI, LM Studio, etc.) pass requests through without transformation.

llama.cpp Backend¶

The llama.cpp backend uses pass-through behavior:

vLLM Backend¶

vLLM provides an OpenAI-compatible API but reasoning effort support depends on the model:

Summary Table¶

Response Format¶

When reasoning/thinking is enabled, the response includes the model's reasoning process:

OpenAI Format (with reasoning_content)¶

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "Let me think through this step by step..."
    },
    "finish_reason": "stop"
  }]
}

Claude Extended Thinking (transformed to OpenAI format)¶

The router transforms Claude's thinking blocks to the reasoning_content field:

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "The answer is 42.",
      "reasoning_content": "I need to consider multiple factors..."
    },
    "finish_reason": "stop"
  }]
}

Best Practices¶

1. Use appropriate effort levels: Higher effort = better reasoning but slower and more expensive
2. Check model support: Not all models support reasoning parameters
3. Handle xhigh carefully: Only GPT-5.2 family supports xhigh; use high for other models
4. Consider cost: Extended thinking consumes additional tokens (especially with Anthropic's budget_tokens)
5. Test with your backend: Generic backends may have varying support levels

Related Documentation¶

• API Reference - Chat Completions
• API Reference - Responses API
• Configuration Guide