Skip to content

Installation Guide

This guide covers various methods to install and set up the Continuum Router.

System Requirements

  • Operating System: Linux, macOS, or Windows
  • Architecture: x86_64 or ARM64
  • Memory: Minimum 256MB RAM (512MB recommended)
  • Disk Space: 100MB for binary and configuration

Installation Methods

The easiest way to get started is by downloading pre-built binaries from GitHub releases.

Linux (x86_64)

# Download the latest release
curl -L https://github.com/lablup/continuum-router/releases/latest/download/continuum-router-linux-x86_64.tar.gz | tar -xz

# Move to PATH
sudo mv continuum-router /usr/local/bin/

# Verify installation
continuum-router --version

Linux (aarch64)

# Download the latest release
curl -L https://github.com/lablup/continuum-router/releases/latest/download/continuum-router-linux-aarch64.tar.gz | tar -xz

# Move to PATH
sudo mv continuum-router /usr/local/bin/

# Verify installation
continuum-router --version

macOS (Apple Silicon)

# Download the latest release
curl -LO https://github.com/lablup/continuum-router/releases/latest/download/continuum-router-macos-aarch64.zip

# Extract
unzip continuum-router-macos-aarch64.zip

# Move to PATH
sudo mv continuum-router /usr/local/bin/

# Verify installation
continuum-router --version

Windows (x86_64)

# Download the latest release
Invoke-WebRequest -Uri https://github.com/lablup/continuum-router/releases/latest/download/continuum-router-windows-x86_64.zip -OutFile continuum-router.zip

# Extract and place continuum-router.exe somewhere on your PATH
Expand-Archive continuum-router.zip -DestinationPath .

# Verify installation
.\continuum-router.exe --version

Static musl Builds (Linux)

For distributions with older glibc, or minimal container images such as Alpine, statically linked musl builds are available: continuum-router-linux-x86_64-musl.tar.gz and continuum-router-linux-aarch64-musl.tar.gz.

From Source

Building from source gives you the latest features and allows customization.

Prerequisites

  • Rust (latest stable toolchain recommended)
  • Git
  • C compiler (gcc, clang, or MSVC)

Build Steps

# Clone the repository
git clone https://github.com/lablup/continuum-router.git
cd continuum-router

# Development build (faster compilation, larger binary)
cargo build

# Run development build
./target/debug/continuum-router --version

# Optimized release build (slower compilation, smaller & faster binary)
cargo build --release

# Run release build
./target/release/continuum-router --version

# Install to system (optional)
cargo install --path .

Using Docker

Docker provides a consistent environment across different systems.

Quick Start with Docker

# Run with default settings
docker run -p 8080:8080 ghcr.io/lablup/continuum-router:latest

# Run with backend configuration
docker run -p 8080:8080 \
  -e CONTINUUM_BACKEND_URLS="http://host.docker.internal:11434" \
  ghcr.io/lablup/continuum-router:latest

# Run with configuration file
docker run -p 8080:8080 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/lablup/continuum-router:latest

Docker Compose Setup

Create a docker-compose.yml file:

version: '3.8'
services:
  continuum-router:
    image: ghcr.io/lablup/continuum-router:latest
    ports:
      - "8080:8080"
    environment:
      - CONTINUUM_BACKEND_URLS=http://ollama:11434,http://lm-studio:1234
      - CONTINUUM_LOG_LEVEL=info
    volumes:
      - ./config.yaml:/app/config.yaml
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 5s

  # Example: Ollama backend
  ollama:
    image: ollama/ollama:latest
    volumes:
      - ollama_data:/root/.ollama
    ports:
      - "11434:11434"

volumes:
  ollama_data:

Run with Docker Compose: 

docker-compose up -d

Debian/Ubuntu (.deb Packages)

Each GitHub release includes .deb packages for supported Ubuntu LTS versions, in amd64 and arm64 variants. Download the package matching your distribution from the releases page and install it with dpkg:

# Example: Ubuntu 24.04 (noble) on amd64 — adjust version and distribution
sudo dpkg -i continuum-router_<version>_ubuntu24.04.noble_amd64.deb

# Verify installation
continuum-router --version
man continuum-router

The package installs the continuum-router binary and its man page.

Post-Installation Setup

1. Generate Configuration

# Generate sample configuration (annotated, all available options)
continuum-router --generate-config > config.yaml

# Or generate a TOML-format sample
continuum-router --generate-toml-config > config.toml

# Edit configuration
nano config.yaml

2. Set Up as System Service

Linux (systemd)

Create /etc/systemd/system/continuum-router.service:

[Unit]
Description=Continuum Router - LLM API Router
After=network.target
Wants=network.target

[Service]
Type=simple
User=continuum
Group=continuum
WorkingDirectory=/opt/continuum-router
ExecStart=/usr/local/bin/continuum-router --config /etc/continuum-router/config.yaml
Restart=always
RestartSec=10

# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/log/continuum-router

# Environment
Environment=RUST_LOG=continuum_router=info
Environment=RUST_BACKTRACE=1

[Install]
WantedBy=multi-user.target

Enable and start the service: 

sudo systemctl daemon-reload
sudo systemctl enable continuum-router
sudo systemctl start continuum-router
sudo systemctl status continuum-router

macOS (launchd)

Create ~/Library/LaunchAgents/com.lablup.continuum-router.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.lablup.continuum-router</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/continuum-router</string>
        <string>--config</string>
        <string>/usr/local/etc/continuum-router/config.yaml</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/usr/local/var/log/continuum-router.log</string>
    <key>StandardErrorPath</key>
    <string>/usr/local/var/log/continuum-router.error.log</string>
</dict>
</plist>

Load the service: 

launchctl load ~/Library/LaunchAgents/com.lablup.continuum-router.plist
launchctl start com.lablup.continuum-router

3. Verify Installation

# Check version
continuum-router --version

# Test health endpoint
curl http://localhost:8080/health

# List available models
curl http://localhost:8080/v1/models

# Check backend status
curl http://localhost:8080/admin/backends

Upgrading

Binary Installation

# Download new version
curl -L https://github.com/lablup/continuum-router/releases/latest/download/continuum-router-linux-x86_64.tar.gz | tar -xz

# Backup old binary
sudo mv /usr/local/bin/continuum-router /usr/local/bin/continuum-router.backup

# Install new binary
sudo mv continuum-router /usr/local/bin/

# Restart service
sudo systemctl restart continuum-router

Docker

# Pull latest image
docker pull ghcr.io/lablup/continuum-router:latest

# Restart container
docker-compose down
docker-compose up -d

From Source

cd backend.ai-continuum/continuum-router
git pull
cargo build --release
sudo mv target/release/continuum-router /usr/local/bin/
sudo systemctl restart continuum-router

Uninstallation

Binary Installation

# Stop service
sudo systemctl stop continuum-router
sudo systemctl disable continuum-router

# Remove binary
sudo rm /usr/local/bin/continuum-router

# Remove configuration
sudo rm -rf /etc/continuum-router

# Remove service file
sudo rm /etc/systemd/system/continuum-router.service
sudo systemctl daemon-reload

Docker

# Stop and remove containers
docker-compose down

# Remove image
docker rmi ghcr.io/lablup/continuum-router:latest

# Remove volumes (if any)
docker volume prune

Troubleshooting

Permission Denied

# Make binary executable
chmod +x continuum-router

# For system directories, use sudo
sudo mv continuum-router /usr/local/bin/

Port Already in Use

# Check what's using port 8080
lsof -i :8080

# Use a different port
continuum-router --bind-address 0.0.0.0:9090

Service Won't Start

# Check logs
journalctl -u continuum-router -f

# Validate configuration
continuum-router --config config.yaml --validate

# Run in foreground for debugging
continuum-router --config config.yaml --log-level debug

Docker Network Issues

# Use host network mode
docker run --network host ghcr.io/lablup/continuum-router:latest

# Or specify backend IPs explicitly
docker run -e CONTINUUM_BACKEND_URLS="http://172.17.0.2:11434" ...

Execution Modes

Continuum Router supports two execution modes, selected via the --mode flag:

HTTP Mode (Default)

continuum-router --config config.yaml
# equivalent to:
continuum-router --config config.yaml --mode http

Binds TCP and/or Unix socket listeners and serves the OpenAI-compatible REST API. This is the standard mode for production deployments.

Stdio Mode (ACP)

continuum-router --config config.yaml --mode stdio

Runs the router as an ACP (Agent Communication Protocol) server that reads JSON-RPC 2.0 messages from stdin and writes responses to stdout. HTTP listeners are not started. This mode is designed for IDE and tool integrations that spawn the router as a subprocess.

See the ACP Usage Guide for configuration and usage details.

Next Steps