Skip to content

Development Guide

This guide covers the development setup, architecture, testing, and contribution guidelines for the Continuum Router project.

Table of Contents

Prerequisites

Required Tools

  • Rust: 1.75 or later (Install Rust)
  • Git: For version control
  • C Compiler: gcc, clang, or MSVC (for native dependencies)
  • cargo-watch: For auto-recompilation during development
  • cargo-edit: For managing dependencies
  • cargo-audit: For security vulnerability checking
  • cargo-flamegraph: For performance profiling
  • rust-analyzer: IDE support (VS Code, IntelliJ, vim/neovim)

Install Development Tools

# Install essential development tools
cargo install cargo-watch cargo-edit cargo-audit cargo-flamegraph

# Install code coverage tools
cargo install cargo-tarpaulin

# Install benchmarking tools
cargo install cargo-criterion

Development Setup

Clone and Setup

# Clone the repository
git clone https://github.com/lablup/backend.ai-continuum.git
cd backend.ai-continuum/continuum-router

# Check that everything compiles
cargo check

# Run initial test suite
cargo test

# Build development version
cargo build

IDE Setup

VS Code

  1. Install the rust-analyzer extension
  2. Install the CodeLLDB extension for debugging
  3. Use the provided .vscode/settings.json:
{
  "rust-analyzer.cargo.features": "all",
  "rust-analyzer.checkOnSave.command": "clippy",
  "editor.formatOnSave": true,
  "[rust]": {
    "editor.defaultFormatter": "rust-lang.rust-analyzer"
  }
}

IntelliJ IDEA / CLion

  1. Install the Rust plugin
  2. Open the project root
  3. Configure Cargo settings in Preferences → Build → Cargo

Vim/Neovim

  1. Install rust-analyzer via Mason or manually
  2. Configure LSP client (nvim-lspconfig, coc.nvim, etc.)
  3. Example nvim-lspconfig setup:
require('lspconfig').rust_analyzer.setup({
  settings = {
    ["rust-analyzer"] = {
      cargo = { features = "all" },
      checkOnSave = { command = "clippy" }
    }
  }
})

Project Structure

Layer Architecture

The project follows a clean 4-layer architecture:

Directory Structure

src/
├── main.rs                 # Application entry point and CLI
├── lib.rs                  # Library exports and module declarations
│
├── core/                   # Core domain layer (no external dependencies)
│   ├── mod.rs             # Core module exports
│   ├── models/            # Domain models
│   │   ├── backend.rs     # Backend configuration and state
│   │   ├── model.rs       # LLM model representations
│   │   ├── request.rs     # Request/response models
│   │   └── health.rs      # Health check models
│   ├── errors.rs          # Error types and handling
│   ├── traits.rs          # Domain traits and interfaces
│   ├── retry/             # Retry logic and policies
│   │   ├── policy.rs      # Retry policies
│   │   └── backoff.rs     # Backoff strategies
│   └── config/            # Configuration models
│       ├── mod.rs         # Config module exports
│       ├── models.rs      # Configuration structures
│       └── validation.rs  # Config validation logic
│
├── services/              # Business logic layer
│   ├── mod.rs            # Service registry and DI container
│   ├── backend_service.rs    # Backend management and selection
│   ├── model_service.rs      # Model aggregation and caching
│   ├── proxy_service.rs      # Request routing and proxying
│   ├── health_service.rs     # Health monitoring
│   └── streaming/        # SSE streaming support
│       ├── mod.rs        # Streaming module exports
│       ├── parser.rs     # SSE event parsing
│       └── handler.rs    # Stream handling logic
│
├── infrastructure/        # External integrations layer
│   ├── mod.rs            # Infrastructure exports
│   ├── backends/         # Backend client implementations
│   │   ├── mod.rs        # Backend trait and factory
│   │   ├── openai.rs     # OpenAI-compatible backend
│   │   ├── ollama.rs     # Ollama backend
│   │   └── vllm.rs       # vLLM backend
│   ├── cache/            # Caching implementations
│   │   ├── mod.rs        # Cache traits
│   │   ├── lru.rs        # LRU cache
│   │   └── ttl.rs        # TTL-based cache
│   ├── config/           # Configuration loading
│   │   ├── loader.rs     # Config file loading
│   │   └── watcher.rs    # Hot-reload support
│   └── http_client.rs    # HTTP client with pooling
│
├── http/                  # HTTP layer
│   ├── mod.rs            # HTTP module exports
│   ├── routes.rs         # Route definitions
│   ├── handlers/         # Request handlers
│   │   ├── chat.rs       # Chat completion endpoints
│   │   ├── models.rs     # Model listing endpoints
│   │   ├── health.rs     # Health check endpoints
│   │   └── admin.rs      # Admin endpoints
│   ├── middleware/       # HTTP middleware
│   │   ├── auth.rs       # Authentication
│   │   ├── logging.rs    # Request logging
│   │   ├── metrics.rs    # Metrics collection
│   │   └── error.rs      # Error handling
│   └── dto/              # Data Transfer Objects
│       ├── request.rs    # Request DTOs
│       └── response.rs   # Response DTOs
│
└── metrics/              # Metrics and monitoring
    ├── mod.rs            # Metrics module exports
    ├── backend.rs        # Backend-specific metrics
    ├── routing.rs        # Routing metrics
    ├── model.rs          # Model service metrics
    └── streaming.rs      # Streaming metrics

tests/
├── integration/          # Integration tests
│   ├── health_test.rs    # Health endpoint tests
│   ├── chat_test.rs      # Chat completion tests
│   ├── models_test.rs    # Model listing tests
│   └── routing_test.rs   # Routing logic tests
├── unit/                 # Unit tests
│   ├── core/            # Core layer tests
│   ├── services/        # Service layer tests
│   └── infrastructure/  # Infrastructure tests
└── common/              # Test utilities
    ├── fixtures.rs      # Test fixtures
    ├── mocks.rs         # Mock implementations
    └── helpers.rs       # Test helper functions

examples/
├── standalone_server.rs  # Minimal server from a config file
├── axum_integration.rs   # Embedding router in an existing Axum app
├── programmatic_config.rs  # Configuration without YAML
└── hot_reload.rs         # Hot-reload and config-change subscriptions

Running Examples

The examples/ directory contains runnable Rust programs that demonstrate the library API. Run any example with:

cargo run --example <name> [-- <args>]

For example:

# Start a standalone server
cargo run --example standalone_server -- --config config.yaml

# Embed the router in a custom Axum app
cargo run --example axum_integration -- --config config.yaml

# Configure programmatically (no YAML file)
OPENAI_API_KEY=sk-... cargo run --example programmatic_config

# Demonstrate hot-reload
cargo run --example hot_reload -- --config config.yaml

See Library Usage for a full guide to the embeddable crate API.

Architecture Overview

Dependency Flow

HTTP Layer → Services Layer → Infrastructure Layer → Core Layer
     ↓            ↓                ↓                    ↑
     └────────────┴────────────────┴────────────────────┘
                        Uses Core Types

Key Design Patterns

  1. Dependency Injection: Services are injected via a container
  2. Repository Pattern: Backend implementations abstract data access
  3. Strategy Pattern: Load balancing strategies are pluggable
  4. Observer Pattern: Health monitoring uses event-driven updates
  5. Circuit Breaker: Prevents cascading failures
  6. Retry with Backoff: Handles transient failures

Building and Running

Development Build

# Quick development build
cargo build

# Run with debug output
RUST_LOG=debug ./target/debug/continuum-router

# Run with auto-reload
cargo watch -x run

# Run with specific config
cargo run -- --config dev-config.yaml

Release Build

# Optimized release build (all features enabled by default)
cargo build --release

# Run release build
./target/release/continuum-router

# Build with specific features only (minimal library build)
cargo build --lib --no-default-features --features "metrics,hot-reload"

Cargo Feature Flags

The crate exposes feature flags to enable optional components. The full feature set is enabled by default when building the binary.

Feature Description Default
full Enables all features below yes (via default)
cli CLI binary support via clap yes
admin Admin API endpoints yes
metrics Prometheus metrics endpoint yes
files Files API (/v1/files) yes
health-check Background health monitoring yes
hot-reload Config file watching for live reload yes
webui Embedded WebUI assets yes
gcp-auth Google Cloud auth for Gemini Service Account yes
redis-cache Redis/Valkey backend for the response cache (deadpool-redis) no

When using continuum-router as a library crate, you can select only the features you need to reduce compile times and binary size:

[dependencies]
continuum-router = { git = "...", default-features = false, features = ["metrics", "hot-reload"] }

Build examples:

# Minimal core library (no optional deps)
cargo build --lib --no-default-features

# Library with metrics support
cargo build --lib --no-default-features --features "metrics"

# Library with admin and hot-reload
cargo build --lib --no-default-features --features "admin,hot-reload"

# Full binary (default)
cargo build --release

Cross-Compilation

# Install cross
cargo install cross

# Build for Linux x86_64
cross build --release --target x86_64-unknown-linux-gnu

# Build for macOS ARM64
cross build --release --target aarch64-apple-darwin

# Build for Windows
cross build --release --target x86_64-pc-windows-gnu

Testing

Running Tests

# Run all tests
cargo test

# Run with output
cargo test -- --nocapture

# Run specific test
cargo test test_health_check

# Run tests in single thread (for debugging)
cargo test -- --test-threads=1

# Run integration tests only
cargo test --test integration

# Run unit tests only
cargo test --lib

# Run doc tests
cargo test --doc

Writing Tests

Unit Test Example

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_backend_selection() {
        let backends = vec![
            Backend::new("backend1", "http://localhost:8000"),
            Backend::new("backend2", "http://localhost:8001"),
        ];

        let selector = RoundRobinSelector::new(backends);
        let selected = selector.select().unwrap();

        assert_eq!(selected.name(), "backend1");
    }

    #[tokio::test]
    async fn test_async_health_check() {
        let backend = Backend::new("test", "http://localhost:8000");
        let result = backend.health_check().await;

        assert!(result.is_ok());
    }
}

Integration Test Example

// tests/integration/chat_test.rs
use continuum_router::test_utils::TestServer;

#[tokio::test]
async fn test_chat_completion() {
    let server = TestServer::new().await;

    let response = server
        .post("/v1/chat/completions")
        .json(&json!({
            "model": "gpt-3.5-turbo",
            "messages": [{"role": "user", "content": "Hello"}]
        }))
        .send()
        .await;

    assert_eq!(response.status(), 200);
}

Test Coverage

# Generate coverage report
cargo tarpaulin --out Html

# Generate coverage with specific threshold
cargo tarpaulin --fail-under 80

# Generate Codecov report
cargo tarpaulin --out Xml

Code Quality

Formatting

# Format all code
cargo fmt

# Check formatting without changes
cargo fmt -- --check

# Format specific file
cargo fmt -- src/main.rs

Linting

# Run clippy with warnings as errors
cargo clippy -- -D warnings

# Run clippy with pedantic lints
cargo clippy -- -W clippy::pedantic

# Run clippy with auto-fix
cargo clippy --fix

Security Audit

# Check for known vulnerabilities
cargo audit

# Fix vulnerable dependencies
cargo audit fix

# Generate security report
cargo audit --json > audit-report.json

Documentation

# Build documentation
cargo doc

# Build and open documentation
cargo doc --open

# Build documentation with private items
cargo doc --document-private-items

# Check documentation examples
cargo test --doc

Debugging

Debug Logging

# Enable debug logging
RUST_LOG=debug cargo run

# Enable trace logging for specific module
RUST_LOG=continuum_router::services=trace cargo run

# Enable structured logging
RUST_LOG=info RUST_LOG_FORMAT=json cargo run

Using Debugger

VS Code

  1. Create .vscode/launch.json:
{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "lldb",
      "request": "launch",
      "name": "Debug Router",
      "cargo": {
        "args": ["build", "--bin=continuum-router"],
        "filter": {
          "name": "continuum-router",
          "kind": "bin"
        }
      },
      "args": ["--config", "dev-config.yaml"],
      "cwd": "${workspaceFolder}"
    }
  ]
}
  1. Set breakpoints and press F5 to debug

Command Line (lldb)

# Build with debug symbols
cargo build

# Run with lldb
lldb target/debug/continuum-router

# Set breakpoint
(lldb) b src/main.rs:50

# Run program
(lldb) r --config config.yaml

Memory Debugging

# Use valgrind (Linux)
valgrind --leak-check=full ./target/debug/continuum-router

# Use Address Sanitizer
RUSTFLAGS="-Z sanitizer=address" cargo build -Z build-std --target x86_64-unknown-linux-gnu
./target/x86_64-unknown-linux-gnu/debug/continuum-router

Performance Profiling

CPU Profiling

# Generate flame graph
cargo flamegraph -- --config config.yaml

# Profile specific scenario
cargo flamegraph --bin continuum-router -- --benchmark

# Use perf (Linux)
perf record -g ./target/release/continuum-router
perf report

Benchmarking

# Run benchmarks
cargo bench

# Run specific benchmark
cargo bench routing

# Compare benchmark results
cargo bench -- --save-baseline main
git checkout feature-branch
cargo bench -- --baseline main

Writing Benchmarks

// benches/routing_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn benchmark_routing(c: &mut Criterion) {
    c.bench_function("round_robin_selection", |b| {
        let selector = RoundRobinSelector::new(backends);
        b.iter(|| {
            selector.select(black_box(&request))
        });
    });
}

criterion_group!(benches, benchmark_routing);
criterion_main!(benches);

Memory Profiling

# Use heaptrack (Linux)
heaptrack ./target/release/continuum-router
heaptrack_gui heaptrack.continuum-router.*.gz

# Use Instruments (macOS)
cargo instruments -t Allocations

Contributing

Development Workflow

  1. Fork and Clone 

    git clone https://github.com/YOUR_USERNAME/backend.ai-continuum.git
cd backend.ai-continuum/continuum-router

  2. Create Feature Branch 

    git checkout -b feature/amazing-feature

  3. Make Changes

  4. Write code following Rust conventions
  5. Add tests for new functionality

  6. Update documentation

  7. Test Your Changes 

    cargo test
cargo fmt
cargo clippy -- -D warnings

  8. Commit Changes 

    git add .
git commit -m "feat: Add amazing feature"

  9. Push and Create PR 

    git push origin feature/amazing-feature

Code Standards

  • Follow Rust naming conventions (snake_case for functions, CamelCase for types)
  • Write comprehensive tests (aim for >90% coverage)
  • Document public APIs with doc comments
  • Use cargo fmt for consistent formatting
  • Address all cargo clippy warnings
  • Keep functions small and focused
  • Prefer composition over inheritance
  • Use appropriate error handling (Result/Option)

Commit Message Format

Follow the conventional commits specification:

<type>(<scope>): <subject>

<body>

<footer>

Types: - feat: New feature - fix: Bug fix - docs: Documentation only - style: Code style changes - refactor: Code refactoring - perf: Performance improvements - test: Adding tests - chore: Maintenance tasks

Pull Request Guidelines

  1. Title: Clear and descriptive
  2. Description: Explain what, why, and how
  3. Tests: Include tests for new features
  4. Documentation: Update relevant docs
  5. Breaking Changes: Clearly marked
  6. Issues: Reference related issues

Review Process

  1. Automated checks must pass
  2. At least one maintainer approval
  3. No unresolved conversations
  4. Up-to-date with main branch

Troubleshooting

Common Issues

Compilation Errors

# Clear build cache
cargo clean

# Update dependencies
cargo update

# Check for conflicting features
cargo tree --duplicates

Test Failures

# Run tests verbosely
cargo test -- --nocapture --test-threads=1

# Check for race conditions
cargo test -- --test-threads=1

# Verify test environment
cargo test -- --ignored

Performance Issues

# Profile release build
cargo build --release
perf record ./target/release/continuum-router
perf report

# Check for blocking operations
RUST_LOG=trace cargo run 2>&1 | grep "blocking"

Resources