Skip to content

Development Guide

This guide covers the development setup, architecture, testing, and contribution guidelines for the Continuum Router project.

Table of Contents

Prerequisites

Required Tools

  • Rust: 1.75 or later (Install Rust)
  • Git: For version control
  • C Compiler: gcc, clang, or MSVC (for native dependencies)
  • cargo-watch: For auto-recompilation during development
  • cargo-edit: For managing dependencies
  • cargo-audit: For security vulnerability checking
  • cargo-flamegraph: For performance profiling
  • rust-analyzer: IDE support (VS Code, IntelliJ, vim/neovim)

Install Development Tools

# Install essential development tools
cargo install cargo-watch cargo-edit cargo-audit cargo-flamegraph

# Install code coverage tools
cargo install cargo-tarpaulin

# Install benchmarking tools
cargo install cargo-criterion

Development Setup

Clone and Setup

# Clone the repository
git clone https://github.com/lablup/backend.ai-continuum.git
cd backend.ai-continuum/continuum-router

# Check that everything compiles
cargo check

# Run initial test suite
cargo test

# Build development version
cargo build

IDE Setup

VS Code

  1. Install the rust-analyzer extension
  2. Install the CodeLLDB extension for debugging
  3. Use the provided .vscode/settings.json:
{
  "rust-analyzer.cargo.features": "all",
  "rust-analyzer.checkOnSave.command": "clippy",
  "editor.formatOnSave": true,
  "[rust]": {
    "editor.defaultFormatter": "rust-lang.rust-analyzer"
  }
}

IntelliJ IDEA / CLion

  1. Install the Rust plugin
  2. Open the project root
  3. Configure Cargo settings in Preferences → Build → Cargo

Vim/Neovim

  1. Install rust-analyzer via Mason or manually
  2. Configure LSP client (nvim-lspconfig, coc.nvim, etc.)
  3. Example nvim-lspconfig setup:
require('lspconfig').rust_analyzer.setup({
  settings = {
    ["rust-analyzer"] = {
      cargo = { features = "all" },
      checkOnSave = { command = "clippy" }
    }
  }
})

Project Structure

Layer Architecture

The project follows a clean 4-layer architecture:

┌─────────────────────────────────┐
│         HTTP Layer              │  ← Routes, Middleware, DTOs
├─────────────────────────────────┤
│       Services Layer            │  ← Business Logic, Orchestration  
├─────────────────────────────────┤
│    Infrastructure Layer         │  ← Backends, Cache, HTTP Client
├─────────────────────────────────┤
│         Core Layer              │  ← Domain Models, Traits, Errors
└─────────────────────────────────┘

Directory Structure

src/
├── main.rs                 # Application entry point and CLI
├── lib.rs                  # Library exports and module declarations
│
├── core/                   # Core domain layer (no external dependencies)
│   ├── mod.rs             # Core module exports
│   ├── models/            # Domain models
│   │   ├── backend.rs     # Backend configuration and state
│   │   ├── model.rs       # LLM model representations
│   │   ├── request.rs     # Request/response models
│   │   └── health.rs      # Health check models
│   ├── errors.rs          # Error types and handling
│   ├── traits.rs          # Domain traits and interfaces
│   ├── retry/             # Retry logic and policies
│   │   ├── policy.rs      # Retry policies
│   │   └── backoff.rs     # Backoff strategies
│   └── config/            # Configuration models
│       ├── mod.rs         # Config module exports
│       ├── models.rs      # Configuration structures
│       └── validation.rs  # Config validation logic
│
├── services/              # Business logic layer
│   ├── mod.rs            # Service registry and DI container
│   ├── backend_service.rs    # Backend management and selection
│   ├── model_service.rs      # Model aggregation and caching
│   ├── proxy_service.rs      # Request routing and proxying
│   ├── health_service.rs     # Health monitoring
│   └── streaming/        # SSE streaming support
│       ├── mod.rs        # Streaming module exports
│       ├── parser.rs     # SSE event parsing
│       └── handler.rs    # Stream handling logic
│
├── infrastructure/        # External integrations layer
│   ├── mod.rs            # Infrastructure exports
│   ├── backends/         # Backend client implementations
│   │   ├── mod.rs        # Backend trait and factory
│   │   ├── openai.rs     # OpenAI-compatible backend
│   │   ├── ollama.rs     # Ollama backend
│   │   └── vllm.rs       # vLLM backend
│   ├── cache/            # Caching implementations
│   │   ├── mod.rs        # Cache traits
│   │   ├── lru.rs        # LRU cache
│   │   └── ttl.rs        # TTL-based cache
│   ├── config/           # Configuration loading
│   │   ├── loader.rs     # Config file loading
│   │   └── watcher.rs    # Hot-reload support
│   └── http_client.rs    # HTTP client with pooling
│
├── http/                  # HTTP layer
│   ├── mod.rs            # HTTP module exports
│   ├── routes.rs         # Route definitions
│   ├── handlers/         # Request handlers
│   │   ├── chat.rs       # Chat completion endpoints
│   │   ├── models.rs     # Model listing endpoints
│   │   ├── health.rs     # Health check endpoints
│   │   └── admin.rs      # Admin endpoints
│   ├── middleware/       # HTTP middleware
│   │   ├── auth.rs       # Authentication
│   │   ├── logging.rs    # Request logging
│   │   ├── metrics.rs    # Metrics collection
│   │   └── error.rs      # Error handling
│   └── dto/              # Data Transfer Objects
│       ├── request.rs    # Request DTOs
│       └── response.rs   # Response DTOs
│
└── metrics/              # Metrics and monitoring
    ├── mod.rs            # Metrics module exports
    ├── backend.rs        # Backend-specific metrics
    ├── routing.rs        # Routing metrics
    ├── model.rs          # Model service metrics
    └── streaming.rs      # Streaming metrics

tests/
├── integration/          # Integration tests
│   ├── health_test.rs    # Health endpoint tests
│   ├── chat_test.rs      # Chat completion tests
│   ├── models_test.rs    # Model listing tests
│   └── routing_test.rs   # Routing logic tests
├── unit/                 # Unit tests
│   ├── core/            # Core layer tests
│   ├── services/        # Service layer tests
│   └── infrastructure/  # Infrastructure tests
└── common/              # Test utilities
    ├── fixtures.rs      # Test fixtures
    ├── mocks.rs         # Mock implementations
    └── helpers.rs       # Test helper functions

Architecture Overview

Dependency Flow

HTTP Layer → Services Layer → Infrastructure Layer → Core Layer
     ↓            ↓                ↓                    ↑
     └────────────┴────────────────┴────────────────────┘
                        Uses Core Types

Key Design Patterns

  1. Dependency Injection: Services are injected via a container
  2. Repository Pattern: Backend implementations abstract data access
  3. Strategy Pattern: Load balancing strategies are pluggable
  4. Observer Pattern: Health monitoring uses event-driven updates
  5. Circuit Breaker: Prevents cascading failures
  6. Retry with Backoff: Handles transient failures

Building and Running

Development Build

# Quick development build
cargo build

# Run with debug output
RUST_LOG=debug ./target/debug/continuum-router

# Run with auto-reload
cargo watch -x run

# Run with specific config
cargo run -- --config dev-config.yaml

Release Build

# Optimized release build
cargo build --release

# Run release build
./target/release/continuum-router

# Build with specific features
cargo build --release --features "metrics,tracing"

Cross-Compilation

# Install cross
cargo install cross

# Build for Linux x86_64
cross build --release --target x86_64-unknown-linux-gnu

# Build for macOS ARM64
cross build --release --target aarch64-apple-darwin

# Build for Windows
cross build --release --target x86_64-pc-windows-gnu

Testing

Running Tests

# Run all tests
cargo test

# Run with output
cargo test -- --nocapture

# Run specific test
cargo test test_health_check

# Run tests in single thread (for debugging)
cargo test -- --test-threads=1

# Run integration tests only
cargo test --test integration

# Run unit tests only
cargo test --lib

# Run doc tests
cargo test --doc

Writing Tests

Unit Test Example

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_backend_selection() {
        let backends = vec![
            Backend::new("backend1", "http://localhost:8000"),
            Backend::new("backend2", "http://localhost:8001"),
        ];

        let selector = RoundRobinSelector::new(backends);
        let selected = selector.select().unwrap();

        assert_eq!(selected.name(), "backend1");
    }

    #[tokio::test]
    async fn test_async_health_check() {
        let backend = Backend::new("test", "http://localhost:8000");
        let result = backend.health_check().await;

        assert!(result.is_ok());
    }
}

Integration Test Example

// tests/integration/chat_test.rs
use continuum_router::test_utils::TestServer;

#[tokio::test]
async fn test_chat_completion() {
    let server = TestServer::new().await;

    let response = server
        .post("/v1/chat/completions")
        .json(&json!({
            "model": "gpt-3.5-turbo",
            "messages": [{"role": "user", "content": "Hello"}]
        }))
        .send()
        .await;

    assert_eq!(response.status(), 200);
}

Test Coverage

# Generate coverage report
cargo tarpaulin --out Html

# Generate coverage with specific threshold
cargo tarpaulin --fail-under 80

# Generate Codecov report
cargo tarpaulin --out Xml

Code Quality

Formatting

# Format all code
cargo fmt

# Check formatting without changes
cargo fmt -- --check

# Format specific file
cargo fmt -- src/main.rs

Linting

# Run clippy with warnings as errors
cargo clippy -- -D warnings

# Run clippy with pedantic lints
cargo clippy -- -W clippy::pedantic

# Run clippy with auto-fix
cargo clippy --fix

Security Audit

# Check for known vulnerabilities
cargo audit

# Fix vulnerable dependencies
cargo audit fix

# Generate security report
cargo audit --json > audit-report.json

Documentation

# Build documentation
cargo doc

# Build and open documentation
cargo doc --open

# Build documentation with private items
cargo doc --document-private-items

# Check documentation examples
cargo test --doc

Debugging

Debug Logging

# Enable debug logging
RUST_LOG=debug cargo run

# Enable trace logging for specific module
RUST_LOG=continuum_router::services=trace cargo run

# Enable structured logging
RUST_LOG=info RUST_LOG_FORMAT=json cargo run

Using Debugger

VS Code

  1. Create .vscode/launch.json:
{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "lldb",
      "request": "launch",
      "name": "Debug Router",
      "cargo": {
        "args": ["build", "--bin=continuum-router"],
        "filter": {
          "name": "continuum-router",
          "kind": "bin"
        }
      },
      "args": ["--config", "dev-config.yaml"],
      "cwd": "${workspaceFolder}"
    }
  ]
}
  1. Set breakpoints and press F5 to debug

Command Line (lldb)

# Build with debug symbols
cargo build

# Run with lldb
lldb target/debug/continuum-router

# Set breakpoint
(lldb) b src/main.rs:50

# Run program
(lldb) r --config config.yaml

Memory Debugging

# Use valgrind (Linux)
valgrind --leak-check=full ./target/debug/continuum-router

# Use Address Sanitizer
RUSTFLAGS="-Z sanitizer=address" cargo build -Z build-std --target x86_64-unknown-linux-gnu
./target/x86_64-unknown-linux-gnu/debug/continuum-router

Performance Profiling

CPU Profiling

# Generate flame graph
cargo flamegraph -- --config config.yaml

# Profile specific scenario
cargo flamegraph --bin continuum-router -- --benchmark

# Use perf (Linux)
perf record -g ./target/release/continuum-router
perf report

Benchmarking

# Run benchmarks
cargo bench

# Run specific benchmark
cargo bench routing

# Compare benchmark results
cargo bench -- --save-baseline main
git checkout feature-branch
cargo bench -- --baseline main

Writing Benchmarks

// benches/routing_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};

fn benchmark_routing(c: &mut Criterion) {
    c.bench_function("round_robin_selection", |b| {
        let selector = RoundRobinSelector::new(backends);
        b.iter(|| {
            selector.select(black_box(&request))
        });
    });
}

criterion_group!(benches, benchmark_routing);
criterion_main!(benches);

Memory Profiling

# Use heaptrack (Linux)
heaptrack ./target/release/continuum-router
heaptrack_gui heaptrack.continuum-router.*.gz

# Use Instruments (macOS)
cargo instruments -t Allocations

Contributing

Development Workflow

  1. Fork and Clone 

    git clone https://github.com/YOUR_USERNAME/backend.ai-continuum.git
cd backend.ai-continuum/continuum-router

  2. Create Feature Branch 

    git checkout -b feature/amazing-feature

  3. Make Changes

  4. Write code following Rust conventions
  5. Add tests for new functionality

  6. Update documentation

  7. Test Your Changes 

    cargo test
cargo fmt
cargo clippy -- -D warnings

  8. Commit Changes 

    git add .
git commit -m "feat: Add amazing feature"

  9. Push and Create PR 

    git push origin feature/amazing-feature

Code Standards

  • Follow Rust naming conventions (snake_case for functions, CamelCase for types)
  • Write comprehensive tests (aim for >90% coverage)
  • Document public APIs with doc comments
  • Use cargo fmt for consistent formatting
  • Address all cargo clippy warnings
  • Keep functions small and focused
  • Prefer composition over inheritance
  • Use appropriate error handling (Result/Option)

Commit Message Format

Follow the conventional commits specification:

<type>(<scope>): <subject>

<body>

<footer>

Types: - feat: New feature - fix: Bug fix - docs: Documentation only - style: Code style changes - refactor: Code refactoring - perf: Performance improvements - test: Adding tests - chore: Maintenance tasks

Pull Request Guidelines

  1. Title: Clear and descriptive
  2. Description: Explain what, why, and how
  3. Tests: Include tests for new features
  4. Documentation: Update relevant docs
  5. Breaking Changes: Clearly marked
  6. Issues: Reference related issues

Review Process

  1. Automated checks must pass
  2. At least one maintainer approval
  3. No unresolved conversations
  4. Up-to-date with main branch

Troubleshooting

Common Issues

Compilation Errors

# Clear build cache
cargo clean

# Update dependencies
cargo update

# Check for conflicting features
cargo tree --duplicates

Test Failures

# Run tests verbosely
cargo test -- --nocapture --test-threads=1

# Check for race conditions
cargo test -- --test-threads=1

# Verify test environment
cargo test -- --ignored

Performance Issues

# Profile release build
cargo build --release
perf record ./target/release/continuum-router
perf report

# Check for blocking operations
RUST_LOG=trace cargo run 2>&1 | grep "blocking"

Resources