Fine-Tuning Passthrough: Design Record¶

This note records the design for the /v1/fine_tuning/jobs passthrough (issue #2358). The routes are implemented in src/http/handlers/fine_tuning.rs and src/services/fine_tuning.rs; this page records the design they follow.

Implemented surface¶

The /v1 surface covers chat, completions, responses, embeddings, rerank, images, audio, Anthropic Messages, files, tokenize, models, realtime, batches (only when the control-plane feature is compiled), and the fine-tuning job routes below. The job routes are mounted on every build.

Each response is the backend's own status, content type, and body. A list body is the exception to byte-identical relaying: its data array keeps only the caller's own jobs (see Job bindings).

Policy layers¶

Job CRUD is not a chat dispatch. The router applies the API route group's auth, global, per-client, and per-key rate limits to every fine-tuning call, charges the per-backend dispatch limit on create and list, and deliberately does not run the layers that assume a completion payload:

• Request-parameter policy: not applied. It rewrites completion sampling and tool fields that a job payload does not carry.
• Guardrails: not applied. The input and output gates judge message and completion text; the router never sees the training data.
• Response caching: not applied. A cached listing or create would hide job state changes.
• Per-model generation budget: not charged. A job call generates no tokens, and charging it would spend a model's completion budget on control-plane traffic.
• Metering: applied as a non-completion request. The router records request count, latency, and key and backend attribution with zero tokens (token_source: none), because the answer has no usage block to count.

Design¶

• Backend scope: OpenAI-wire types on http/https only (openai and generic). Azure OpenAI is excluded because its fine-tuning endpoint is a different path with a required api-version. A backend that does not serve the Fine Tuning API is excluded from the candidate set before selection, and a request that no capable backend can serve answers a structured 400 fine_tuning_not_supported, not a router-level 404.
• Backend selection: the closest existing shape is /v1/batches (#880), but that is Hub-gated provider batch dispatch with a router-side tracker (src/control_plane/batch/mod.rs). Fine-tuning takes the same lifecycle rules without the Hub gate. A create selects a backend from the request's model; list selects one capable backend for the caller; retrieve and cancel use a binding.

Job bindings¶

A create answer is buffered long enough to read the job id. The router then records (principal, job id) -> (backend, model) in a per-AppState LRU table, the same shape as the Responses passthrough table (services::responses::PassthroughRoutes, issue #1941). Retrieve and cancel look up that binding and re-check it against the live configuration and the caller's allowed_backends; an unknown or foreign id answers 404 fine_tuning_job_not_found and nothing is sent upstream.

GET /v1/fine_tuning/jobs applies the same ownership rule. The provider's list carries every job on the backend account, including jobs other principals created through this router on the same credential, so the router keeps only the entries whose id it bound to this caller, exactly as /v1/batches filters its provider list by owner. A job this router did not bind (created before a restart, or through another router process) is therefore not listed, which matches retrieve and cancel answering not-found for the same job.

The binding table is process memory. A job created before a router restart still exists on the backend, but this router no longer knows which backend to call, so it answers not-found. This is the open question below.

Training files¶

A job's training_file and validation_file ids must exist on the selected backend. The router's own /v1/files stores uploads on local disk (files.storage_path) and resolves them into inference requests; it does not upload them to a provider. A router file id therefore is not a provider training file, and this design adds no provider upload step.

Remaining open questions¶

• Job state persistence: how the router-side job state is stored and recovered across restarts. The in-memory binding enforces ownership for the process lifetime only. Persisting it, as the batch tracker does, would let retrieve and cancel survive a restart.
• Training-file upload: whether training files are uploaded through a provider-file passthrough or by mapping router file ids to provider uploads.
• Fine-tuned model listing: whether the fine-tuned-models listing merges into GET /v1/models or stays a separate passthrough route. It is not part of this implementation.
• List fan-out: whether GET /v1/fine_tuning/jobs should merge pages from every capable backend instead of selecting one. Merging would invent a normalized job schema, which the issue asks the router not to do.