Skip to content

Prompt and Request Logging: Design Record

This note records what Continuum Router records about request and reply content, and the design of the opt-in prompt-logging capability (epic #2294, issues #2301, #2309, #2324, #2328, #2333). It stays as the design record and describes what shipped. To enable and operate the feature, read the operator guide, Prompt Logging.

With the capability off, which is the default, no log line, trace span, or usage record carries prompt or reply text.

What the router records today

  • Process logs and tracing spans: metadata only. A log line names the endpoint, the backend, the model, the status, and the latency. The logging middleware redacts a denylist of credential headers (authorization, x-api-key, x-goog-api-key, proxy-authorization, cookies, and WebSocket subprotocol tokens) at trace level, and prompt or reply text is never written to a log line.
  • The usage ledger: per-request token counts, the model, the backend, the outcome, and latency. The write-ahead attempt log records the same attribution facts for crash recovery. No prompt or reply text is recorded.
  • Refusal logs: one rate-limited line per log site, refusal reason, and model. No content.
  • The Hub (control plane): receives the usage records the ledger pushes: counts and attribution, never text. The record schema has no field that can hold content.

Some features store or send content because serving the request needs it. They are opt-in and are not request logs:

  • Response cache: stores cached replies in memory, or in Redis or Valkey when response_cache.backend is redis or tiered.
  • Semantic cache: sends the last user message to the configured embedding model.
  • Files and Batch APIs: store uploaded files and batch inputs and outputs until they are deleted or expire.

Still open

Capture, consent, redaction, retention, and export have shipped. These parts have not, and no router setting stands in for them:

Design and what shipped

  • Config surface (#2301, shipped): an opt-in prompt_logging section. Absent or disabled is the default, and the behavior is then exactly the metadata-only recording described above. Every retention bound must be at least 1, so retention is always finite.
  • Capture (#2320, shipped): the request path passes each captured record through the redaction pass (#2309) before anything is written, and fails open: a capture error never fails the request. It covers POST /v1/chat/completions (the A2A bridge included), POST /v1/responses, and POST /anthropic/v1/messages. /v1/completions, embeddings, and ACP are not captured, and a response-cache hit writes no record. The design allowed consent per API key or team. Only per-key consent exists: the router reads KeyEntry.prompt_logging_consent, advertises prompt_logging_consent_v1, and keeps the value in the persisted key table, and no prompt_logging field sets a team's consent.
  • Consent gate (#2324, shipped): capture calls the gate for each request. enabled: false captures nothing, a key's allowed included. Otherwise denied is never captured, allowed is captured even under default_consent: denied, and a key that states nothing, or a request with no key, follows default_consent. While a Hub policy store is attached but holds no policy (before the first sync, or after a restored key table expires), a key-authenticated request is not captured whatever default_consent says, because its consent cannot be known. A keyless request still follows default_consent. A deployment with no Hub policy store is outside this window, and there a presented token that fails local validation is not captured whatever default_consent says (#2879), so an unknown, deleted, rotated, disabled, or expired token never follows it. A locally configured key states consent through api_keys[].prompt_logging_consent (#2819), and a key that states nothing follows default_consent.
  • Record schema and writer (#2309, #2320, shipped): a size-rotated JSONL file under directory, written by one dedicated thread behind a queue of 256 records. The schema carries the attribution facts the usage ledger records, plus the captured prompt and reply text after redaction. Schema version 2 adds an optional consent field (key_allowed or deployment_default) and, additively, a truncated list. A request the consent gate refuses writes no record. A request an input guardrail refuses writes a refused record with the prompt and no reply.
  • Retention (#2301, shipped): the section bounds the local files by size, file count, and age. Age is held by rolling the active file at one eighth of max_age_days and removing files early, so max_files needs to be at least 9 for age to apply.
  • Export (#2328, shipped): captured records leave the box only through a configured sink (webhook, Langfuse over its OTLP endpoint, or S3). No default sink exists; with none configured, records stay in the local rotated files until retention prunes them. Each sink tails the files on its own persisted cursor, masks every record again with the configured patterns, holds one bounded batch in memory, and leaves a failed batch on disk to retry with backoff. All three sinks refuse an endpoint on a private, loopback, link-local, or metadata address unless the operator allows a private collector, and none follows a redirect. A new sink exports only what is written after it starts unless it sets backfill: true. Delivery is at least once. A webhook or S3 refusal is retried and blocks that sink until it is fixed, while Langfuse spans reported as rejected are skipped. Details are in Export Sinks.
  • Hub push (#2764, shipped): export.hub_push runs one more task of the export loop against POST /api/agent/v1/prompt-logs with the enrollment credential, in a control-plane build with control_plane.policy.enabled. The design gated the push on the record's consent field; what shipped is stricter: a record leaves only when its key is in the Hub's own key table with allowed, it was captured on that allowed rather than on default_consent, and no local denied outranks it, because the Hub refuses a batch whole for any key its table does not allow and a local operator's allowed is not tenant consent. A new cursor starts at the end of the files with no backfill, each side is cut to 64 KiB, a request holds at most the enrollment's prompt_log_batch_max_records and 16 MiB of encoded JSON, a 404 drops the backlog to the tail, a 413 drops the request, 400, 403, and 409 are re-sent per record, and the rest retries with backoff. The design's local-retention rule (the files as a delivery buffer only, removed once pushed) is not implemented: the files are shared by every sink's cursor and retention bounds them. Details are in Hub push.
  • Hub viewing (planned): the Hub stores pushed records apart from usage records (continuum-hub#1622, shipped) and will show them in a per-request view that is redacted by default, with an audited reveal (continuum-hub#1623). Usage records stay metadata-only.