Prompt Logging¶

Prompt logging records the text of requests and replies to local files and, optionally, to an external destination. It is off by default. This guide is for operators who decide whether to turn it on, and how to run it once it is on. The field-by-field reference is Prompt Logging Section; the design history is in Prompt and Request Logging: Design Record.

What leaves the box by default¶

With prompt_logging.enabled: false, or no prompt_logging section at all, no request path reads, stores, or exports a prompt or reply. Nothing about the feature runs: there is no writer, no capture queue, and no export task.

What the router records without it is metadata:

• Process logs and tracing spans: endpoint, backend, model, status, and latency. Credential headers are masked, and no prompt or reply text is written.
• The usage ledger: token counts, model, backend, outcome, and latency per request.
• The Hub: when the control plane is attached, it receives usage records (counts and attribution). The record schema has no field for text.

A few features hold content because serving the request requires it, and they are not request logs: the response cache, the semantic cache (which sends the last user message to the embedding model), and the Files and Batch APIs. Prompt logging does not change them.

Enable it¶

Four settings decide where and what is recorded:

• enabled: the master switch. false also overrides any key's allowed consent.
• directory: where record files are written. Default ./prompt-logs. The router creates it with mode 0700 when it is missing (an existing directory keeps its mode, and one that its group or everyone can write draws a warning at startup), and creates each file with mode 0600. Use a directory that belongs to one router process, because retention applies to every matching file in it.
• capture: which bodies a record holds. prompts, completions, and tool_calls each default to true. Turning all three off while enabled is true is a load error.
• Restart: every prompt_logging field is read at startup. A hot reload reports the section under requires_restart.

A minimal configuration that records prompts and completions but not tool calls, keeps files for 14 days, and masks one extra pattern:

prompt_logging:
  enabled: true
  directory: /var/lib/continuum-router/prompt-logs
  capture:
    prompts: true
    completions: true
    tool_calls: false
  redaction:
    patterns:
      - 'employee-[0-9]{6}'
  retention:
    max_files: 16
    max_age_days: 14

Check it before restarting:

continuum-router config validate config.yaml

The command prints "valid": true for this section. Startup fails if the directory cannot be created or written.

To stop recording, set enabled: false and restart. The router then runs no writer and no retention, so existing record files stay on disk until you delete them.

Consent¶

Consent decides which requests are recorded. enabled is checked first, then the request's key, then default_consent:

1. enabled: false: nothing is recorded, whatever a key states.
2. A key states denied: the request is not recorded, under either default.
3. A key states allowed: the request is recorded, even under default_consent: denied.
4. A key states nothing, or the request has no key: default_consent applies. allowed (the default) records the request, denied does not.

A record carries a consent field naming the grounds: key_allowed or deployment_default.

Where a key's consent comes from¶

A key states consent in one of two places:

• A locally configured key: prompt_logging_consent on the key, in api_keys.api_keys[], in an external keys file, or through the Admin API and the WebUI. It takes allowed or denied. A key without the field states nothing. No Hub is needed, so a deployment without one can run opt-in capture with default_consent: denied, or exclude one key under default_consent: allowed.
• A Hub-synced key: the key table a Continuum Hub syncs to the router. The Hub composes a team's consent into each key's value when it builds the table. The router has no team consent setting.

prompt_logging:
  enabled: true
  default_consent: denied
api_keys:
  api_keys:
    - key: "${RESEARCH_KEY}"
      id: research
      user_id: research
      organization_id: acme
      prompt_logging_consent: allowed

When a token is present in both tables, the values combine in this order:

1. denied from the local key or the Hub key.
2. A local allowed.
3. A Hub allowed.
4. default_consent.

The fail-closed window outranks all of them, a local allowed included.

A local value applies to a key that authenticates, in both permissive and blocking mode, on Authorization: Bearer and on x-api-key. A presented token that does not authenticate has no local value: blocking mode refuses the request unless the token is also a Hub key, and permissive mode serves it as a request with no key.

The remaining cases:

• A disabled or expired key stating denied: not captured in either mode, so disabling a key never widens consent.
• No Hub policy store, and a presented token that does not authenticate: not captured whatever default_consent says. A disabled or expired key stating allowed or nothing opts nothing in, and the old value of a deleted or rotated key, or any unknown token, is not captured either.
• A placeholder token to a permissive router: not captured. The request must carry no token to follow default_consent.
• A Hub policy store, and a token no table recognizes: not captured, and only the stated consent of a key a table admits applies.
• api_keys_file missing or unreadable, or persistence_file unreadable: the router logs a warning and starts anyway. Until a restart loads the file, no presented token that fails local validation is captured, a Hub key included. An unreadable api_keys_file also stops api_keys[] from loading. An unreadable persistence_file is left as it is, and runtime keys stay in memory until a restart.
• A request with no key: still follows default_consent.

Changes take effect as other api_keys changes do. A change to a file key through the file watcher needs a restart, and an Admin API change applies to the next admitted request. An Admin API change to a file key lasts until restart, when the key returns to the value in its file, so withdraw consent in the file as well. A request already in flight keeps the consent read when it was admitted.

Note: A Hub at or after continuum-hub#1994 delivers per-key consent through the key table. A Hub older than that delivers none, so a Hub-synced key follows default_consent. Under default_consent: denied and such a Hub, only a local key stating allowed is recorded.

Fail-closed window¶

A key's denied can be read only from a key table the Hub built knowing that this router reads consent. While no such table is in force, a key-authenticated request is not recorded, whatever default_consent says. A request with no key still follows default_consent, and a deployment with no Hub policy store is not affected. The cases:

• No policy yet: before the first sync, or after a restored key table expires, the store holds no table at all.
• Capability not yet stored by the Hub: the Hub delivers consent only to a router whose stored capabilities include prompt_logging_consent_v1, which the router advertises in its heartbeats. A table requested before the Hub accepted a heartbeat may have been built without the capability, so the router treats it as carrying no consent. The first heartbeat is sent at startup and the policy is requested again as soon as it is acknowledged, so this window normally closes within a second.
• Heartbeats never succeed: the window does not close, and the router logs one warn naming the consequence on the first failed heartbeat.
• Older Hub: a Hub before continuum-hub#1994 accepts the heartbeat and still sends no consent. The window closes on the next table and a Hub-synced key follows default_consent. The router logs one info line with the number of keys stating consent when the window first closes.
• Restored key table: a table restored from the key-table file (persist_key_table) carries consent only when the file was written from a table that did, recorded as consent_confirmed. A file written by an older router build restores as carrying no consent, and the window stays open until the next table from the Hub.
• unmatched_key: fail_open: a presented token that no table recognizes is not recorded. A key that local authentication accepted but the Hub table does not know follows its local value and default_consent, so a key present in both tables whose Hub row has not synced yet follows its local value for up to one poll interval.

The state is fixed when the request is admitted. A sync that lands while a long request runs does not turn that request into a recorded one.

What a record contains¶

Each request that is recorded yields one JSON line in a file named prompt-log-<sequence>.jsonl under directory:

{
  "schema_version": 2,
  "request_id": "req-0f3a",
  "created_at_ms": 1785000000000,
  "written_at_ms": 1785000002113,
  "model": "gpt-5.6-sol",
  "provider": "openai",
  "key_id": "team-a",
  "consent": "deployment_default",
  "prompt": "user: Summarize the attached report.",
  "completion": "The report covers ...",
  "usage": { "input_tokens": 412, "output_tokens": 96 },
  "outcome": "ok"
}

• Attribution: request_id, model (the client-facing name), provider (the backend that answered, absent when no backend was reached), and key_id (the key's id, never the key).
• Content: prompt, completion, and tool_calls, each present only when its capture scope is on and the request produced it. Non-text parts such as images are written as placeholders, and reasoning output is not recorded.
• outcome: ok, refused (a guardrail blocked the request or reply), backend_error, client_cancelled, or stream_errored. A non-ok outcome can carry a short error summary, which is also masked.
• truncated: names each field that does not hold everything. Fields are cut at 256 KiB, a stream that ended early stops where it ended, and masking keeps at most 1 MiB per field.

A streamed reply is written once, when the stream ends. A request that fails over writes one record, for the hop that answered.

Redaction¶

Every record is masked once, on the request task, before it is queued. Unmasked text never reaches the file, a log line, or an error. Masking has two layers, applied in this order:

• Built-in credential patterns: OpenAI and Anthropic style sk- keys, GitHub tokens (ghp_ and the related prefixes, github_pat_), AWS access key ids, Google AIza keys, Slack xox tokens, JWTs, PEM private-key blocks, and Bearer credentials.
• redaction.patterns: regular expressions you add. Each match is replaced with redaction.masked_replacement (default [redacted]), as literal text. A pattern that is invalid or matches the empty string is a load error, and the error names the entry by index, never the pattern.

Fields masked are model, prompt, completion, each tool call, and error.

Masking on export¶

Export masks again with the current configuration before it sends a record. A pattern added after capture therefore also covers older files. The masking covers the fields above plus request_id, provider, and key_id.

Warning: Redaction is pattern matching. It masks text that looks like the patterns and nothing else. Personal data, internal names, and a credential in a format no pattern covers are recorded as written. Treat the record files as sensitive, and review the patterns against your own traffic before you export anywhere.

Retention¶

Local files are bounded by three settings, and each must be at least 1: 0 is rejected, not read as unlimited.

Age is enforced by rolling the active file once its first record is one eighth of max_age_days old, then removing files early enough that no record outlives the bound. Traffic therefore produces about eight files per max_age_days, so set max_files to at least 9 for age to be the limit that applies. Below 9, the file count can remove a record before its age does. The timing rules are in Age Retention.

Retention runs only while the router does. Retention also does not wait for an export sink: a file removed before a sink read it is skipped and counted as pruned.

When the writer falls behind¶

A dedicated thread writes records, so a slow disk never delays a request. The request path hands each record to a queue of 256. When the queue is full, or a write fails (a full disk, a removed directory), the record is dropped and the request is not affected. Two counters report it:

• prompt_log_records_total{result="queue_full"}: the queue was full.
• prompt_log_records_total{result="sink_error"}: the write failed, and one content-free ERROR line is logged.

A dropped record is not retried and is not sent to any sink.

Export¶

With no sink configured, records stay in the local files until retention removes them. A sink delivers them elsewhere. Sinks run only while enabled is true, each on its own task and cursor, so one failing destination holds back only itself. Reference tables for every sink field are under Export Sinks.

prompt_logging:
  enabled: true
  export:
    sinks:
      - type: webhook
        endpoint: "https://logs.example.com/ingest"
        headers_env:
          Authorization: COLLECTOR_AUTH
      - type: langfuse
        endpoint: "https://langfuse.example.com"
      - type: s3
        bucket: prompt-logs
        prefix: router-1/
        region: us-east-1

Destinations¶

• webhook: a POST of a JSON array of records. Credentials come from headers_env, which maps a header name to an environment variable. It can name any variable the router process can read, and the value goes to the endpoint on every batch: whoever can edit this section, including through the Admin API, can read the router's environment through a collector they control. Keep that environment to what the router needs.
• langfuse: a POST to {endpoint}/api/public/otel/v1/traces as OTLP JSON, with Basic auth from public_key_env and secret_key_env. Each record becomes one generation span. Langfuse v4 does not reliably deduplicate a span it receives twice, so a redelivered record can appear twice there.
• s3: one JSONL object per batch, named <prefix>/prompt-log-<sequence>-<offset>.jsonl. It needs the prompt-log-s3 build feature, which official release binaries include. A binary built without it loads the configuration, warns that the sink is inert, and sends nothing.

Every credential is an environment variable name, read once at startup. A variable that is unset or empty stops the router from starting, and config validate warns about it first.

Delivery guarantees¶

Delivery is at least once. A sink keeps a cursor (file sequence and byte offset) in .export-cursor-<id>.json under directory, created with mode 0600, and moves it after a batch is delivered. A crash or restart can redeliver the batch that was in flight, so the receiver must tolerate duplicates.

Where a new sink starts:

• backfill: false (default): the sink starts at the end of the retained records and sends only what is written after it starts. Old records reach a new destination only by an explicit choice.
• backfill: true: the sink first sends every record retention still holds.

backfill has no effect once a cursor exists. Deleting a cursor file makes the sink new again. Changing a sink's destination, including a token in a webhook URL, also makes it a new sink with a new cursor, unless the sink sets cursor_id.

Keeping a cursor across a token rotation¶

cursor_id names the cursor file (.export-cursor-<type>-<cursor_id>.json) in place of a hash of the destination, so rotating a token in a webhook URL keeps the position and the undelivered records still go out.

• Format: 1 to 64 letters, digits, _, or -. Two sinks of one type cannot share a value, ignoring case; either is a load error.
• Adding it to an existing sink: the sink starts a new cursor once, following backfill. Records the old cursor had not reached are not sent unless backfill is true.

Warning: With cursor_id, the sink sends its backlog to whatever destination it names now, including a host that never received these records. Make sure the new destination may receive them before you change it.

When delivery fails:

• Delivery errors: the batch stays on disk and is retried with exponential backoff, from interval_seconds up to max_backoff_seconds. Each attempt counts as failed, and the first failure logs a warning.
• Permanent refusals: see Records the destination refuses.
• A record that does not parse: a line that is not a record, is too long, or is torn is skipped and counted as malformed. It does not block the sink.
• A cursor file that cannot be read (an I/O error such as EACCES): the sink retries every interval_seconds and sends nothing, and never jumps to the oldest file or the tail. A cursor file that holds no valid cursor is treated as no cursor and follows backfill.
• Startup: see A cursor that cannot be read at startup. A file that holds no valid cursor does not affect startup.

Records the destination refuses¶

By default (on_permanent_failure: block) any refused batch is retried, so a record the destination will never accept stops that sink until you fix the cause or retention removes the file. Spans Langfuse reports as rejected in partialSuccess are the exception: they are always counted as rejected and passed.

With on_permanent_failure: skip, a batch refused with an eligible status is given up on:

• Eligible: 400, 409, 413, 415, and 422. These statuses can describe the record or batch payload.
• Always retried: every other status, including authentication, proxy, path, routing, rate-limit, and server failures. Failures with no status, such as a timeout or refused connection, are also retried.
• S3 error codes: on an s3 sink, a 400 or 409 whose body names an S3 error code is retried unless the code is EntityTooLarge. S3 stores an object as opaque bytes, so errors such as an expired session token (ExpiredToken), a wrong region, a stalled upload, or OperationAborted do not blame the records.
• Accounting: what the destination accepted before the refusal stays delivered. The rest of the batch counts as dropped, including records the sink had not sent yet, and one warning names the sink, the status, and the count. The batch is the unit: the router cannot tell which record caused the refusal.

Warning: skip loses records, and a misbehaving destination can drop far more than one bad record. A gateway that answers a bad credential with 400 reads as a refusal, and the sink gives up on each batch with no backoff, so it can discard the backlog quickly. Alert on dropped.

A cursor that cannot be read at startup¶

The first record file number must be above every cursor, or a sink would read the new file as already consumed and skip new records. So the router does not write while a cursor cannot be read, and a lost observability path does not stop traffic:

1. The router reads the cursor again up to 5 times, 200 ms apart.
2. If it still fails, the router logs one ERROR naming the error kind and sets prompt_log_capture_disabled{reason="cursor_unreadable"} to 1.
3. The router serves requests with no capture and no export task until the next start.

To recover, fix the file and restart. A directory that cannot be read or created still stops startup.

Note: Retention continues while capture is disabled this way. A retention-only timer applies max_age_days and max_files without opening or creating a record file.

Consent withdrawal¶

A sink checks each record's key again before delivering it, so withdrawing consent stops future capture and any export that has not happened yet. A backfilled sink follows the same rule. Each record is decided by its key's consent at that moment:

During a hold the sink delivers what precedes the record, logs one info line, and tries again every interval_seconds. It logs one more line when the hold clears, and the backlog gauge keeps reporting.

The re-check has these limits:

• Delivered records: they cannot be recalled. One batch may be in flight when consent changes, because a batch is not checked again between the filter and the delivery.
• Local files: they are not rewritten. A withdrawn key's records stay on disk until retention removes them.
• Removing a key: a key removed from the local table and the Hub table counts as consent_key_absent, so its undelivered records are not sent. To withdraw a local key without removing it, state denied on it in its configuration or through the Admin API or the WebUI; that counts as consent_withdrawn.
• A key source that failed to load: a missing or unreadable api_keys_file or persistence_file may have held a key's denied, so every record with a key holds until the file is fixed and the router restarts. Requests are still served, and the backlog gauge keeps rising.
• Revoked Hub keys: removal is the only withdrawal by absence. A revoked, disabled, or expired Hub key stays in its table, so its stated consent still applies, and one that states nothing follows default_consent. To stop such a key's export, have the Hub state denied for it.
• AppProxy ROUTER keys: they state no consent, so while the router is enabled and its table holds the key, its records follow default_consent. A key removed from that table counts as consent_key_absent, and a record of such a key holds until the router has its first key snapshot. With the router disabled, these records count as consent_key_absent.
• Masked key ids: consent is decided on the key id as captured, before the redactor masks it, so a redaction.patterns entry or a built-in pattern such as sk- followed by 10 or more token characters cannot hide a key from the tables. The masked id is still the only one written to a record or sent on the wire. The Hub push is the exception: the Hub matches the id against its own api_keys table and accepts only [A-Za-z0-9._-]{1,128}, so a record whose id masking changed is refused locally, counted as rejected without a POST, and the cursor passes it. One warn per process, without the id, says that a redaction pattern matches a Hub key id, so that key's records cannot be pushed. If a key's records are skipped unexpectedly, check whether a redaction pattern matches its id.

Warning: Export does not re-check capture scope. A backlog written under a wider scope still delivers until retention removes it, after you narrow capture or on a backfilled sink. To stop that delivery, remove the sink from export.sinks.

Network rules¶

• No redirects: a 3xx answer is a failure and is never followed, so content cannot move to another host.
• Private addresses: an endpoint on a private, loopback, link-local, carrier-grade NAT (100.64.0.0/10), benchmarking, multicast, or metadata address is refused before every delivery. The check covers plain, IPv4-mapped, NAT64, 6to4, and Teredo forms. Set allow_private_endpoint: true for a collector inside your network.
• Name resolution: the endpoint's hostname is resolved before every delivery, and a name that does not resolve fails the delivery. The connection goes only to the addresses that passed the check, so a name whose answer changes afterwards cannot move the delivery elsewhere.
• S3 environment endpoints: AWS_ENDPOINT_URL_STS, AWS_METADATA_ENDPOINT, and AWS_CONTAINER_CREDENTIALS_FULL_URI get the same check before each delivery and share the checked-address table with the S3 destination. A hostname that cannot resolve no longer stops router startup; the sink stalls with failed rising until resolution recovers. The fixed ECS and EKS Pod Identity hosts 169.254.170.2, 169.254.170.23, and [fd00:ec2::23] are exempt. Other private or loopback credential hosts require allow_private_endpoint: true, which waives their address check.
• S3 cursor identity: a change to AWS_ENDPOINT_URL_STS or AWS_METADATA_ENDPOINT gives the sink a new cursor. AWS_CONTAINER_CREDENTIALS_FULL_URI is a credential source rather than a destination, so it does not change the cursor.
• S3 destination variables: a change to AWS_ENDPOINT, AWS_ENDPOINT_URL, or AWS_ENDPOINT_URL_S3 also gives the sink a new cursor.
• Egress proxies: under HTTP_PROXY or HTTPS_PROXY the proxy resolves the destination and S3 environment endpoints and chooses their addresses, so the address restriction above does not apply to those connections. Enforce egress policy at the proxy. The checks still run first, so the router itself must be able to resolve each name or the sink stalls with failed rising. allow_private_endpoint: true waives the address checks where the router cannot resolve them.
• Clear text: an http:// endpoint on a host that does not look private draws a warning from config validate and at startup, because content and credentials would travel unencrypted. The sink still runs. Use https://.

Hub push¶

export.hub_push: true sends captured records to the Continuum Hub the router is enrolled with, at POST /api/agent/v1/prompt-logs, with the enrollment credential. It is the one channel on which content reaches a Hub; usage records stay metadata-only. It needs a binary with the control-plane feature and control_plane.policy.enabled: true, because consent is read from the Hub's key table. Otherwise the setting logs a warning and sends nothing.

prompt_logging:
  enabled: true
  export:
    hub_push: true
control_plane:
  enabled: true
  hub_url: "https://hub.example.com"
  enrollment_token: "${CONTINUUM_HUB_ENROLLMENT_TOKEN}"
  policy:
    enabled: true

The push is a further task of the export loop: its own cursor (.export-cursor-hub-<hash>.json, the hash covering the Hub URL and the router id), the same masking on read, metrics under sink="hub", and at-least-once delivery. It differs from the sinks in these ways:

• Consent: a record leaves only when its key is in the Hub's key table with allowed, the record was captured on that allowed rather than on default_consent, and no local denied outranks it. A keyless record, a record captured on the deployment default, a key only the local table knows, and a key stating nothing never leave; they count as consent_keyless, consent_deployment_default, consent_key_absent, and consent_withdrawn. A record whose key cannot be read yet holds the push, as it holds a sink.
• Start: a router with no cursor starts at the end of the record files, and there is no backfill. A re-enrollment or another Hub starts at its own end.
• Shape: a record carries request_id, key_id, completed_at_ms, the masked prompt and completion cut to 64 KiB each, and the idempotency key <router_id>:<request_id>. Tool calls, usage, the model, and the outcome do not leave. A request holds at most the Hub's prompt_log_batch_max_records records (default 100) and 16 MiB of encoded JSON, and the push runs every usage_push_interval_secs.
• Hub answers: a 404 drops the backlog only when its body is the ingest handler's own {"error":"prompt-log ingest disabled"}. Any other 404, such as one from a proxy or a mistyped path, is retried, so the records stay on disk. On a closed channel the cursor moves to the end of the files and the records it jumps over count as dropped, an upper bound that includes lines the strict consent filter would have skipped. 413 drops that request whole. 400, 403, and 409 are re-sent one record at a time, with a short pause between the single-record requests, so one withdrawn key refuses only its own record. A batch refused whole pauses the push for one interval. 401, 408, 429, a 5xx, and a transport failure retry the same records after a backoff of at most 60 seconds. A 3xx is not followed: it retries, and one warning per process names the status and says hub_url must name the final location. Any other 4xx counts as rejected. The Hub deduplicates by idempotency key, so a redelivery after a crash adds no row; records the Hub reports as already stored count as duplicate.
• Operator limits: a closed channel discards the whole backlog for good, so turn ingest on at the Hub before enabling the push if old content should be kept out. A router the Hub has retired has every record refused one by one, each counted as rejected, until it is removed or re-enrolled; the pause after a wholly refused batch limits how quickly that backlog is lost but does not prevent it. A team's denied is not a router setting: it reaches the router only through the Hub's key table, folded into each key's consent, so a Hub that predates continuum-hub#1994 never delivers it.
• Clear text: an http:// hub_url on a non-loopback host is refused when the configuration loads, because prompt content and credentials would travel unencrypted. Use https://.
• Local files: a pushed record is not removed from the record files. retention bounds them for every sink alike.

Which requests are recorded¶

A request is also not recorded in these cases:

• Response-cache hit: the cached reply is served without a record.
• Refused before dispatch by validation or request policy: no record is written.
• Not admitted by consent: see Consent.
• A buffered request the client cancelled while the router read an internal stream.

A request that an input guardrail blocks is recorded with outcome refused, with its prompt and no reply.

Not available yet¶

• Hub viewing: the Hub stores pushed records (continuum-hub#1622) but does not display them yet (continuum-hub#1623).
• Per-key consent from an older Hub: a Hub older than continuum-hub#1994 delivers none, so a Hub-synced key follows default_consent. See Where a key's consent comes from.

Metrics and troubleshooting¶

sink is the sink type, with -1, -2, and so on for a further sink of the same type, or hub for the Hub push.