← Back to blog

OpenTelemetry GenAI Spans Can Turn Prompts Into Persistent Logs

OpenTelemetry’s GenAI conventions include configurable capture of model inputs and outputs. Treat prompt content as sensitive data, not routine observability metadata.

An OpenTelemetry trace can become a second copy of every prompt, document excerpt, or model response your application handles. That is useful for debugging, but it can also move sensitive content into collectors, exporters, and retention systems that were designed for operational metadata.

The OpenTelemetry GenAI semantic conventions define spans, events, and metrics for model and agent operations. The conventions are currently marked Development, and their event guidance says instrumentations may capture user inputs and model responses when content capture is configured. The design decision is yours: a trace can describe an inference without storing its payload.

Separate operation metadata from content

A useful inference span can record the operation, model name, provider, duration, status, token counts, and a low-cardinality error type. Those fields help answer operational questions: which model is slow, how often requests fail, or whether token use changed after a deployment.

Prompt and response bodies answer a different question. They may contain account details, personal information, credentials pasted by a user, health data, customer documents, or internal instructions. Once captured, that content can be copied into an SDK buffer, collector queue, trace backend, search index, backup, or incident screenshot. Each destination becomes another place to control access and deletion.

The GenAI conventions make content capture configurable; examples include event-only capture and capture on both spans and events. Do not confuse that switch with a complete privacy control. An instrumentation library may add attributes, and application code or an exporter may independently attach request and response data.

Make no-content the production baseline

Keep prompt and response content out of routine production telemetry. Emit metadata that supports service health and cost analysis, and use prompt templates or version labels instead of the rendered prompt. Avoid recording tool arguments, retrieved passages, user identifiers, or conversation history as “debug context.”

If a production investigation genuinely requires content, treat it as a temporary, purpose-limited diagnostic mode. Require an explicit change, narrow the affected service and time window, restrict access to named responders, and set a short deletion deadline before collection begins. Do not enable content capture globally because it is convenient during a live incident.

Sampling is not a privacy boundary: the sampled trace still contains the full payload. Likewise, replacing a name with a hash does not make a prompt anonymous when the text or surrounding attributes can identify the person. The safer choice is to avoid collecting content that is not necessary to resolve the operational question.

Filter at the source and at the collector

Use instrumentation settings to prevent sensitive content from being emitted in the first place. Add collector-side filtering or redaction as defense in depth, before telemetry reaches external exporters or long-term storage. The OpenTelemetry security guidance describes attribute, filter, redaction, and transform processors, but post-processing cannot undo exposure in an earlier buffer or exporter.

Test the actual exported payload, not just the dashboard view. Send synthetic prompts containing canary strings, inspect spans and events at the collector boundary, and assert that prompts, responses, secrets, and personal fields are absent. Repeat the test after instrumentation upgrades: semantic conventions and library defaults can evolve, and a newly added attribute can widen the data flow.

Finally, give trace data its own retention and access policy. Observability systems often have broader engineering access and longer retention than the application database. Keep debugging metadata lean, separate any approved diagnostic stream from routine telemetry, and make deletion behavior explicit for queues, indexes, and backups.

Sources

Comments

Loading comments…