Telemetry

The Telemetry page stores named, account-scoped OpenTelemetry Protocol (OTLP) exporter settings and privacy options. Open the page under Observability to create or update a configuration. Use separate configurations to assign different telemetry destinations to different agents.

Configuration fields

• Enable telemetry: Starts an OTLP exporter and emits PinionAI runtime spans for this agent.
• Configuration name: A unique name within the account, used to select and manage multiple configurations.
• Provider: Identifies the destination as a local collector, Honeycomb, Datadog, New Relic, Arize Phoenix Cloud, or another OTLP provider. This is a label/preset only; it does not automatically configure the endpoint or protocol.
• OTLP endpoint: The HTTP(S) collector or managed-provider endpoint. Do not include credentials in the URL.
• OTLP protocol: Choose grpc or http/protobuf to match the configured exporter and receiver.
• API Key connector: An optional active API Key connector for managed-provider authentication. The telemetry row stores the connector name, not the secret. Create the connector on the Connectors page; see Connectors.
• Agent(s): Select agents that should use this telemetry configuration. An empty selection means the configuration is account-wide. When creating a new record, the currently selected agent is preselected when possible.
• Privacy options: Toggle credit-card-like value scrubbing and explicitly labeled name scrubbing. Name scrubbing handles labeled forms such as name: Jane Doe; it is not general-purpose name recognition.

Speech-to-text and text-to-speech operations emit pinionai.audio_call spans. When an audio provider returns token usage metadata, token counts and configured per-million-token cost attributes are attached to the active span. The runtime does not infer tokens from audio duration or transcript length.

Example: self-hosted collector

For a gRPC collector reachable from the application:

Field	Example
Configuration name	local-development
Enable telemetry	On
OTLP endpoint	http://otel-collector:4317
OTLP protocol	grpc
API Key connector	None
Agent(s)	Empty for all agents, or select specific agents
Service name	pinionai-production

Other local development receivers include Jaeger with its OTLP receiver enabled (http://jaeger:4317, grpc) and Phoenix (http://phoenix:6006, http/protobuf). Use a hostname reachable from the PinionAI runtime. In Docker Compose, localhost refers to the PinionAI container itself, not another container or the host.

Managed provider examples

These are starting points, not universal account settings. Endpoints may vary by region, account, and product tier; use your provider's current OTLP setup instructions when they differ.

Provider	Endpoint example	Protocol	Connector auth
Honeycomb US	https://api.honeycomb.io:443	grpc	Header x-honeycomb-team; scheme none; Honeycomb API key
Honeycomb EU	https://api.eu1.honeycomb.io:443	grpc	Header x-honeycomb-team; scheme none; Honeycomb API key
New Relic US	https://otlp.nr-data.net:4318	http/protobuf	Header api-key; scheme none; New Relic license key
New Relic EU	https://otlp.eu01.nr-data.net:4318	http/protobuf	Header api-key; scheme none; New Relic license key
Datadog direct trace intake	Site-specific OTLP traces endpoint	http/protobuf only	Header dd-api-key; scheme none; Datadog API key
Grafana Cloud	OTLP endpoint from the stack's OpenTelemetry setup	As provided by Grafana	Header Authorization; scheme Basic; base64-encoded instance-id:token
Phoenix Cloud	Collector endpoint from Phoenix instance settings	As provided by Phoenix	Configure the documented auth header; see single-header limitation below

For Datadog direct trace intake, use the endpoint and site-specific host from Datadog's instructions. Its direct trace endpoint supports HTTP/protobuf, not gRPC. A Datadog Agent or Collector can be used as an alternative receiver.

An API Key connector stores the credential separately from the OTLP endpoint. For each service, set the connector form fields as follows:

Connector form field	Value
Connector Type	API Key
API Key Name	A descriptive name such as honeycomb_otel
API Key	The provider key/token (for Grafana, see below)
Auth header	The provider-specific header from the table
Auth scheme	none for raw-key headers, or the provider-required scheme
Connector URL	Not shown for API Key connectors; configure the endpoint in Telemetry
Client ID, Grant Type, Content Type	Not used by OTLP API-key authentication
Agent(s)	Restrict the connector to the agents that require it

In the Telemetry form, select the API Key connector and set the endpoint and protocol shown above. The runtime applies /v1/traces to HTTP/protobuf base endpoints when it is missing.

For Grafana Cloud, follow the stack's setup instructions to create the Basic credential (instance-id:token, base64-encoded). Store that encoded value in the connector's API Key field. The connector adds the literal Basic prefix; it does not encode the value.

The connector supports one authentication header. If Phoenix Cloud (or another backend configuration) requires multiple headers, route through an OpenTelemetry Collector or proxy that adds them, and point PinionAI to that collector. A local Phoenix instance commonly requires no authentication.

See the providers' current documentation for account-specific endpoints and auth requirements: OpenTelemetry Collector quick start, Honeycomb OTLP, Datadog OTLP traces, New Relic OTLP, Grafana Cloud OTLP, and Phoenix tracing setup.

Example: Honeycomb API Key connector

1. On Connectors, choose Connector Type: API Key and set API Key Name to honeycomb_otel.
2. Paste the Honeycomb key into API Key, set Auth header to x-honeycomb-team, and set Auth scheme to none.
3. On Telemetry, set the endpoint to https://api.honeycomb.io:443 (US) or https://api.eu1.honeycomb.io:443 (EU), choose grpc, and select honeycomb_otel.
4. Choose the agents that should use this configuration, or leave the selection empty for account-wide scope.

Do not include API keys in the endpoint URL, service name, or agent selection. Keep keys in connectors and follow the provider's current documentation for endpoint paths, headers, and OTLP protocol support.

Runtime behavior and privacy

The PinionAI client creates a separate OpenTelemetry provider for each client instance and starts it from the agent's telemetry object. Disabled or missing configuration creates no exporter. Client shutdown flushes and closes the provider; changing agents or refreshed configuration replaces it. Telemetry initialization and export failures are logged without including connector secrets and do not prevent agent execution.

Install the client package with pinionai[telemetry] to enable exporter support. The Streamlit application includes the OTLP exporter dependencies in its requirements.

An API Key connector is resolved through the virtualized connector endpoint. Its configured auth header and scheme are used to create OTLP headers; resolved secrets are held only in client memory and are not added to spans or logged. For HTTP/protobuf, a /v1/traces path is added when the configured endpoint does not already end with that path.

Initial instrumentation creates spans for user turns, prompt execution, and model calls, with operational metadata such as agent, prompt, provider, model, and reported token counts. Model-call spans include gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and cache token counts when reported by the provider. When the matching model configuration has complete prices, spans also include the estimated per-call cost as pinionai.usage.estimated_cost_usd, with per-category attributes for input, output, cache creation, and cache read. These are estimates based on the configured USD-per-million-token rates, not provider invoices. The model's existing *CostPerToken JSON fields are retained for compatibility but now carry per-million-token rates. Prompt text, completions, user input, tool arguments, and credentials are not captured. The scrubbing switches apply to string attributes before they are recorded: credit-card candidates are redacted only when they pass a Luhn check, and names are redacted in explicitly labeled fields such as name: Jane Doe. This is not general-purpose name recognition and does not enable content capture.

Agent variables marked redact: true are also masked as *** in PinionAI runtime log messages, telemetry attributes, and the current_var_data reporting property. The client exposes redact_output() to create a redacted copy of other application reporting data. Live variables and the normal session update payload retain their original values for agent execution and persistence.

The API must select the intended telemetry row for the active agent before returning the single telemetry object. Prefer an exact agent assignment over All, then account-wide configuration, with a deterministic tie-breaker. The runtime cannot resolve ambiguity after the API has reduced the result to one object.