Skip to content

Voice

PinionAI answers phone calls with an agent and connects callers to people, over Telnyx Call Control. It is a service of its own, python -m pinion_voice (container pinionai-voice, 127.0.0.1:8096), and configured in the Studio's Data menu, on the Voice page, with every call on the Calls page.

No audio is ever recorded. Calls between people are never transcribed. While the agent is on a call, Telnyx's Conversation Relay turns the caller's speech into text and the agent's text into speech; that text is the agent's session, kept like a web or SMS chat. Telnyx keeps no copy (relay webhooks carry no transcript, and no AI conversation is stored). The service refuses, in code, any command or parameter that would record, fork, stream or transcribe a call (pinion_voice/telnyx_port.py).

Numbers, ring groups, members

  • A number (E.164, e.g. +19018443774) is on a Telnyx Call Control application whose webhook is https://<api host>/voice/telnyx/webhook. It names:
  • the answering agent and its greeting (English and Spanish);
  • the voices (Telnyx voice ids such as Telnyx.Ultra.<id> or Telnyx.KokoroTTS.af; AWS Polly voices are silent on Conversation Relay);
  • the ring group a caller asking for a person reaches;
  • the fallback number;
  • the Telnyx API Key connector (else TELNYX_API_KEY) and webhook public key (else TELNYX_PUBLIC_KEY);
  • a context API (an API row run when a call starts, to give the agent's tools what they need);
  • limits.
  • A ring group has members in priority order, a strategy (all at once, or one after another), how long each phone rings, how long a caller waits before the agent takes a callback request, its hours (time zone, weekly ranges, closures, overrides) and the whispers a member hears before pressing 1. A group may instead connect on answer: the first phone that answers is bridged at once, with no whisper and nothing to press — for a staffed store line (a voicemail would be connected too, so only for lines people answer within the group's hours).
  • A member is a person's own phone, with the languages they speak and optional hours of their own. A member takes a call by pressing 1, so a personal voicemail or call screening never takes it. Only verified phones ring.

How a call is handled

  • Webhooks. Each Telnyx webhook is verified (Ed25519, five-minute window) and stored once in voice_call_events by its Telnyx event id, so Telnyx retries are dropped. It then goes to the call's own worker, which handles a call's events one at a time.
  • The state machine. The call's machine (pinion_voice/machines/, pure functions) decides the next state and the commands. The decision, the leg's state (legs only move forward: initiated, answered, bridged, hangup), a timeline entry and the commands are written in one transaction (voice_command_outbox), then the commands are sent with a deterministic command_id (Telnyx ignores repeats within 60 s).
  • Recovery. At start, the service replays webhooks stored but not handled. Every minute, a reconciler closes calls whose end never arrived (outcome lost).
  • Test numbers. Calls from 555-01xx numbers (+1NXX55501xx) use the simulator, never Telnyx, so checks can post signed webhooks without placing calls.

The agent on a call

A number with an answering agent is answered at once, and the agent's session starts while the call connects (the number's context API, if set, is told the call and the session and may answer with variables for the agent's tools). When the call is up, Telnyx's Conversation Relay connects the caller to the agent at wss://<api host>/voice/relay/<call>/<n>?k=<token> (one token per relay, signed with VOICE_RELAY_SECRET); the relay speaks the number's greeting, then each thing the caller says is a turn of the agent, in the session's variables channel=voice, phone_number, locale, business_open, next_open_at.

  • Speech. Replies are spoken as one short paragraph (no links, lists or code; at most 450 characters). Telnyx can split one utterance in two, so prompts are joined for a moment before the turn starts; words said during a turn join the next. A slow turn gets "One moment." at 1.2 s and another filler at 9 s; at 20 s the caller is asked to say it again.
  • Speech recognition is Telnyx's default unless the number's relay settings name an engine per language, e.g. "speech": {"en": {"transcription_engine": "Deepgram", "transcription_engine_config": {"transcription_model": "deepgram/flux"}}} (Flux ends the caller's turn sooner).
  • Keys. 0 a person, 2 Spanish (saying "español" works too), * repeats the last answer.
  • Silence. After 15 s: "Are you still there?"; 15 s later the agent says goodbye and hangs up. Past the number's agent minutes (limits, default 20) the agent also says goodbye.
  • When the agent can't take the call (it could not start, a turn stuck twice, the relay ended unasked, or the number is on its agent_calls limit) the call goes to the number's fallback. A caller over per_caller_hour is rejected.

Transfer to people

Asking the agent for a person (its transfer intent), or pressing 0, reaches the number's ring group when the group is open and has an eligible member (active, phone verified, not paused, within their own hours, not on another call, not the caller; language match first, then priority, then whoever was offered a call longest ago).

  1. The relay stops and the caller hears a ringback tone (/voice/audio/ringback.wav). If the relay's end is never reported (a failed stop, a lost webhook) the phones ring anyway after 5 s.
  2. Members' phones ring from the number (all at once, up to 5, or one after another). Each member hears the group's whisper and presses 1 to take the call or 2 to skip. A personal voicemail or call screening never presses 1. A group that connects on answer skips the whisper: the first phone that answers is bridged.
  3. The first 1 is bridged to the caller; the other phones are hung up, and a later 1 hears "This call was already answered." A bridge that is not up within 8 s is given up: the member's leg is hung up and the caller is back with the agent, as in 4.
  4. If nobody takes it (every phone ended, or the group's max wait passed) the caller is back with the agent on a new relay, which runs the number's callback intent (callback_cause = no_answer). A closed group, or nobody free, runs it at once (closed, no_member).

The transfer is an engagement (queue voice: Waiting, Accepted, Completed or Missed), written before the agent's session is saved, so the Studio's live-agent queue never offers a phone call as a chat.

Click-to-call

A page asks the number to call someone back now (the account API below, usually through an agent: an event route whose intent has an API step to /voice/v1/calls). The customer's phone rings first, from the number; no one else's phone rings until they press 1.

  1. When the call is due, the number is dialed to the customer and the call's agent session starts.
  2. When they answer they hear the agent's voice call confirm intent (a fixed reply; variables call_title, call_name), e.g. "This is City Auto, calling about your request for the 2023 RAV4. Press 1 to talk to a specialist now." Without that intent (or if it takes over 2 s) a built-in sentence is used.
  3. 1: the ring group's members ring with the group's click_to_call whisper (name, title); the first 1 is bridged; nobody free or nobody takes it: the agent takes a callback request, as on an inbound call.
  4. No 1 (voicemail, someone who did not ask) or no answer: the agent's voice call later words, and one more try 20 minutes later (or at the group's next opening + 2 minutes). Outcome customer_no_answer.

A request when the group is closed, or nobody is free, is scheduled for the next opening + 2 minutes. Requests are refused ({"status": "rejected", "error": ...}) for a non-US number (not_us; Canada, the Caribbean, toll-free and premium numbers included), a blocked number, a number with a call already under way (busy), over the number's limits (limit: click_to_call_destination a day, default 50; click_to_call_ip_hour, 50; click_to_call_day, 50), or without consent (consent: {text, version} and whatever else the page records, e.g. ip, user agent, page).

Staff dial

A specialist presses "Call" beside a customer's number in their own system (City Auto's Ops Console). Their own phone rings first, from the number; the customer's phone rings only after they pressed 1.

  1. POST /voice/v1/dial names the member by member_ref (how the calling system knows the person, e.g. the staff e-mail: the member's Reference on the Voice page). Only an active, phone-verified member is dialed, so two arbitrary numbers are never bridged.
  2. The member's phone answers and hears "Press 1 to call Ana Lopez." (a voicemail never presses 1). A member whose "Staff dial asks 'press 1' first" is off is told "Calling Ana Lopez now." instead.
  3. The customer's phone rings while the member hears ringback; when it answers the two are bridged.
  4. No answer: the member hears "Ana Lopez did not answer. Goodbye." Outcomes: connected, declined (no 1), customer_no_answer, staff_no_answer, canceled (the member hung up first), failed.

A dial is refused ({"status": "rejected", "error": ...}) for no_member, not_us, blocked, busy (the member is on a call, or that customer is being called), hours (staff dials happen between 08:00 and 21:00 in the customer's own time: customer_tz, else the number's ring group's zone, else America/Chicago) and limit (staff_dial_day, default 200). The call's end is voice.call.ended with kind: staff_dial and member {name, external_ref}; the Studio's chat queue never sees these calls (no engagement row).

Verifying a member's phone

On the Voice page, Verify by call rings the member's phone from one of the account's numbers: "This is the phone system for . Press 1 to confirm this phone can take calls." Pressing 1 sets the member's verified time; a voicemail or a wrong number never does. The call is kind verify, placed by the scheduler within 15 s, and emits no event.

Forgetting a number

POST /voice/v1/forget {phone}: the account's ended calls with that number lose the number, their context, consent and state; their legs lose the numbers and their timeline its details. The rows stay (counts and costs). A call still going keeps its number until it ends. City Auto calls this from its own forget when a customer's account is deleted.

Account API

Bearer token as for the PinionAI API (a tokens row of the account, not expired).

  • POST /voice/v1/calls {number, to, name, title, locale, consent, idempotency_key, external_ref} → 201 {call_uid, status: "calling"|"scheduled", scheduled_for}; the same idempotency_key again → 200 with the same call.
  • GET /voice/v1/calls/{uid} → {status: scheduled|calling|ended, outcome, scheduled_for, attempts, external_ref}.
  • GET /voice/v1/availability?number= → {open, next_open_at, people}.
  • POST /voice/v1/dial {number, member_ref, to, name, title, locale, customer_tz, idempotency_key, external_ref, requested_by} → 201 {call_uid, status: "calling", member}, or 200 {status: "rejected", error}.
  • POST /voice/v1/forget {phone} → {forgotten: {calls, legs, events}}.

Each answer to POST /voice/v1/calls and POST /voice/v1/dial is also the event voice.call.requested (request_id = external_ref, status, call_uid, scheduled_for, or error), and a retry is one too (retry_of).

Events

Into the account's event inbox (routes send them on):

  • voice.callback.requested (once per call): call_uid, from, to, locale, session_uid, assistant_session_id, name, reason, time, cause. The agent's callback intent fills callback_name, callback_reason, callback_time.
  • voice.call.ended (once per call; not for verification calls): the above ids plus kind, outcome (completed, connected, queue_abandoned, forwarded, abandoned, rejected, lost; a click-to-call's customer_no_answer; a staff dial's declined, staff_no_answer, canceled, failed), the times, agent_seconds, talk_seconds, member {name, external_ref}, callback_requested, external_ref and transcript (the agent's part of the call as text).

Phases.

  • 4.15: a number answers with its greeting and hangs up.
  • 4.16: the agent answers over Conversation Relay; transfer to the ring group and back to the agent for a callback request; call events.
  • 4.17: click-to-call and the account API.
  • 4.18: staff dial (the staff phone first), phone verification by a call, forget by number.

When the service is down

A Call Control number cannot use Telnyx's own call forwarding (error 10015). Instead, the application's failover webhook URL points at https://<api host>/voice-failover/telnyx/webhook. That is a separate tiny service, python -m pinion_voice fallback (container pinionai-voice-fallback, 127.0.0.1:8097), which transfers each new inbound call to the number's fallback (VOICE_FALLBACK_E164 when the database is unavailable). Telnyx uses it when the main webhook fails or answers with an error.

Setup

  1. Run db/0.4.15_voice.sql.
  2. Create a Telnyx outbound voice profile (US only, a rate cap, a daily spend limit, recording off) and a Call Control application:
  3. webhook https://<api host>/voice/telnyx/webhook;
  4. failover https://<api host>/voice-failover/telnyx/webhook;
  5. first-command timeout on, call cost in webhooks.
  6. Put the number on the application.
  7. Proxy /voice/ to 127.0.0.1:8096 (with WebSocket upgrade) and /voice-failover/ to 127.0.0.1:8097.
  8. Run the two containers from the Studio image. They need the Studio's database environment and host_url plus TELNYX_API_KEY, TELNYX_PUBLIC_KEY, VOICE_FALLBACK_E164, VOICE_PUBLIC_URL (https://<api host>/voice) and VOICE_RELAY_SECRET (random, e.g. openssl rand -hex 32).
  9. Add the number on the Voice page.

NOTE: The Voice configuration is not part of the Import page capabilities; it is account-level, and not tied to an agent.