ElevenLabs API on a Live Phone Call

Scope: This walkthrough covers streaming audio between the ElevenLabs API and a live PSTN call over a WebSocket media bridge, verified against the ElevenLabs v1 API surface documented in September 2026. It does not cover SIP-side PBX configuration or ElevenLabs agent billing.

Quick Answer: The ElevenLabs API connects to a live phone call through a WebSocket media bridge that passes 8 kHz PCM audio between Teler and ElevenLabs, which supports 8 kHz PCM natively then injects synthesized speech back into the call. In 2026, ElevenLabs does not provision phone numbers or SIP trunks, so a telephony layer such as Teler must sit between the carrier and the ElevenLabs WebSocket endpoint.

Putting the ElevenLabs API on a live phone call means bridging two audio environments that were never designed to meet: a WebSocket-based speech platform that offers many audio formats and a PSTN carrier that delivers 8 kHz companded audio. Every production deployment of this integration requires a media bridge and explicit call contro that ElevenLabs itself does not provide.

What is the ElevenLabs API?

ElevenLabs is a voice API platform that provides real-time speech synthesis and transcription over WebSocket connections, sitting at the speech layer for developers building conversational audio applications. It handles text-to-speech and speech-to-text; it does not handle call routing, number provisioning, SIP signaling, or carrier connectivity. The platform exposes a WebSocket endpoint at wss://api.elevenlabs.io for agent conversations and a separate endpoint for streaming TTS, and the JavaScript SDK also supports WebRTC for browser-side voice sessions. A developer using the ElevenLabs API must supply the transport layer, the telephony infrastructure, audio format conversion, jitter buffering, and any call control logic such as DTMF handling or call transfer.

Why is connecting the ElevenLabs API to a phone call hard?

The core problem is a format mismatch that sits at every hop. PSTN carriers deliver audio encoded as G.711 mu-law or A-law at 8 kHz, as specified in ITU-T G.711. ElevenLabs offers many formats, from 8 kHz PCM up to 48 kHz Opus, so the formats have to be chosen to match at every hop. The ElevenLabs multi-context TTS WebSocket is also limited to five concurrent contexts per connection, which matters when rapid interruptions or parallel turns push context creation faster than the server releases closed contexts.

Turn-taking adds a second constraint. A live phone call has no built-in barge-in signal at the media layer. Voice activity detection (VAD) must run on the inbound audio stream, and when a caller interrupts, the system must flush queued synthesis audio before the next turn begins. ElevenLabs surfaces an event_id on user_transcript messages for interruption handling, but the flush and context teardown logic belongs to the bridge, not to ElevenLabs.

What does the audio path look like end to end?

The path from a caller’s handset to ElevenLabs and back involves at least four distinct hops, each with its own format, transport, and owner. The table below maps each hop so you can see where conversion work lives.

The bridge server is the piece you write. It holds two WebSocket connections open simultaneously: one to the telephony layer and one to ElevenLabs. Because ElevenLabs can take and return 8 kHz PCM, audio chunks pass through the bridge without resampling. The bridge only re-wraps them in each side’s JSON message format

Prerequisites

• An ElevenLabs account with an API key and a configured agent or voice ID
• A Teler account with a provisioned phone number and WebSocket media streaming enabled; see the Teler API documentation for credential setup
• Python 3.11 or later, or Node.js 20 or later
• The official ElevenLabs Python SDK from PyPI or the Node SDK from npm (both maintained in the elevenlabs-python and elevenlabs-js repositories)
• Network reachability from your bridge server to api.elevenlabs.io on port 443 (WSS) and to the Teler WebSocket endpoint
• An ELEVENLABS_API_KEY environment variable and your AGENT_ID or VOICE_ID

How do you implement the ElevenLabs API bridge for a live call?

Step 1: Open the Teler media stream

When Teler receives an inbound call, it fires a webhook and opens a WebSocket to your bridge server. Configure the stream at 8 kHz with 20 ms chunks to minimize latency. Teler’s chunk size range is 20 to 2000 ms in multiples of 20, with a default of 400 ms; for conversational voice, 20 ms gives you the tightest VAD window.

import asyncio
import websockets
import json

async def handle_teler_stream(teler_ws):
    # Teler sends audio as base64-encoded 16-bit PCM at 8 kHz.
    # The loop ends by itself when Teler closes the socket at the end of the call.
    async for message in teler_ws:
        event = json.loads(message)
        if event["type"] == "audio":
            raw_audio = event["data"]["audio_b64"]  # base64 PCM16, 8 kHz
            await process_inbound_audio(raw_audio, teler_ws)

The type field controls routing: only audio events carry audio payload. Teler closes the socket when the call ends, so there is no separate stop message.

Step 2: Send inbound audio to ElevenLabs at 8 kHz

ElevenLabs accepts 8 kHz PCM directly, so no resampling is needed. Set the input format to pcm_8000: use audio_format=pcm_8000 on the realtime speech-to-text WebSocket, or asr.user_input_audio_format on a Speech Engine. Teler’s audio is already 16-bit PCM at 8 kHz, so pass the base64 audio through unchanged

def teler_audio_to_elevenlabs(event: dict) -> str:
    # Teler audio is base64 16-bit PCM at 8 kHz. ElevenLabs accepts pcm_8000,
    # so the base64 string is passed through unchanged. No resampling.
    return event["data"]["audio_b64"]

Step 3: Open the ElevenLabs Speech Engine WebSocket

ElevenLabs Speech Engine lets your server own the LLM while ElevenLabs handles STT and TTS. The platform opens a WebSocket to your server (the “brain” connection), and you stream LLM responses back as text. Authentication uses an X-Elevenlabs-Speech-Engine-Authorization header containing a short-lived JWT signed with HS256 using the API key hash as the HMAC secret.

import websockets
import json

async def elevenlabs_brain_handler(websocket):
    # ElevenLabs connects to your server; you receive transcripts
    async for message in websocket:
        event = json.loads(message)
        if event["type"] == "user_transcript":
                        transcript = event["user_transcript"]  # list of {role, content} messages
            event_id = event["event_id"]  # use for interruption tracking
            llm_response = await call_your_llm(transcript)
                        await websocket.send(json.dumps({
                "type": "agent_response",
                "content": llm_response,
                "event_id": event_id,
                "is_final": True
            }))
        elif event["type"] == "ping":
            await websocket.send(json.dumps({"type": "pong"}))
        elif event["type"] == "close":
            break

Respond to every ping with a pong; the Speech Engine upstream protocol uses these to keep the connection alive, and a missed pong can trigger a server-side close.

Step 4: Request 8 kHz output from ElevenLabs and inject it into the call

Request output_format=pcm_8000 when you open the ElevenLabs text-to-speech WebSocket, for example wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/multi-stream-input?model_id=eleven_flash_v2_5&output_format=pcm_8000. ElevenLabs then returns 16-bit PCM at 8 kHz, which Teler plays as is, so each audio field can be forwarded unchanged. Send at least 500 ms of audio per chunk to avoid choppy playback

async def send_audio_to_teler(teler_ws, audio_b64: str, chunk_id: int):
    # ElevenLabs returns base64 16-bit PCM at 8 kHz (output_format=pcm_8000),
    # which is the format Teler plays, so it is forwarded unchanged.
    await teler_ws.send(json.dumps({
        "type": "audio",
        "audio_b64": audio_b64,
        "chunk_id": chunk_id,
    }))

Step 5: Handle barge-in and interruption

When a caller speaks while ElevenLabs is synthesizing, you must flush queued audio and close the active TTS context before starting the next turn. Send a {"type": "clear"} message to Teler to flush queued playback, then close the current ElevenLabs context before opening a replacement. The ElevenLabs multi-context TTS WebSocket enforces a hard limit of five simultaneous contexts per connection; exceeding it triggers a 1008 policy violation close.

async def handle_interruption(teler_ws, elevenlabs_ws, current_context_id: str):
    # Flush Teler playback queue
    await teler_ws.send(json.dumps({"type": "clear"}))
    # Close the active ElevenLabs TTS context before opening a new one
        await elevenlabs_ws.send(json.dumps({
        "context_id": current_context_id,
        "close_context": True
    }))

Wait for server acknowledgment of the context close before opening the next context; opening a replacement before the server releases the slot is the root cause of the 1008 error reported across multiple Pipecat and LiveKit integrations.

Most teams discover their latency problem is not the model, it is the resampling hop they added between the carrier and the STT engine.

– Subhash Kalluri, Founder & CEO, FreJun

What breaks in production with the ElevenLabs API?

The failure modes below come from confirmed GitHub issues across Pipecat and LiveKit integrations. Frequency is stated honestly: a single report is not a common failure.

The context-limit failure catches most teams first. Rapid interruptions during load testing push context creation faster than the server releases slots, and the 1008 close arrives without warning. The fix is not to retry the open; it is to serialize context lifecycle so a new context never opens until the previous one is confirmed closed. One honest limitation of Teler’s current WebSocket transport is that it does not expose a built-in context-tracking layer for ElevenLabs sessions, so that serialization logic must live in your bridge code.

Which integration approach fits your architecture?

ElevenLabs supports several integration patterns. The right one depends on where your media terminates and how much of the speech stack you want to own.

For most teams building an AI phone agent in 2026, Speech Engine is the right starting point. ElevenLabs handles the speech primitives, your server handles the LLM, and the telephony layer handles the carrier. Use the raw WebSocket bridge when you already control the media path and need to swap speech providers without rearchitecting. The direct agent session is the fastest path to a working demo but the hardest to adapt when telephony requirements arrive later.

Frequently asked questions

What audio format does the ElevenLabs API output for real-time TTS?

ElevenLabs documents API output formats as MP3 at 22.05 kHz and 44.1 kHz, PCM at 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 44.1 kHz, and 48 kHz, Opus at 48 kHz, and mu-law and A-law at 8 kHz. For telephony integration, request PCM at 8 kHz or mu-law at 8 kHz to eliminate the downsampling step on the return path. The real-time TTS WebSocket guide notes that the v1/text-to-speech/{voice_id}/stream-input endpoint does not support the eleven_v3 model.

Does ElevenLabs provide phone numbers or SIP trunking?

No. ElevenLabs does not provision phone numbers, SIP trunks (defined as the signaling and media path between a PBX and a carrier, per RFC 3261), or PSTN connectivity of any kind. A separate telephony provider must sit between the carrier and the ElevenLabs WebSocket endpoint. Teler provisions numbers in 50 or more countries via API and streams call audio over WebSocket as 16-bit PCM at 8 kHz, which ElevenLabs accepts directly as pcm_8000 See Teler’s documentation for SIP trunk configuration and number provisioning details.

What causes the 1008 policy violation WebSocket close?

ElevenLabs enforces a limit of five concurrent TTS contexts per WebSocket connection. The 1008 close fires when a new context opens before the server has released a previously closed one. This happens most often during rapid interruptions or fast tool-call chains. The fix is to wait for server-side confirmation of a context close before opening the next one, not to retry the failed open. Three separate integrations reported this failure in 2026 across Pipecat and LiveKit.

Should I use WebSocket or WebRTC for the ElevenLabs API on a phone call?

WebSocket is the right choice for server-side telephony bridges. WebRTC, as defined in the W3C WebRTC specification, is designed for browser-to-browser or browser-to-server media and uses Opus at 48 kHz by default. Bridging a WebRTC session to PSTN requires a media gateway that resamples Opus 48 kHz to G.711 8 kHz, which adds a conversion hop that WebSocket avoids. Use the ElevenLabs JavaScript SDK with WebRTC when the first-party client is a browser; use the WebSocket API when the source is a carrier call.

What happens to the ElevenLabs connection under load?

Under load, the five-context limit becomes the binding constraint before any other rate limit surfaces in the documented API. The STT WebSocket can also drop mid-stream under network instability, and the LiveKit plugin does not reconnect automatically on retryable errors. At scale, run one WebSocket connection per concurrent call rather than multiplexing calls across a shared connection, and implement reconnection logic at the bridge layer for both the STT and TTS sockets.

What must I build myself when using the ElevenLabs API for phone calls?

You must build or source: a telephony transport layer for carrier connectivity, phone number provisioning, the resampling and context management remain your responsibility in the bridge server”. Add: “context management remains your responsibility in the bridge server, a jitter buffer for network-induced timing variation, VAD for barge-in detection, call control logic including DTMF handling and call transfer, and context lifecycle management for the ElevenLabs TTS WebSocket. Teler covers the telephony transport, number provisioning, and call control; the resampling and context management remain your responsibility in the bridge server.

How do I handle caller interruptions cleanly with the ElevenLabs API?

When VAD detects caller speech during synthesis, send a {"type": "clear"} message to Teler to flush queued playback, then send close_context to ElevenLabs with the active context ID. Wait for server acknowledgment before opening the next context. The ElevenLabs Python SDK cancels the previous response handler automatically when a new user_transcript arrives in Speech Engine mode, but in a raw WebSocket bridge you must implement this teardown sequence yourself.

What can you build now with the ElevenLabs API on a live call?

With the ElevenLabs API connected to a telephony layer in 2026, you can build a fully conversational AI phone agent where ElevenLabs handles speech synthesis and transcription, your server handles LLM orchestration, and Teler handles carrier connectivity and call control. The one decision that matters most is where your media terminates: choose Speech Engine if you want ElevenLabs to own the speech primitives, and choose the raw WebSocket bridge if you need to swap speech providers or control every audio hop. If your primary constraint is browser-first interaction rather than PSTN, the WebRTC path through the JavaScript SDK is the lower-friction option, but it requires a media gateway the moment a carrier call enters the picture. For teams evaluating the top use cases of ElevenLabs AI voice tools in production, the telephony integration path described here is the one that requires the most infrastructure work and the most careful attention to audio format constraints. You can also explore how VoIP calling API integration for ElevenLabs improves AI voice apps for a broader view of the integration surface. For the telephony infrastructure side, Teler’s programmable voice API handles the carrier layer so the ElevenLabs API can focus on what it does well.

About the Author: Subhash Kalluri is the Founder & CEO of FreJun, which he has been building since 2019. With over 8 years in voice communication and SaaS, he works on the problem of getting real-time AI onto live phone calls: raw media streaming, interruption handling, and carrier connectivity. That work became FreJun Teler, FreJun’s programmable voice API for developers building AI phone agents. Connect with him on LinkedIn.