AI Summary: This article examines how many concurrent calls a voice agent deployment can sustain in 2026, and is written for backend and API integration engineers evaluating telephony infrastructure. The OpenAI Realtime API removed its simultaneous-session ceiling on February 3, 2025, but account-level rate limits still govern throughput at each tier. Engineers must account for three independent ceilings: model-layer sessions, telephony-layer channels, and audio pipeline capacity. Teler’s programmable voice API handles the telephony and streaming layer, letting teams focus on model and application logic rather than carrier plumbing.
Scope: This walkthrough covers concurrent call capacity for AI voice agents using the OpenAI Realtime API over WebSocket and SIP transports, combined with Teler’s telephony infrastructure, verified against API documentation current as of September 2026. It does not cover Azure OpenAI deployments or on-premises PBX configurations.
Quick Answer: In 2026, the OpenAI Realtime API no longer caps simultaneous sessions at the account level, but your concurrent calls ceiling is still set by your rate-limit tier (RPM ranges from 200 at Tier 1 to 20,000 at Tier 5), your telephony channel allocation, and whether your audio pipeline can sustain the required 8 kHz or 16 kHz streams without queuing delay.
The number of concurrent calls a voice agent can handle is determined by three independent ceilings: the model provider’s session and rate limits, the telephony layer’s channel capacity, and the audio pipeline’s ability to process and inject audio without introducing latency that breaks conversational turn-taking.
What is concurrency in voice agent deployments?
Concurrency in voice agent deployments is the number of simultaneous, independent phone calls that an AI voice system can sustain without degrading audio quality, response latency, or call reliability. It occupies the intersection of three layers: the AI model session layer, the telephony transport layer, and the real-time audio processing pipeline. It does not describe batch throughput or queued call volume. Each concurrent call requires a dedicated, stateful connection to the model, a live audio stream to and from the carrier, and enough compute to run speech-to-text (STT), the large language model (LLM), and text-to-speech (TTS) within the latency budget of a natural conversation. Engineers must provision all three layers independently; a ceiling on any one of them caps the whole system.
Why is scaling concurrent calls harder than scaling HTTP requests?
HTTP requests are stateless and short-lived. A concurrent call is a stateful, long-lived, bidirectional audio stream where every 20-millisecond packet must arrive in order, on time, or the caller hears a glitch. The engineering constraints compound across layers in ways that do not appear in standard web scaling.
Three specific problems make this hard. First, audio format mismatches: the public switched telephone network (PSTN) delivers audio at 8 kHz G.711 PCM, while the OpenAI Realtime API operates at 24 kHz internally, so every call requires a resampling hop that adds CPU load and latency. Second, turn-taking: a voice agent must detect when the caller has finished speaking (voice activity detection, or VAD), stop its own playback if the caller interrupts (barge-in), and respond within roughly 500 to 800 milliseconds or the conversation feels broken. Third, jitter: the PSTN and SIP trunks introduce packet timing variation (jitter), and without a properly sized jitter buffer, audio arrives out of order and the STT engine produces garbage transcripts.
Each of these problems scales linearly with concurrent call count. At 10 calls the resampling load is manageable on a single instance. At 500 calls it requires horizontal scaling with careful session affinity, because a WebSocket session cannot migrate between nodes mid-call.
What does the audio path look like end to end for concurrent calls?
Understanding where audio travels on each concurrent call is the prerequisite for knowing where the bottlenecks appear. The path from caller to model and back involves at least five distinct hops, each with its own format, owner, and latency contribution.
| Hop | Format | Sample rate | Transport | Owner |
|---|---|---|---|---|
| Caller to carrier | G.711 PCM (ITU-T G.711) | 8 kHz | PSTN / SIP | Carrier |
| Carrier to telephony API | G.711 or resampled PCM | 8 kHz or 16 kHz | SIP trunk (RFC 3261) | Teler |
| Telephony API to your server | 16-bit linear PCM, base64 in JSON | 8 kHz or 16 kHz (configurable) | WebSocket (RFC 6455) | Teler to your backend |
| Your server to model | Base64 audio bytes, up to 15 MB per chunk | 24 kHz (OpenAI Realtime internal) | WebSocket or WebRTC | Your backend to OpenAI |
| Model response to caller | Resampled PCM injected back | 8 kHz (PSTN requirement) | WebSocket to SIP to PSTN | Your backend, Teler, carrier |
The resampling between 24 kHz (OpenAI Realtime) and 8 kHz (PSTN) is a step your backend must perform. Teler’s WebSocket stream is configurable at 8k or 16k, defaulting to 8k, which matches the PSTN floor. The OpenAI Realtime API operates at 24 kHz internally, so if you stream at 8k you need to upsample before sending audio to the model and downsample before injecting the response back. This resampling cost multiplies with every concurrent call.
What do you need before building for concurrent calls at scale?
Before writing a line of code, confirm you have the following in place:
- An OpenAI account at the tier matching your target concurrency: Tier 1 supports 200 RPM, Tier 5 supports 20,000 RPM. Check your current tier in the OpenAI platform dashboard.
- A Teler account with phone numbers provisioned in your target countries. Teler provisions numbers in 50+ countries via API. See the Teler documentation for number provisioning endpoints.
- A server-side WebSocket handler capable of maintaining one persistent connection per concurrent call. Node.js with the
wslibrary or Python withwebsocketsare common choices. - A streaming resampling library: python-soxr (ResampleStream) in Python, or a streaming resampler library in Node.js. Pin the version.
- Network reachability from your backend to
wss://api.openai.com/v1/realtimeand to Teler’s WebSocket endpoint. - Environment variables: OPENAI_API_KEY, TELER_API_KEY, and your Teler phone number in E.164 format.
How do you wire concurrent calls through Teler and the OpenAI Realtime API?
The implementation has three distinct steps: accepting the inbound call and opening the Teler media stream, opening a Realtime API session per call, and bridging audio bidirectionally. Each step must be isolated per call so that one session’s failure does not cascade.
Step 1: Accept the inbound call and open the media stream
When a call arrives, Teler POSTs to your Incoming Call URL (or the flow_url you passed when starting an outbound call). Your handler responds with a stream Call Flow whose ws_url points at your wss:// media endpoint. Set the sample rate explicitly; the default is 8k. Chunk size defaults to 400 ms and must be a multiple of 20 ms.
import asyncio
import websockets
import json
async def handle_teler_stream(websocket, call_id):
# Teler sends raw PCM audio as Base64-encoded chunks
async for message in websocket:
event = json.loads(message)
if event["type"] == "audio":
yield event["data"]["audio_b64"] # base64, 16-bit PCM at 8kThe call_id in Teler’s start message is your session key. Every concurrent call gets its own coroutine; never share a WebSocket connection across calls.
Step 2: Open a Realtime API session per call
Open one WebSocket connection to wss://api.openai.com/v1/realtime per concurrent call. The connection needs the model as a query parameter and an Authorization: Bearer YOUR_API_KEY header. The OpenAI-Beta: realtime=v1 header belongs to the old beta interface. The session is stateful; it holds conversation context for the duration of the call.
import websockets
import os
async def open_realtime_session():
uri = "wss://api.openai.com/v1/realtime?model=gpt-realtime"
headers = {"Authorization": f"Bearer {os.environ['OPENAI_API_KEY']}"}
# additional_headers requires websockets 14+
ws = await websockets.connect(uri, additional_headers=headers)
return ws
Watch the connection count against your tier’s RPM limit. At Tier 1 (200 RPM), a burst of inbound calls can exhaust your request budget before sessions are established.
Step 3: Bridge audio bidirectionally
Audio from Teler arrives as Base64 PCM at 8 kHz. Before sending it to the Realtime API, upsample to 24 kHz. Send it using the input_audio_buffer.append client event. The audio chunk cap is 15 MB per append; in practice, 400 ms chunks at 8 kHz are well under 100 KB, so this limit is not a practical concern at normal chunk sizes.
import base64
import json
async def send_audio_to_realtime(realtime_ws, pcm_base64_8k):
# Decode, upsample 8k -> 24k, re-encode
pcm_bytes = base64.b64decode(pcm_base64_8k)
# upsample(pcm_bytes) is your resampling function
upsampled = upsample(pcm_bytes, from_rate=8000, to_rate=24000)
payload = {
"type": "input_audio_buffer.append",
"audio": base64.b64encode(upsampled).decode()
}
await realtime_ws.send(json.dumps(payload))
When the caller interrupts (barge-in), use the Realtime API’s input_audio_buffer.speech_started event as the trigger and send {“type”: “clear”} to that call’s Teler WebSocket to discard queued audio. Then send conversation.item.truncate with how much of the reply was actually played, so the model’s context matches what the caller heard.
For the full Teler streaming reference, including chunk size parameters and the clear event specification, see the Teler media streaming documentation.
Most teams discover their latency problem is not the model response time. It is the resampling hop they inserted between the carrier stream and the STT engine, running synchronously in the same thread that handles WebSocket I/O. At 50 concurrent calls that single-threaded resampler becomes the bottleneck, and the symptom looks like model latency because the audio arrives late.
– Subhash Kalluri, Founder & CEO, FreJun
What breaks in production when concurrent calls scale up?
The failure modes below come from documented developer reports. Each one has appeared in production deployments of the OpenAI Realtime API. Frequency is noted honestly; a single report is not a systemic issue, but it is worth knowing before you hit it at 3 a.m.
| Symptom | Root cause | Fix | Frequency |
|---|---|---|---|
WebSocket drops mid-call; client throws RealtimeAPI is not connected on input_audio_buffer.append | Server pings on a roughly 20-second keepalive; the client library did not respond with matching pong bytes | Implement explicit ping-pong keepalive handling in your WebSocket client | Reported once (OpenAI Community, December 2024) |
| Session closes after 5 to 10 seconds of inactivity even during active voice turns | Application-level bugs in the audio loop; not a platform defect in the confirmed case | Audit your audio send loop for gaps that look like inactivity to the server | Reported once (GitHub openai-dotnet, October 2024) |
| Browser WebSocket closes immediately with code 1006 | Close originates from the browser side; root cause unresolved in the thread | No confirmed fix; check browser CORS and ephemeral token expiry (tokens expire one minute after issuance) | Reported once (GitHub BerriAI/litellm, November 2024) |
10-second stall before rate_limits.updated, delaying all subsequent server events | Unresolved; appeared to correlate with the mini model variant | Treat as transient; implement retry with exponential backoff on response.done timeout | Reported once (OpenAI Community, February 2025) |
Telephony bridge returns invalid_request_error / invalid_value on the realtime WebSocket path | Provider-side routing or phone number configuration mismatch on the Twilio side | Verify the phone numbers and routing configuration on your SIP provider | Reported once (OpenAI Community, March 2025) |
| Historical hard ceiling on simultaneous sessions in beta | Account-level simultaneous-session limit; Tier 5 was previously around 100 sessions | OpenAI removed the simultaneous-session limit on February 3, 2025 | Resolved platform change |
The keepalive failure is the one that catches most teams. A WebSocket that drops silently under load looks like a model timeout or a network blip. The actual cause is that the base client library does not respond to server ping frames, so the server closes the connection after the keepalive window expires. At low concurrent call counts the reconnect happens fast enough to be invisible. At high concurrent call counts, many sessions drop simultaneously and the reconnect storm can exhaust your RPM budget.
One honest limitation of Teler worth stating: the telephony layer controls channel allocation and audio streaming, but it does not manage OpenAI Realtime session state. If the model-side WebSocket drops, your backend is responsible for detecting the disconnect, re-establishing the session, and replaying any buffered audio. Teler does not do this automatically.
Which architecture handles concurrent calls most reliably?
Three distinct approaches exist for connecting concurrent calls to a real-time AI model. Each makes a different trade-off between where audio terminates, latency cost, and operational complexity.
| Approach | Audio terminates at | Latency cost | Operational burden | Lock-in |
|---|---|---|---|---|
| Server-side WebSocket bridge | Your backend | Resampling and buffering in your code; you control the pipeline | High: session management, keepalive, resampling, scaling | Low: swap the model endpoint without changing the telephony layer |
| Browser WebRTC gateway | Browser or WebRTC edge client | Low for browser apps; not applicable to PSTN calls | Medium: ephemeral token minting, browser media plumbing | Medium: WebRTC peer connection is browser-native (W3C WebRTC 1.0) |
| SIP direct integration | SIP endpoint or media server | Depends on SIP trunk quality and codec negotiation | Medium: SIP routing, codec compatibility, DTMF handling | Low: SIP is a standard (RFC 3261); provider-portable |
| Telephony API with channel management | Provider voice layer (e.g., Teler) | Sub-100ms packet delivery target; resampling to the model’s rate stays in your bridge | Low: carrier, number provisioning, and streaming abstracted | Medium: provider-specific WebSocket protocol and webhook format |
For teams building AI phone agents at scale, the telephony API approach reduces the surface area your team must own. The server-side WebSocket bridge gives you the most control and the lowest model-layer lock-in, but it requires you to own session management, keepalive, resampling, and horizontal scaling. The SIP direct integration is the right choice when you already operate a media server or PBX and want to connect it to a model without adding a new intermediary layer. Browser WebRTC is not applicable to inbound PSTN calls.
If your team does not have telephony engineering experience, start with a managed telephony API. You can always migrate the media path later once you understand the failure modes. If you do have that experience and need maximum control over codec negotiation and jitter buffer tuning, the SIP direct path is worth the complexity. For more on scaling AI voice agent APIs for contact center traffic, the linked post covers horizontal scaling patterns in detail.
Frequently asked questions
Does the OpenAI Realtime API still limit concurrent calls per account?
No. OpenAI removed the simultaneous-session limit on February 3, 2025. Before that date, Tier 5 accounts were limited to roughly 100 simultaneous sessions. The current ceiling is your rate-limit tier: Tier 1 allows 200 RPM, Tier 2 allows 400 RPM, Tier 3 allows 5,000 RPM, Tier 4 allows 10,000 RPM, and Tier 5 allows 20,000 RPM. Each new session establishment counts against your RPM budget, so a burst of concurrent calls can exhaust it even if the session count itself is no longer capped.
What audio format does the PSTN deliver, and does it match what the Realtime API expects?
No, they do not match. The PSTN delivers audio at 8 kHz G.711 PCM, as defined in ITU-T G.711. The OpenAI Realtime API operates at 24 kHz internally. Your backend must upsample inbound audio from 8 kHz to 24 kHz before sending it via input_audio_buffer.append, and downsample the model’s audio response from 24 kHz to 8 kHz before injecting it back into the call. Teler’s WebSocket stream is configurable at 8k or 16k, defaulting to 8k, which matches the PSTN floor but still requires upsampling before the model.
What happens if a WebSocket session drops mid-call?
The caller hears silence or a dead line. The OpenAI Realtime API WebSocket is stateful, so a dropped connection loses the session context. Your backend must detect the disconnect via the WebSocket close event, re-establish the session, and decide whether to replay buffered audio or restart the conversation. The most common cause of unexpected drops is a missing keepalive: the server sends ping frames on roughly a 20-second interval, and clients that do not respond with pong frames are disconnected. Implement explicit ping-pong handling in your WebSocket client.
Should I use SIP or WebSocket to connect concurrent calls to the model?
Use WebSocket if your backend is the media bridge and you want to control the audio pipeline directly. Use SIP if you already operate a PBX, media server, or SIP trunk infrastructure and want to connect it to the model without adding a new intermediary. The OpenAI Realtime API supports both, as well as WebRTC for browser clients. SIP integration is documented for connecting to the public phone network, PBX systems, and desk phones. For more on improving SIP compatibility with voice API integration, the linked post covers codec negotiation and trunk configuration.
What does Teler handle, and what must my team build?
Teler handles carrier connectivity, phone number provisioning in 50+ countries, SIP trunking, real-time audio streaming over WebSocket, interrupt and clear messages for barge-in (detection is yours to build), and live call control including hangup, mute, DTMF, playback, and transfer. Your team must build the session management logic for the model-side WebSocket, the resampling pipeline between 8 kHz and 24 kHz, the STT-LLM-TTS orchestration, and the reconnect logic when a model session drops. Teler does not manage OpenAI Realtime session state on your behalf.
How do I handle barge-in across many concurrent calls without audio bleed?
Each concurrent call runs in an isolated session with its own WebSocket connection to both Teler and the model. When the caller interrupts, send {"type": "clear"} to the Teler WebSocket for that specific call. This flushes queued playback for that session only and does not affect other concurrent calls. The isolation is per-connection, so audio bleed between sessions is not a platform-level risk. The risk is in your own code: Teler opens a separate WebSocket per call, so keep a per-call map from call_id to its socket and make sure each clear goes to the right one.
What is the practical concurrent call ceiling for a single backend instance?
The documentation does not specify a per-instance ceiling. In practice, the limit is set by CPU load from resampling, memory from holding audio buffers per session, and the event loop’s ability to service WebSocket I/O without queuing delay. Teams running synchronous resampling in the same thread as WebSocket I/O typically see degradation around 50 concurrent calls on a single instance. Moving resampling to a worker pool and using async I/O throughout extends this significantly. Horizontal scaling with session affinity is the standard approach for production deployments handling hundreds of concurrent calls.
Does the ephemeral client secret for WebRTC work for PSTN calls?
No. The ephemeral client secret is for browser-side WebRTC connections, where the secret is minted server-side and passed to the browser. It expires one minute after issuance. PSTN calls use the server-side WebSocket path with a standard API key, or the SIP integration path. WebRTC is not applicable to inbound phone calls from the PSTN. If you are building a browser-based voice agent alongside a phone agent, you need separate session management for each transport.
How many concurrent calls can your voice agent handle in 2026?
The answer depends on which of the three ceilings you hit first: your OpenAI rate-limit tier, your telephony channel allocation, or your audio pipeline’s processing capacity. The simultaneous-session cap that previously limited concurrent calls to roughly 100 at Tier 5 was removed in February 2025, so the model layer is no longer the binding constraint for most deployments. The binding constraint is now the combination of your RPM tier and your backend’s ability to maintain isolated, stateful WebSocket sessions with proper keepalive, resampling, and reconnect logic. Teler handles the telephony and streaming layer, which removes carrier complexity from the equation, but your team still owns the session orchestration between the telephony stream and the model. If your concurrent calls target exceeds what a single backend instance can sustain, horizontal scaling with session affinity is the path forward, not a provider upgrade.
