Barge-In and Interruption Handling in Voice AI

Scope: This walkthrough covers barge-in and interruption handling as a capability pattern across voice AI stacks, with specific reference to Teler’s WebSocket transport, LiveKit Agents, Amazon Nova Sonic, Hume EVI, and Cognigy Voice Gateway, verified against documentation current as of September 2026. It does not cover SIP-side dial plan configuration or carrier-level echo cancellation tuning.

Quick Answer: Barge-in voice AI stops agent audio the moment a caller speaks, using voice activity detection (VAD) or adaptive acoustic classification to detect the interruption, then flushes queued playback and restarts the turn. In Teler’s WebSocket transport, sending {"type": "clear"} flushes the playback buffer immediately. The client must manage its own state machine to discard in-flight audio chunks that arrive after the interruption signal in 2026 deployments.

Barge-in voice AI is the mechanism that lets a caller speak over an agent mid-utterance and have the system stop, process the new input, and continue from the new turn rather than finishing the interrupted response. Getting this right is the difference between a voice agent that feels like a phone call and one that feels like a voicemail.

What is barge-in and interruption handling?

Barge-in is a real-time voice-agent capability that lets a user interrupt an assistant mid-utterance, stop the current audio generation and playback, and continue from the new user input within the same session. It occupies the media and session-control layer of a voice pipeline, sitting between the audio transport and the language model. It does not handle transcription accuracy, speaker diarization, or call routing. The developer must supply the playback queue, the state machine that decides when to discard in-flight audio, and the resampling logic between carrier audio and model audio formats. On telephony paths, the developer also handles DTMF-triggered interruption separately from speech-triggered interruption, because keypad digits travel on a different signaling path defined by RFC 4733.

Why is barge-in voice AI hard to get right?

The core problem is that audio is full-duplex but most pipeline components are not. The agent is speaking, the model is still generating tokens, and the caller starts talking. Three things must stop simultaneously: local playback, the TTS stream, and the LLM generation. Each lives in a different process, often on a different host, and they receive the interruption signal at different times.

Network latency makes this worse. When the server emits an interruption event, audio chunks that were already in flight continue arriving at the client. If the client does not flush its playback buffer immediately and discard those chunks, the caller hears stale agent speech after the interruption. This is the most common production failure: the agent keeps talking for a fraction of a second after the caller has already started speaking.

A second problem is distinguishing a real barge-in from a backchannel. When a caller says “uh-huh” or “right” while the agent is speaking, that is conversational acknowledgment, not an instruction to stop. A VAD-only system treats both identically. An adaptive system runs a secondary classifier to decide whether the detected speech is intentional interruption or backchannel noise, which reduces false stops but adds model latency and tuning complexity.

How does the audio path work end to end for barge-in?

The path from caller to model and back involves several hops, each with its own format and owner. Understanding which hop owns what determines where you place your interruption logic.

The resampling hop between Teler’s 8 kHz stream and a model that expects a different rate, such as OpenAI Realtime at 24 kHz, is where many teams discover their latency problem. That conversion adds buffering delay that compounds with the interruption detection window. The RTP specification (RFC 3550) defines the timing and packetization model that governs the carrier side of this path.

What do you need before implementing barge-in?

• A Teler account with an active API key and at least one provisioned phone number. See Teler documentation for number provisioning in 50+ countries.
• A wss:// (TLS) WebSocket server that handles JSON text messages in both directions, plus an HTTPS flow endpoint that returns the stream Call Flow. Python 3.11+ with websockets 12+, or Node.js 20+ with ws 8+.
• An STT provider that returns interim (partial) transcripts, not only final results. Barge-in detection depends on detecting speech onset, not waiting for a complete utterance.
• A TTS provider that supports stream cancellation or that you can stop mid-stream by closing the connection.
• An LLM that supports streaming token output so generation can be cancelled mid-response.
• Audio resampling if your STT or TTS model operates at a sample rate other than 8 kHz. python-soxr (ResampleStream) for Python; a streaming resampler library for Node.
• Network reachability between your WebSocket server and Teler’s edge nodes. Teler targets sub-100ms packet delivery as a design specification.

How do you implement barge-in voice AI with Teler’s WebSocket transport?

Step 1: Receive the inbound audio stream and run VAD

Teler streams caller audio to your server over a WebSocket as JSON “audio” messages, each carrying a base64-encoded chunk of 16-bit PCM in data.audio_b64. Each chunk arrives as a binary frame. You run VAD on each chunk to detect speech onset. When VAD fires while the agent is in a speaking state, that is your interruption trigger.

import asyncio
import base64
import json

async def handle_call(websocket):
    state = CallState()  # defined in Step 2
    async for message in websocket:
        event = json.loads(message)
        if event["type"] == "start":
            state.agent_speaking = False
        elif event["type"] == "audio":
            # 16-bit linear PCM, 8 kHz by default
            pcm = base64.b64decode(event["data"]["audio_b64"])
            if run_vad(pcm) and state.agent_speaking:
                await on_barge_in(websocket, state)

The {"type": "clear"} message is the key: it tells Teler to flush any audio queued for playback. Send it before you start processing the new caller input, not after.

Step 2: Manage the state machine across three concurrent processes

Interruption handling requires coordinating three concurrent operations: stopping TTS playback, cancelling LLM generation, and beginning STT on the new caller input. Cancelling the TTS task and tagging every chunk with a turn ID works better than a shared asyncio.Event, which can be set and cleared again before the TTS coroutine ever checks it.

class CallState:
    def __init__(self):
        self.agent_speaking = False
        self.turn_id = 0
        self.chunk_counter = 0
        self.tts_task = None  # task running stream_tts_to_teler for the current turn

async def stream_tts_to_teler(websocket, state, tts_stream, turn_id):
    state.agent_speaking = True
    async for audio_chunk in tts_stream:  # 16-bit PCM at 8 kHz
        if turn_id != state.turn_id:
            break  # stale turn: discard the rest
        state.chunk_counter += 1
        await websocket.send(json.dumps({
            "type": "audio",
            "audio_b64": base64.b64encode(audio_chunk).decode("utf-8"),
            "chunk_id": f"{turn_id}-{state.chunk_counter}",
        }))

async def on_barge_in(websocket, state):
    state.turn_id += 1           # every chunk from the old turn is now stale
    if state.tts_task:
        state.tts_task.cancel()  # stops TTS writes and the LLM stream feeding it
    await websocket.send(json.dumps({"type": "clear"}))
    state.agent_speaking = False

The ordering matters: bump the turn ID and cancel the TTS task before sending clear, so no audio from the old turn is written to the socket after the flush.

Step 3: Discard in-flight audio chunks after the interruption signal

Messages on one WebSocket arrive in order, so anything you sent before the clear is flushed with it. The risk is chunks your server sends after the clear, before your TTS task has stopped: those will still play. This is why the state machine in step 2 must stop the TTS write loop before the clear message, not after.

On the receive side, if you are using a platform that emits an explicit interruption event, such as input_audio_buffer.speech_started in the OpenAI Realtime API or server_content.interrupted = true in Gemini Live, you must also discard any audio chunks that arrive after that event. A dedicated turn counter or generation ID is the cleanest way to tag which chunks belong to the current turn and which are stale.

Step 4: Configure adaptive interruption handling where the platform supports it

LiveKit Agents exposes three interruption modes: mode="adaptive", mode="vad", and enabled=false. Adaptive mode runs an acoustic classifier on top of VAD to distinguish intentional barge-in from backchannel speech. For Cognigy Voice Gateway, the parameters are bargeIn, bargeInOnSpeech, and bargeInOnDtmf, with DTMF-triggered interruption handled separately from speech. Chunk size and sample rate are set in the stream Call Flow (chunk_size, sample_rate). Chunk size must be a multiple of 20 ms and defaults to 400 ms, which means VAD only sees caller audio in 400 ms blocks. Set it to 20 ms for faster detection. Smaller chunks reduce interruption detection latency but increase WebSocket frame overhead.

Most teams discover their latency problem is not the model, it is the resampling hop they added between the carrier and the STT engine. That extra buffering delay compounds directly with the VAD window, so a 200ms resampler buffer turns a 150ms interruption detection target into a 350ms one before you have written a single line of barge-in logic.

– Subhash Kalluri, Founder & CEO, FreJun

What breaks in production barge-in voice AI deployments?

The failure modes below come from real GitHub issues and practitioner reports. Frequency is noted honestly: most are single reports, which means they are real but not universal.

The stale-audio failure catches most teams in their first production deployment. The interruption event arrives, the client stops sending new audio, but chunks already in the network continue to play. The fix is not to wait for the clear acknowledgment before stopping playback: stop playback first, send the clear, then discard everything that arrives with a stale turn ID. Teler numbers incoming messages with a monotonic message_id and requires a unique chunk_id on every chunk you send, so you can encode the turn ID in the chunk_id. For finer control than clear, {“type”: “interrupt”, “chunk_id”: …} stops playback from a specific chunk onward.

Which barge-in approach fits your architecture?

There is no single correct approach. The right choice depends on where your media terminates, how much false-positive tolerance your use case allows, and how much custom client logic you want to own.

For most teams starting with barge-in voice AI in 2026, VAD-only with a well-tuned silence timeout is the right starting point. Move to adaptive classification when you have call recordings showing that backchannels are causing false stops at a rate that affects completion metrics. The opposite choice, starting with adaptive classification, is right when your use case involves noisy environments or languages with frequent overlapping speech patterns, because VAD-only will produce an unacceptable false-positive rate before you have enough data to tune it.

For deeper context on how real-time audio streaming interacts with interruption handling across the full pipeline, the article on handling real-time audio streams with voice API integration covers the buffering and transport layer in detail. If you are building on LiveKit specifically, the top voice APIs for LiveKit in the US comparison covers provider options that affect which interruption modes are available to you.

Frequently asked questions

What audio format does Teler use for barge-in streams?

Teler streams 16-bit linear PCM over WebSocket, base64-encoded inside JSON messages, at 8 kHz by default. 16 kHz is configurable, though the start message currently always reports 8000. Audio you send back must be 8 kHz. Chunk size must be a multiple of 20 ms, defaulting to 400 ms. When streaming to OpenAI Realtime, which operates at 24 kHz, you must resample in your application layer. The ITU-T G.711 standard defines the narrowband PCM format used on the PSTN side of the path.

What happens when the interruption signal arrives but audio chunks are still in flight?

In-flight chunks continue arriving after the interruption event because they were already in the network buffer. Stop local playback immediately, send {"type": "clear"} to flush Teler’s playback queue, and discard any chunks that arrive with a stale turn identifier. Do not wait for a clear acknowledgment before stopping playback: the ordering matters because chunks your server sends between the clear and the TTS stop will still play.

Should I use SIP or WebSocket for barge-in in a telephony context?

SIP, defined in RFC 3261, handles call setup and teardown but carries media over RTP, not over a WebSocket. For barge-in logic, the relevant transport is the WebSocket connection between your server and the voice platform, not the SIP leg. Teler terminates the SIP and RTP path from the carrier and exposes a WebSocket to your application, so you write interruption logic against the WebSocket protocol regardless of whether the underlying call is PSTN or VoIP.

Does barge-in handling degrade under high call volume?

VAD and adaptive classification run per-session, so they scale horizontally with call volume as long as your WebSocket server and STT provider scale. The risk under load is increased network RTT between your server and the voice platform, which widens the window during which in-flight audio chunks arrive after an interruption signal. Teler’s edge-distributed points of presence reduce this RTT, but the application-layer state machine must still handle late-arriving chunks correctly regardless of load.

What must I build myself that the platform does not provide?

You must build the playback queue and the state machine that coordinates stopping TTS, cancelling LLM generation, and beginning STT on the new input. You must also handle audio resampling between carrier format and model format, DTMF-triggered interruption if your flow requires it, and the turn-ID tagging system that lets you discard stale audio chunks. Platforms like Hume EVI and LiveKit abstract some of this, but on a raw WebSocket transport like Teler’s, the control loop is yours to own.

How do I handle false barge-ins from backchannel speech?

Start with VAD tuning: increase min_interruption_duration and min_interruption_words where the platform exposes them, as LiveKit does. If false stops persist after tuning, switch to adaptive mode, which runs an acoustic classifier to distinguish intentional interruption from backchannel acknowledgment. Adaptive mode adds model latency, so measure the detection delay against your latency budget before committing. Yellow.ai’s context-aware approach, which evaluates interruption validity against conversational state, is a third option for high-stakes flows.

Is DTMF-triggered barge-in handled the same way as speech barge-in?

No. DTMF digits travel on a separate signaling path defined by RFC 4733 and are not detected by VAD. Cognigy Voice Gateway exposes bargeInOnDtmf as a separate parameter from bargeInOnSpeech, and the two can be enabled independently. If your flow requires both, you need separate handling paths: VAD or adaptive classification for speech, and a DTMF event listener for keypad input.

What changed in Cognigy Voice Gateway 4.80 for barge-in?

Before release 4.80, barge-in was always active when a call was transferred to a contact center and could not be controlled in that path. Starting with 4.80, the bargeIn parameter can be enabled or disabled after transfer, giving developers explicit control over whether the caller can interrupt the agent during a transferred call. If you are running an older release and seeing unexpected barge-in behavior on transferred calls, this is the likely cause.

What barge-in voice AI gives you and when to choose differently

With a working barge-in voice AI implementation, callers can interrupt an agent mid-utterance and receive a response to their actual question rather than waiting for the agent to finish a now-irrelevant answer. The one decision that matters most is where you place your interruption detection: in the platform’s adaptive classifier, in your own VAD loop, or in a context-aware policy layer. Choose VAD-only when simplicity and low latency matter more than false-positive tolerance. Choose adaptive classification when call recordings show that backchannels are breaking flows. And choose a different approach entirely if your documentation does not specify the detection mechanism, because an undocumented behavior is an untestable one.

For teams building on Teler’s WebSocket transport, the {"type": "clear"} flush message and a well-ordered state machine cover the majority of barge-in voice AI production requirements without additional platform dependencies. For broader context on building real-time voice agents, the guide on building real-time assistants using an AI voice agent API covers the full pipeline architecture, and the post on enabling real-time interruptions while building voice bots addresses the organizational and tooling decisions that surround the technical implementation.

About the Author: Subhash Kalluri is the Founder & CEO of FreJun, which he has been building since 2019. With over 8 years in voice communication and SaaS, he works on the problem of getting real-time AI onto live phone calls: raw media streaming, interruption handling, and carrier connectivity. That work became FreJun Teler, FreJun’s programmable voice API for developers building AI phone agents. Connect with him on LinkedIn.