Full-Duplex Interaction

Sources#

Summary#

"Full-duplex" = the model perceives and responds at the same time, in a constant two-way exchange — as opposed to half-duplex turn-taking (one party at a time). Interaction Models generalize the audio full-duplex idea across audio, video, and text. The phrase the post uses: an experience that "feels more like collaborating and less like prompting."

The interaction modes it enables#

All of these are special-purpose harnesses today; in an interaction model they're special cases of model behavior (see Time-Aligned Micro-Turns):

Proactive interjection — "interrupt when I say something wrong"; the model jumps in mid-turn when context warrants, not only at end-of-turn.
Visual-cue reactions — "tell me when I've written a bug in my code"; "count how many pushups I do"; requires acting on a visual change with no audio cue (audio-only turn-detection harnesses fail this — they say "Sure thing!" then go silent).
Simultaneous speech — user and model speak concurrently: "translate Spanish→English live."
Speak-while-watching — "live-commentate this sports game."
Time-aware speech — "remind me to breathe in and out every 4 seconds until I stop"; "how long did it take me to write this function?"
Codeswitch correction — "every time I use another language, give me the correct word in the original language" (requires speaking at the same time as the user).

The model implicitly tracks whether the speaker is thinking, yielding, self-correcting, or inviting a response — no separate dialog-management component.

Concurrent non-speech action#

While listening and speaking, the model can simultaneously call tools, search, browse, or generate UI — weaving results back into the conversation when appropriate. The deeper/longer of these are delegated to the background model.

Prior art it builds on#

Audio full-duplex models are the existing example of bidirectional/continuous interaction; robotics and autonomous vehicles are cited as domains where real-time perception+action is a given. Interaction models apply the principle across all modalities.

In production: GPT-Live (July 2026)#

OpenAI's GPT-Live ships audio full-duplex at ChatGPT scale: "its voice model is full-duplex, which means it can listen and speak at the same time. That eliminates the need for a separate detector" — the turn detector is out of the audio path entirely, and turn-taking, interruption, and floor-holding become model behavior. Scope difference worth keeping: GPT-Live is audio-only full-duplex with concurrent tool use delegated out (Interaction / Background Model Split); TML's cross-modal generalization (visual-cue reactions, speak-while-watching) remains a research preview. The systems consequence of full-duplex — the conversation becomes a continuous media loop where every frame must arrive on schedule — is what forces the serving architecture in Live-Path Minimalism.

Connections#

Interaction Models — parent concept
Time-Aligned Micro-Turns — the mechanism (no turn boundaries) that makes full-duplex possible
Encoder-Free Early Fusion — joint multimodal reasoning is what lets a visual change trigger speech
Turn-Based Interface Bottleneck — the half-duplex status quo this replaces
Interactivity Benchmarks — TimeSpeak / CueSpeak / RepCount-A / ProactiveVideoQA / Charades measure exactly these modes
Interaction / Background Model Split — where the concurrent deep work goes
TML-Interaction-Small — the model that demonstrates these interaction modes
GPT-Live — audio full-duplex in production; the turn detector eliminated at ChatGPT scale
Live-Path Minimalism — the serving architecture the every-frame-on-schedule constraint forces

Sources#

Interaction Models: A Scalable Approach to Human-AI Collaboration
How we built a realtime system for responsive voice AI in six months — OpenAI, 2026-07-29 (case-study): audio full-duplex in production; detector removal