SEC 03 / TRANSPORT PATTERNS
The six transport patterns
ElevenLabs has several mechanisms that are all casually described as “streaming,” but they solve different problems. Before touching any SDK code, it’s worth fixing these six patterns in your head — almost every design question later on (which package, which auth model, which latency number to expect) reduces to “which of these six am I building.”
| Pattern | Input pattern | Output pattern | Best use |
|---|---|---|---|
| Ordinary REST | Complete request | Complete response/file | Batch generation, simple integrations |
| HTTP chunked streaming | Complete text/request | Audio bytes arrive incrementally | Low time-to-first-audio when all text is known |
| TTS WebSocket | Incremental text | Incremental audio and optional alignment | LLM token streams, dynamic text |
| Realtime STT WebSocket | Incremental audio | Partial and final transcripts | Live captions, dictation, transcription |
| Agents over WebRTC | Live microphone and conversation events | Full duplex audio | Conversational voice applications |
| Speech Engine WebSocket | ElevenLabs connects to your server | Your server streams LLM output | Keep your own LLM logic while using ElevenLabs voice orchestration |
1. Ordinary REST
Use ordinary REST when the full request is already available and first-byte latency is not especially important. It is the easiest option to cache, audit, test, retry, and store.
Examples include:
- generating an MP3 from a complete passage;
- transcribing an uploaded recording;
- managing voices;
- creating pronunciation dictionaries;
- inspecting agent configurations;
- starting dubbing or music jobs.
2. HTTP chunked streaming
The streaming TTS method sends a complete text request but returns audio progressively through HTTP chunked transfer — complete text goes in, audio bytes come out incrementally. ElevenLabs documents chunked streaming for text-to-speech, Voice Changer, and Audio Isolation.
This is generally the best option when:
- the complete text is already known;
- playback should begin before the full file is generated;
- you do not need to send later text fragments into the same generation;
- ordinary HTTP infrastructure is preferred over a long-lived socket.
It is simpler than the TTS WebSocket and often the correct default for web backends.
3. TTS WebSocket
The TTS WebSocket endpoint accepts partial text over time and returns audio as it becomes available. It can also return character- or word-alignment information. It is intended for cases such as streaming output from an LLM.
The key nuance: it is not automatically faster in every situation. When the complete text is already available, ElevenLabs recommends ordinary streaming instead, because WebSocket text buffering and generation-trigger logic can add complexity and sometimes latency. The WebSocket is most valuable when the text itself is not yet complete.
Important TTS WebSocket details include:
- configurable inactivity timeout;
- client-side authentication using a short-lived or single-use credential;
- alignment events;
- audio output formats including telephony-oriented formats such as μ-law;
- generation triggers and automatic buffering modes.
4. Realtime speech-to-text
Realtime STT uses a separate WebSocket. The application sends base64-encoded audio chunks and receives events representing:
- partial transcripts;
- final transcripts;
- committed transcripts;
- word or token timing data;
- VAD- or manually committed segments.
The endpoint
supports formats including PCM at several sample rates and ulaw_8000,
which is particularly relevant to telephone audio.
The browser-facing Scribe SDK can obtain microphone audio directly and supports both microphone-driven and manually supplied chunks. Client-side connections should use a single-use token rather than exposing the API key; the documented Scribe token lifetime is 15 minutes.
5. Agents over WebRTC
Agents SDKs use WebRTC, backed by LiveKit, for low-latency bidirectional voice. They add far more than a transport:
- microphone and speaker management;
- conversation state;
- user and agent speaking state;
- interruption handling;
- client tools;
- audio-device selection;
- public and private agent authentication;
- feedback and event hooks.
The TypeScript Agents monorepo describes WebRTC audio, lifecycle events, client tools, authentication, and device controls as core functionality.
6. Speech Engine
Speech Engine reverses the usual connection direction: ElevenLabs establishes a WebSocket connection to a public endpoint operated by your application. Your endpoint streams LLM output back to ElevenLabs, which handles speech production and conversational behaviour.
This is useful when:
- you already have an LLM orchestration layer;
- proprietary business logic should remain on your server;
- ElevenLabs should manage speech, interruptions, and delivery;
- you do not want to recreate the full voice session protocol.
Speech Engine uses event identifiers to make cancellation and barge-in manageable — stale output associated with interrupted events can be discarded. Authentication uses a short-lived JWT carried in the Speech Engine authorization header, and the SDK verifies the signature by default.
Choosing between them
A compact restatement of the decision:
- Full text known, low time-to-first-byte matters → HTTP chunked streaming.
- Text arrives incrementally (e.g. from an LLM token stream) → TTS WebSocket.
- Conversational voice, microphone in / audio out → Agents over WebRTC.
- You already run your own LLM and want ElevenLabs to just handle voice → Speech Engine.