ElevenLabs

innFactory AI Consulting from Rosenheim, Germany advises DACH-region enterprises on GDPR- and AI-Act-compliant deployment of voice and music AI. Over 2026 ElevenLabs has grown from a pure text-to-speech vendor into a broad audio stack: Eleven v3 went GA in March, followed in May by Eleven v3 Conversational for voice agents and Music v2. Music v2 reached the API in June, the SynthID watermark went live in June, and since 6 August 2026 Dubbing v2 is available via API. On the speech-to-text side, Scribe v2 and Scribe v2 Realtime have been available since January 2026.

One practical point for many organisations: ElevenLabs is the voice layer behind numerous European voice-agent and phone-assistant products. Anyone running an AI phone assistant often uses ElevenLabs indirectly through their vendor – in which case the data protection chain has to be documented all the way down to the sub-processor, including processing region and retention periods.

What is Eleven v3?

Eleven v3 is ElevenLabs’ current flagship model. It generates natural-sounding speech with high emotional range and contextual understanding in more than 70 languages.

Key innovations

Audio tags

For the first time, emotional and acoustic cues can be steered directly in the text – via tags in square brackets:

[excited] We are thrilled, [whispers] that you're here today.
[sighs] After a long day…
[clapping] Well done!

The model interprets tags such as [excited], [whispers], [sighs], [gunshot], [clapping] or [explosion] and adapts tone and audio accordingly.

Text-to-Dialogue API

The Text-to-Dialogue API lets you generate multi-speaker scenarios (podcasts, audio drama, training videos) in a single call – with natural dialogue dynamics between multiple voices.

Language coverage

70+ languages for Eleven v3 and v3 Conversational – covering all major European languages. Coverage differs substantially per model, though: the docs list Flash v2.5 at 32 and Multilingual v2 at 29 languages, while Scribe v2 and Dubbing v2 are listed at 90+.

Eleven v3 Conversational and Music v2 (May 2026)

Eleven v3 Conversational

With Eleven v3 Conversational, ElevenLabs closes the gap between studio quality (Eleven v3) and real-time latency (Flash v2.5). The model docs list roughly 280 ms latency across 70+ languages. Expressive Mode is on by default for this model: it uses real-time signals from Scribe v2 Realtime – not just the transcript but the prosody – to time responses more naturally. One documented limitation: Professional Voice Clone characteristics are not currently preserved reliably by v3 Conversational; where the cloned brand voice is the decisive criterion, the docs point to the Flash models.

Music v2

Music v2 generates complete songs including vocals and arrangement. New features include section-level inpainting (selectively regenerate verse, bridge or chorus) and mid-track genre switching. Relevant for enterprise and marketing use cases: per the vendor, Music v2 was trained exclusively on licensed data and is cleared for commercial use – a material point when AI music is used in advertising. Since June 2026 the model is available via API (model_id: music_v2), where it works with chunk-based composition plans that give finer control over structure, pacing and arrangement than the pure prompt approach. July added a streaming endpoint that returns the composition plan and optional word timestamps.

New since June 2026

Dubbing v2 via API (August 2026)

Dubbing v2 was initially UI-only and has been available via API since 6 August 2026. The model translates content into 90+ languages while conditioning directly on the original recording, so tone, emotion and delivery carry across. The translation logic is sync-aware: starts and stops follow the original. Regional variants – Castilian versus Latin American Spanish, for example – are distinguished. The API is project-based: transcript and translation are held as editable JSON, so after a correction only the changed segments need re-rendering.

Important for EU projects: per the docs, dubbing is not available in the isolated data residency environments. If you need dubbing together with EU data residency, clarify this with ElevenLabs up front.

Scribe v2 and Scribe v2 Realtime

Two models cover speech-to-text. Scribe v2 is the batch model for long recordings, subtitling and audio analysis: 90+ languages, speaker diarization for up to 32 speakers, entity detection and keyterm biasing for domain vocabulary. Scribe v2 Realtime is the streaming model at roughly 150 ms latency per the model docs; it is the default ASR path in ElevenLabs Agents and supplies the signals for Expressive Mode. In August 2026 entity detection was added to the live stream as well. Lighter realtime variants are also documented (scribe_v2_realtime_turbo, scribe_v2_realtime_lite) – worth checking against the current docs, as availability is still moving.

SynthID watermark (June 2026)

Since 25 June 2026, ElevenLabs embeds the SynthID watermark developed with Google DeepMind into generated audio. It is inaudible and, per the vendor, survives trimming, speed changes, format conversion and metadata removal. The rollout began with text-to-speech for free users and was then extended; audio generated before June 2026 carries no SynthID watermark. A free ElevenLabs Audio Detector is available for verification. Caveat: it only detects watermarks from the ElevenLabs ecosystem, not those of other providers.

ElevenLabs Agents

The ElevenLabs conversational AI platform (also referred to as ElevenAgents) combines four building blocks: a fine-tuned speech-to-text model, a freely selectable LLM (including custom models), a low-latency text-to-speech layer with 5,000+ voices in 70+ languages, and a proprietary turn-taking model for timing.

Channels: telephony via SIP trunking and Twilio (including batch calls), WhatsApp, web widget and React SDK, iOS/Android/React Native, plus a WebSocket API for custom integrations.

Operational building blocks: multi-step workflows with a visual builder, knowledge base with RAG (since August 2026 also as a configurable system tool), tool integration with webhook timeouts up to 300 seconds, configurable guardrails, A/B testing, automated agent tests including tool mocks, transcript search and versioning via CLI and MCP.

Pricing (as of 17 August 2026, per the pricing page): USD 0.08 per minute on all tiers, USD 0.16 per minute in burst mode above the concurrency limit, text messages USD 0.003. LLM and telephony costs are billed separately and are not included in any tier – a point regularly underestimated when costing phone assistants. Included minutes and concurrent calls scale with the tier (Free 15 minutes / 4 concurrent calls up to Business 12,375 minutes / 40 concurrent calls).

Pricing and credits

Subscription tiers per the pricing page (as of 17 August 2026, monthly billing):

ElevenLabs states two free months on annual billing. Credits are drawn from a shared pool across all products: text-to-speech costs roughly one credit per character, Flash v2.5 about half that per the model docs, and speech-to-text 330 credits per minute. For reliable budgeting, verify the actual figures against the current pricing page before signing.

Deprecated models

• eleven_monolingual_v1 and eleven_multilingual_v1: removed on 9 July 2026 – migrate to Multilingual v2 or newer
• scribe_v1: removed on 9 July 2026 – migrate to Scribe v2 or Scribe v2 Realtime
• eleven_turbo_v2 and eleven_turbo_v2_5: listed as deprecated in the docs, successors are Flash v2 and Flash v2.5

If you use ElevenLabs through a third-party vendor, check which model they actually call – deprecated model IDs are a common cause of silent quality changes.

Model selection by use case

GDPR and AI-Act compliance

Data residency: an Enterprise feature, not the default

ElevenLabs offers isolated environments with data residency in the EU, India and Singapore; the default is US infrastructure. The docs list data residency explicitly as an Enterprise feature – without that contract, data is processed in the US default environment regardless of where the customer is located. This is the single most important point for GDPR assessments in mid-sized organisations.

Further constraints worth knowing before an architecture decision:

• Dubbing is not available in isolated environments per the docs
• LLM availability in Agents varies by region and provider
• Existing resources migrate only partially: Professional Voice Clones can be shared into an isolated environment, whereas Agents have to be recreated there via the API. Since August 2026 an additional endpoint replicates voices across data residency environments.

Zero Retention Mode

For particularly sensitive workflows ElevenLabs offers a Zero Retention Mode: request and response data is deleted immediately after processing rather than retained. This mode is likewise restricted to Enterprise customers. Technically it is set via the enable_logging=false query parameter, or per agent in the Agents privacy settings. Per the docs it covers text-to-speech, speech-to-text, text-to-dialogue, voice changer and Agents.

Two practical points: web UI and playground traffic is never covered – if you contract for this mode, usage has to be steered onto the API accordingly. And support is limited under Zero Retention Mode because no diagnostic logs exist; troubleshooting generation issues becomes very hard.

EU AI Act and labelling

• Synthetic audio is subject to transparency obligations under the EU AI Act; the relevant provisions apply from August 2026
• Since June 2026 ElevenLabs embeds the SynthID watermark and provides a free Audio Detector. That is a technical provenance signal, but it does not replace a disclosure directed at the person hearing the audio – an audible or visible notice remains an organisational task
• Audio generated before June 2026 carries no watermark
• Recommendation: an internal policy for labelling AI audio in customer and employee communications, particularly for phone assistants

Voice cloning: consent and abuse prevention

• ElevenLabs uses a Voice CAPTCHA for voice cloning: whoever clones a voice has to read a displayed prompt aloud within a few seconds. That evidences that the requesting person was present during the recording – it does not evidence that the voice is theirs
• The terms prohibit non-consensual cloning, deceptive deepfakes and circumventing verification mechanisms including Voice CAPTCHA
• The consequence for organisations: documented consent from the cloned person remains your own obligation, which the platform check does not discharge
• For brand voices: contractual agreements with voice talent are mandatory
• Licence durability: When a relationship with a voice talent ends, clarify whether existing cloned voices may continue to be used

When ElevenLabs runs via a third-party vendor

Many voice-agent and phone-assistant products in the European market use ElevenLabs as their voice layer without that being visible in the frontend. For data processing purposes this means: your contractual counterparty is the product vendor, and ElevenLabs is a sub-processor. Concretely, clarify which region the vendor calls ElevenLabs in, whether they themselves hold an Enterprise contract with EU data residency and Zero Retention Mode, and how long call recordings and transcripts are retained at both levels. An Enterprise feature at the sub-processor only helps if the vendor has actually purchased it.

Integration into enterprise workflows

• REST API with comprehensive streaming options
• WebSocket streams for real-time conversations and live transcription
• SDKs: Python, JavaScript/Node.js, iOS, Android, React Native, plus a CLI and an MCP server
• Agents: freely selectable LLM backbone, knowledge base with RAG, tool calls, guardrails and automated agent tests

Our recommendation

For high-quality speech output, Eleven v3 is the obvious choice; for phone assistants and voice agents, Eleven v3 Conversational – with the caveat that Professional Voice Clones are not currently preserved reliably there. For transcription we recommend Scribe v2 (batch) or Scribe v2 Realtime (in agents), for multilingual video localisation Dubbing v2, and for commercially usable AI music Music v2 thanks to its licensed training data.

For GDPR-critical applications the path leads through the Enterprise tier: EU data residency and Zero Retention Mode are available there and not in the standard plans. If you only use ElevenLabs indirectly through a voice-agent product, get these points in writing from your vendor before personal conversation data starts flowing. We additionally recommend an internal policy for labelling AI audio – SynthID provides technical verifiability, while disclosure towards affected people remains an organisational task.

Depending on the use case we also evaluate OpenAI gpt-4o-mini-tts (well integrated with the OpenAI stack) or Cartesia Sonic (designed for low latency). Contact us for advice on the right audio model strategy.