Azure Voice Live

​

Overview

​

Installation

uv add "pipecat-ai[azure]"

​

Prerequisites

​

Azure Account Setup

1. Azure Account: An Azure subscription
2. Foundry Resource: An Azure AI Foundry resource with Voice Live available
3. Credentials: An API key for the resource, or a Microsoft Entra ID token provider

​

Required Environment Variables

• AZURE_VOICE_LIVE_API_KEY: API key for the resource
• AZURE_VOICE_LIVE_ENDPOINT: Resource endpoint (for example https://<resource>.services.ai.azure.com)

​

Key Features

• Speech-to-Speech: Audio in and audio out through a single realtime session
• Azure Voices: Standard, custom, personal, and realtime-native voices
• Semantic VAD: Server-side turn detection, including a multilingual variant
• Audio Enhancements: Noise suppression and echo cancellation
• Function Calling: Register functions as for any other LLM service
• Caller Transcription: User turns are added to the context as transcripts

​

Configuration

​

AzureVoiceLiveLLMService

​

Settings

​

SessionProperties

​

Usage

​

Basic Setup

import os
from pipecat.services.azure.voice_live.llm import AzureVoiceLiveLLMService

llm = AzureVoiceLiveLLMService(
    api_key=os.getenv("AZURE_VOICE_LIVE_API_KEY"),
    endpoint=os.getenv("AZURE_VOICE_LIVE_ENDPOINT"),
    settings=AzureVoiceLiveLLMService.Settings(
        model="gpt-4o-mini",
        system_instruction="You are a helpful voice assistant.",
    ),
)

​

Custom Session Properties

from pipecat.services.azure.voice_live.events import (
    AzureStandardVoice,
    InputAudioNoiseReduction,
    InputAudioTranscription,
    SessionProperties,
    TurnDetection,
)

llm = AzureVoiceLiveLLMService(
    api_key=os.getenv("AZURE_VOICE_LIVE_API_KEY"),
    endpoint=os.getenv("AZURE_VOICE_LIVE_ENDPOINT"),
    settings=AzureVoiceLiveLLMService.Settings(
        session_properties=SessionProperties(
            modalities=["text", "audio"],
            voice=AzureStandardVoice(name="en-US-Ava:DragonHDLatestNeural"),
            turn_detection=TurnDetection(
                type="azure_semantic_vad",
                silence_duration_ms=500,
                remove_filler_words=True,
            ),
            input_audio_transcription=InputAudioTranscription(model="azure-speech"),
            input_audio_noise_reduction=InputAudioNoiseReduction(),
        ),
    ),
)

​

Pipeline

from pipecat.pipeline.pipeline import Pipeline
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair

context = LLMContext(
    [{"role": "user", "content": "Say hello."}],
)
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)

pipeline = Pipeline(
    [
        transport.input(),
        user_aggregator,
        llm,
        transport.output(),
        assistant_aggregator,
    ]
)

​

Notes

• Turn taking: The service proposes turn boundaries from Voice Live’s server-side VAD events, which the recommended external user turn strategies resolve into UserStartedSpeakingFrame and UserStoppedSpeakingFrame. LLMContextAggregatorPair detects this realtime service automatically.
• Local VAD: To drive turns from a local VAD (LLMUserAggregatorParams.vad_analyzer), pass turn_detection=None in session_properties. While server-side turn detection is on, the strategies this service recommends replace those the analyzer installs, leaving the analyzer with no say in turn-taking.
• Caller transcription: The caller’s turns reach the context as transcripts. If session_properties omits input_audio_transcription, a warning is logged and those turns are missing from the history, including the history reset_conversation sends to a new session. Set it to None explicitly to opt out of the warning.
• Model selection: model is selected by the connection URL, so it is fixed for the session and a runtime change is reported as unsupported.
• Azure OpenAI Realtime: Use AzureRealtimeLLMService for Azure OpenAI Realtime deployments; Voice Live uses different event names.

​

Event Handlers

@llm.event_handler("on_conversation_item_created")
async def on_conversation_item_created(service, item_id, item):
    print(f"Item created: {item_id}")

@llm.event_handler("on_conversation_item_updated")
async def on_conversation_item_updated(service, item_id, item):
    print(f"Item updated: {item_id}")