Speaking Observer

​

Overview

• Speech detection (user_speech_*): What the voice activity detector heard, including speech that never becomes a turn (coughs, false starts, pauses mid-sentence)
• Turn decisions (user_turn_*): The turn strategy’s ruling on that speech, which is what the rest of the pipeline acts on

​

Events

​

on_speech_event

​

Usage

​

Basic Setup

from pipecat.observers.speaking_observer import SpeakingObserver

observer = SpeakingObserver()

@observer.event_handler("on_speech_event")
async def on_speech_event(observer, event):
    logger.info(event.model_dump_json())

worker = PipelineWorker(
    pipeline,
    observers=[observer],
)

​

Logging Speech Intervals

@observer.event_handler("on_speech_event")
async def on_speech_event(observer, event):
    if event.kind == "bot_speech_stopped" and event.started_at:
        duration = event.timestamp - event.started_at
        logger.info(f"Bot spoke for {duration:.2f}s")

​

Separating Detection from Turn Strategy

@observer.event_handler("on_speech_event")
async def on_speech_event(observer, event):
    if event.kind == "user_speech_started":
        # Raw speech detected
        logger.debug("User audio detected")
    elif event.kind == "user_turn_started":
        # Turn strategy confirmed this is a user turn
        logger.info("User turn started")

​

Building a Timeline

events = []

@observer.event_handler("on_speech_event")
async def on_speech_event(observer, event):
    events.append(event)

    # Later: detect interruptions by checking if user started
    # while bot speech was still open

​

Configuration

​

Constructor Parameters

​

Speech Event Kinds

• user_speech_started / user_speech_stopped: Speech as the voice activity detector heard it
• user_turn_started / user_turn_stopped: Turn strategy’s ruling on that speech
• bot_speech_started / bot_speech_stopped: Bot speaking lifecycle
• interruption: An interruption occurred (any processor can trigger one)

​

Notes

• Accurate timing: Speech is timed to when it began and ended, not to when the detector confirmed it. An interval drawn from these timestamps matches what was actually said.
• Self-contained records: Moments that close a stretch of speech include started_at, so you can read duration from a single record without pairing it with the opening moment.
• No turn definition: What counts as a turn stays with the reader. A turn built into the records would freeze one definition into every event, where the moments can be grouped differently later.
• Duplicate prevention: Each frame is reported once, even if relayed through multiple processors. Broadcast interruptions (which arrive as two frames) are reported once.
• Open stretches: A stretch whose closing moment never arrives stays open rather than quietly joining itself to the next one.