Skip to main content

​
Overview

VoxLockFilter is a BaseAudioFilter implementation backed by VoxLock, an open-source (Apache-2.0) voice isolation model. You attach it to a transport’s audio_in_filter, and it keeps the caller’s voice and removes the other people talking in the caller’s room, so VAD and your STT service only hear the caller. It runs locally with onnxruntime on the CPU: no API key and no network calls after the model download. VoxLock locks on to one voice. Until it has heard 3 seconds of the caller’s speech, the filter passes audio through unchanged; it then takes that speech as the voice sample and isolates from then on. If you already have a sample of the caller, pass it in and isolation starts with the first frame.

Source Repository

Code, Pipecat example bot, benchmark harness and changelog

Model

Model card, benchmark results and ONNX files on Hugging Face

PyPI Package

The voxlock package on PyPI

Live Demo

Listen to calls with the filter off and on, in the browser

​
Installation

This is a community-maintained package distributed separately from pipecat-ai:
Requires Python 3.11 or later, like pipecat-ai.

​
Prerequisites

  • No API key. The model (75 MB) downloads from Hugging Face on first use and is cached.
  • CPU. With the default settings the filter uses 2 CPU threads per call. See Choosing the CPU settings.

​
Configuration

​
np.ndarray | None
default:"None"
About 3 seconds of the caller’s voice, 16 kHz mono float32. When given, isolation starts from the first audio frame.
​
float
default:"3.0"
Seconds of speech to collect as the voice sample when voice_sample is not given. Audio passes through unchanged until then.
​
int
default:"80"
Audio is run through the model in chunks of this length. Larger chunks use less CPU and add their length to the delay.
​
int
default:"2"
onnxruntime threads for this filter.
​
str
default:"\"int8\""
Model files: "int8", "int8-full" (fastest), "fp16" or "fp32".
​
str | None
default:"None"
Folder with the ONNX files. None downloads them from Hugging Face.

​
Usage

Create one filter per call and pass it to your transport as audio_in_filter:
The transport drives the filter lifecycle (start(sample_rate), filter(audio), process_frame(frame) and stop()), so no additional wiring is needed. A complete voice agent is in examples/pipecat_bot.py.

​
With a voice sample

​
Choosing the CPU settings

Measured on an 8-vCPU Xeon server; below 1 keeps up with the call. Half a vCPU (Pipecat Cloud agent-1x) is too little.

​
Input Frames

​
Frame
Control frame to toggle isolation on and off at runtime

​
Audio Requirements

  • Input is 16-bit mono PCM at the transport’s sample rate. The filter resamples to 16 kHz for the model and back, so 8 kHz telephony and 48 kHz WebRTC both work.
  • Audio is collected into chunk_ms chunks, so some calls to filter() return no audio; the input transport skips those.

​
Compatibility

Tested with Pipecat 1.12.0. See the changelog for the latest tested version.