Technology1 min read

Rebuilding my local voice assistant pipeline

By · Published by Everything Blog

In short

The developer revamped the Platypush assistant's capture mechanism by transitioning wake word detection to microWakeWord, which is also utilized by ESPHome and Home Assistant. For end-of-speech tracking before Whisper transcription, the system swapped unreliable WebRTC VAD processing for Silero VAD running via onnxruntime. The updated preprocessing layer incorporates hysteresis and sliding windows, while retaining root-mean-square detectors and WebRTC VAD as secondary fallbacks.

Key points

  • Transitioning to microWakeWord resolves distance and noise issues while maintaining famil…: Transitioning to microWakeWord resolves distance and noise issues while maintaining familiar API configuration parameters.
  • Switching to Silero VAD provides more dependable voice activity tracking prior to running…: Switching to Silero VAD provides more dependable voice activity tracking prior to running Whisper speech-to-text.
  • Audio devices are now automatically released whenever an active user conversation begins…: Audio devices are now automatically released whenever an active user conversation begins to optimize hardware usage.

Sources checked by our analysis pipeline: 1

Rebuilding my voice assistant pipeline

Swapping openwakeword for microwakeword and making voice detection smarter

I wrote an article a couple of months ago describing how I rebuilt my old voice assistant pipeline in Platypush.

And I wrote earlier articles describing my journey through self-hosted voice assistants - in 2024 (with Picovoice), in 2020 (diving into the legacy assistant.google

integration, the now defunct assistant.alexa

integration, the now defunct Snowboy integration, the now defunct DeepSpeech integration and Picovoice) and the original article in 2019 describing the Google Assistant integration.

For a couple of months I used the voice pipeline described in my latest article as my main driver, but after getting used to the robustness of commercial voice assistants I was unhappy with the high rates of false positives/negatives of my self-hosted pipeline, as well as its lack of robustness in detecting start/end of speech, especially in noisy environments and using cheap microphones.

After a bit of experimentation, I decided to swap and fine tune a few parts, and after a couple of days of trying the results are satisfying enough for…

Technology