Interesting write up, I've been vibe coding an android voice assistant using a similar pipeline without a wakeword - VAD > ASR > LLM > TTS
I've found parakeet to be much, much better than whisper and I made a 4 bit quant of parakeet 110M for my crappier android test devices, it would probably fit a raspberry pi:
The model is actually a distillation of a different checkpoint called parakeet-ultra, same size and architecture as parakeet-tdt-0.6b-v3 but better, that's why the 1.58bit quant works so well.
older amazon echo dots and shows really make great hardware for this, combined with homeassistant, and can be had for very little cost. e.g. shameless plug: https://github.com/ygelfand/echolocal
I got an echo show rooted and ROMd that I am loving - put it in the kids room as a music player in kiosk mode, but have been now wanting one for myself with wakewords. This looks like the ticket.
Interesting read! I have my own local voice assistant and I implemented exactly the same question mark trick just the other day.
The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.
Home assistant doesn't really provide much out of the box, its mostly a modular/pluggable ecosystem and one of the things you can install are various voice assistants
I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.
I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.
Interesting write up, I've been vibe coding an android voice assistant using a similar pipeline without a wakeword - VAD > ASR > LLM > TTS
I've found parakeet to be much, much better than whisper and I made a 4 bit quant of parakeet 110M for my crappier android test devices, it would probably fit a raspberry pi:
https://huggingface.co/mewse/parakeet-tdt_ctc-110m-onnx-int4
There is also parakeet-redux, a 1.58bits quant of parakeet-tdt-0.6b-v3 that runs as well as the original model: https://huggingface.co/moondream/parakeet-redux
The model is actually a distillation of a different checkpoint called parakeet-ultra, same size and architecture as parakeet-tdt-0.6b-v3 but better, that's why the 1.58bit quant works so well.
Oh crazy I hadn't seen this yet, I might be able to unify my 0.6B vs 110M selector, thanks!
older amazon echo dots and shows really make great hardware for this, combined with homeassistant, and can be had for very little cost. e.g. shameless plug: https://github.com/ygelfand/echolocal
I just commented in another thread that I built this: https://github.com/nlothian/earwing
Beam forming microphone array, custom UI, custom case. Maybe costs more than a second hand echo dot though (hardware is probably around $30).
I got an echo show rooted and ROMd that I am loving - put it in the kids room as a music player in kiosk mode, but have been now wanting one for myself with wakewords. This looks like the ticket.
Interesting read! I have my own local voice assistant and I implemented exactly the same question mark trick just the other day.
The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.
Just what I want to read — favourited to come back to!
Is this complimentary to, or an alternative to Home Assistant?
Home assistant doesn't really provide much out of the box, its mostly a modular/pluggable ecosystem and one of the things you can install are various voice assistants
This is really useful, thanks! The other day I bought a small screen and didn't know what to do with it so I made an assistant that talks to my bot:
https://m.youtube.com/shorts/3FseVyAG5uM
I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.
I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.