Desktop Virtual Assistant
An AI desktop companion — an always-on animated character you talk to by voice.
[ The transformation ]
Desktop AI is a chat box you type into. Nothing that lives on screen, remembers you, and actually operates the machine.
A companion you speak to naturally that remembers context across sessions and can act on the computer, not just chat.
[ The hard problem ]
Gluing a responsive animated character, a live voice loop, persistent memory and OS-level control into one app that doesn't lag — each subsystem has different latency requirements and they all fight for the main thread.
[ How it works ]
I built an Electron app with an always-on-top, draggable animated character (idle/talking/thinking/listening states), wake-word activation, and a full real-time voice loop — Whisper speech-to-text → Claude → ElevenLabs speech — wired to a memory system that learns your preferences and to OS-level control (open apps, search files, run commands). The hard part was gluing a responsive character, a live voice loop, local memory and machine control into one thing that doesn’t lag.
[ Key features ]
Always-on-top draggable animated character with idle/talking/thinking/listening states
Wake-word activation for hands-free interaction
Real-time voice loop: Whisper STT → Claude → ElevenLabs TTS
Persistent memory system that learns user preferences across sessions
OS-level control — open apps, search files, run terminal commands
Sprite animation system with smooth state transitions
[ The result ]
