Most AI chatbots are just conversational dead-ends: you type a question, get a block of text, and then have to do all the actual work yourself. When I build virtual assistants, I take a fundamentally different approach. To me, a real virtual assistant is not a chat toy—it is an autonomous, tool-wielding extension of your workflow.
1. Tools Over Talk (Agentic Architecture)
The core of an effective assistant is agency: the ability to observe state, reason about goals, select tools, execute actions, and verify outcomes. Instead of keeping the language model locked in a walled garden, I wire it directly into real systems via structured protocols like the Model Context Protocol (MCP) and custom REST/WebSocket integrations.
Whether it is querying a calendar, checking email threads, reading files from Google Drive, searching code repositories, or triggering automation workflows in n8n—the assistant has concrete tools to get jobs done end-to-end.
2. Grounded in the Physical World
Software is much more exciting when it interfaces with real hardware. Coming from an embedded engineering background (ESP32, ESPHome, MQTT, and Home Assistant), my assistants are physically aware. They don’t just know what’s on a screen; they understand real-world telemetry:
- Is someone currently home?
- What is the temperature and power consumption across different rooms?
- Which displays or media players are active?
This physical grounding turns an abstract assistant into an ambient, context-aware co-pilot that adapts to what is actually happening in your environment.
3. Local-First and Privacy-Driven
Personal data belongs to the person who created it. My assistants run on self-hosted infrastructure (dedicated home servers like ceto running Docker, local database backends, and reverse proxies). Sensitive credentials, telemetry data, and private context stay local.
When external APIs—such as Google Workspace—are leveraged, access is tightly scoped, authenticated via OAuth with strict credential hygiene, and used strictly to deliver the specific feature requested. No telemetry selling, no third-party data harvesting, no clutter.
4. Engineering for Determinism
Large Language Models are probabilistic by nature, but systems engineering requires determinism. The real challenge of assistant development is building robust guardrails:
- Schema Enforcement: Strictly typed function definitions and validation so the model cannot hallucinate malformed parameters.
- Defensive Execution: Always checking whether an action succeeded before reporting completion to the user.
- Self-Correction Loops: Giving the agent feedback when an operation encounters an error so it can adapt and recover gracefully.
The goal isn’t just to build an AI that can chat—it’s to build an assistant that you can trust to reliably get things done.
Marc Spoorendonk
Interested in how data is handled? Read our Privacy Policy.