Side projectSpecTalk4 min read

SpecTalk: starting software projects by voice from smart glasses

Project ideas tend to show up away from the keyboard and are half gone by the time you sit down. SpecTalk lets you describe a project out loud through Meta Ray-Ban glasses or AirPods: a voice agent called Gervis asks up to three questions, writes the spec and hands it to a coding agent. Built solo in six days.

Two phone screens from the SpecTalk Android app: a live voice transcript with Gervis on the left, and a project plan card with Build it and Change something buttons on the right.

The problem

My best project ideas show up when I'm walking, cycling or standing in the U-Bahn, never when I'm at my desk. By the time I sit down and open an editor, half of the idea is gone. The tools we use to make software make you stop and translate thought into keystrokes, and that translation is slow and loses things.

Voice assistants can take a note, but a note is not a spec, and nothing happens to it until you're back at the keyboard.

What I built

SpecTalk is an Android app plus a backend that lets you describe a project out loud, while walking, and have something real waiting when you get home. It works with Meta Ray-Ban glasses (microphones, speakers and a camera) or AirPods. The voice agent is called Gervis. It was a solo side project: 94 commits, almost all of them between 16 and 21 March 2026, many after midnight. Most were written together with Claude Code, and the commit messages say so.

Dot plot of every commit by day and hour, with a cluster between midnight and 6 AM — Every commit from 16 to 21 March, by hour. The shaded band is midnight to 6 AM. Friday the 20th has 38 commits on its own.

What it does

Wake word. You say "Hey Gervis" with the screen off and the phone in a pocket, and a chime tells you he's listening.
Short interview. You describe the idea. Gervis asks one clarifying question at a time, three at most.
Plan card. He puts a project plan on the screen and asks whether to go ahead. You say yes or tap "Build it", put the phone away and keep walking.
Build and report back. A coding agent builds the project in the background. If your session is still open, Gervis tells you out loud when it's done; if not, a push notification arrives.
Memory and camera. A per-user project registry lets you come back to an earlier project by name and ask for changes, and Gervis can take a photo through the glasses camera and describe what you're looking at.

Two phone screens: a voice transcript with Gervis and a project plan card — Recreated from the Compose source (VoiceSessionScreen and PrdConfirmationCard) with sample data. The backend isn't running anymore, so these are rebuilt, not captured.

Why these tools

Gemini Live, through Google ADK, on a Cloud Run backend. The phone never talks to Gemini directly. It records audio from the glasses or AirPods, streams it over a WebSocket to a FastAPI backend, and plays back what comes down. That keeps API keys out of the app (they live in Secret Manager, and per-user credentials are encrypted in Neon Postgres). It also means the backend holds the live conversation, so it can drop a short message into the session when a job finishes, and Gervis says it out loud mid-conversation. The cost is a Cloud Run instance that has to stay up, with a one-hour request timeout, so a live session doesn't get cut mid-sentence.

Architecture diagram: wearables to Android app to a Cloud Run backend, which talks to Gemini Live, Neon, Cloud Storage and OpenClaw — Rebuilt from the README's ASCII diagram. The glasses stream through the phone, the phone streams to the backend, and only the backend talks to Gemini.

Vosk for the wake word. "Hey Gervis" had to work without sending my whole day to a server, so the wake word runs offline on the phone, in a foreground service with a tiny grammar.

Cloud Tasks and OpenClaw for the building. Gervis doesn't write the code himself. When you confirm the plan, the backend queues a Cloud Tasks job that sends the spec to OpenClaw, which runs a coding agent (Claude Code or Codex CLI) on a machine I control, and streams the result back. Firebase sends the push notification when you've already hung up, and Cloud Storage holds project artifacts.

What was hard

Audio, much more than agents. For two nights Gervis kept answering himself: the phone's microphone picked up his voice and sent it back as if I had said it. One cause was a duplicate environment variable that made the backend open two Gemini connections per session. The other was on the phone. I was playing his voice as media audio, and Android's echo cancellation only works when it knows what's coming out of the speaker. Marking the audio as voice communication fixed it.

The other problem was quieter. You'd ask Gervis to build something, he'd say "Initiating the code generation", and nothing would happen. The model was describing the tool call instead of making it. A much longer system prompt, forcing the tool calls and moving to the newer native-audio Live model fixed it, and I added a log line to every coding tool so I could check that it had really been called.

What it became

On my phone it worked end to end. Say the wake word, talk through the idea, answer Gervis's questions, look at the plan card, tap "Build it", and hear him say the project is ready.

There is no live version you can try today, and the repo is honest about what was left. KNOWN_ISSUES.md still has a high-priority bug: when the backend is redeployed, Cloud Tasks retries jobs that were killed mid-run, the retried job sends a push, and the phone auto-opens the voice session. So on every deploy, Gervis would chime and start talking without anyone calling him. The backend half is fixed with a guard against duplicate jobs. The Android half, making auto-open opt-in, never got merged.

SpecTalk also has a sibling on AWS that I worked on in the weeks around this one, with a council of specialist agents writing the spec and several Claude Code agents building in parallel.