Walk up to Lamar in Grand Theft Auto V, say something rude about his hoodie, and he roasts you back in his own voice about a second later. A cop tells you to file a civil claim and flips you off. Michael defends his robe as if it were a business suit. Nobody scripted those lines. A new open-source project called NPCTalk writes them live, in character, from whatever the player says into a microphone.
What it is
NPCTalk, from the Universal Modder project, is a set of skills, a self-hosted voice server and command-line tools that let an AI coding agent wire talking characters into a game. Each character answers in a cloned voice from Fish Audio’s Drama 3 model, or S2 Pro, and can act on what the player said. The code is MIT licensed. Most of it does not know it is inside GTA, which is the point: the skills describe what a game has to provide and how to build the rest, so the same pieces should work elsewhere.
The GTA V demo
GTA V was the first target. The example mod covers 19 characters and includes an open mic, character homes, actions and a director tool for recording clips. It needs Windows and the .NET SDK, version 8 or higher, to build.
How the agent builds it
An agent starts with a skill called make-npcs-talk and runs the same loop every time. It first scouts the game for its version, how code runs inside it and whether it is single-player. Then it picks engines for voice, language model, speech-to-text and actions. The voice server goes up first, outside the game, and gets tested with the command npctalk say. Reference voices come from recordings the user has the rights to. Then comes one NPC answering typed text, then push-to-talk and open-mic input, then personas, their own lines and scene facts. The last step is verification in the real game with every step’s timing logged.
The credits list the stack: Cerebras for the lines, NVIDIA Nemotron ASR on fal for speech-to-text, and Fish Audio voices served on Modal and fal. Users need accounts there, plus Python 3.10 or newer, Git, uv and ffmpeg. A Claude Code plugin is available, and the toolkit also ships a standard command-line tool and an MCP server.
Guardrails
The project says it is meant for single-player games or servers you run. It will not inject into online games with anti-cheat, and the documentation warns against launching GTA Online with a script hook installed. It will not bypass anti-cheat, DRM or ownership checks, and it blocks committing or publishing voice clips, cloned voices or game audio. Keys stay in environment variables. The agent asks before taking over keyboard and mouse, installing into game folders, keeping a GPU running or publishing anything.
Voice quality gets measured, not assumed. The documentation tells users with a thin-sounding clone to record five 8-second clips of NPCs arguing during normal play, then test whether the reference clip or the model is at fault. On one character, Lamar, Drama 3 and S2 Pro tied on likeness without training; Drama 3 handled crying and laughing better and began talking sooner.
Sources
- GitHub – christos-antono/NPCTalk: Make any NPC in any game talk back with Fish Audio: skills, a self-hosted voice server and tools for AI coding agents, with a full GTA V example.
- npctalk
- npctalk
Photo: Brian Wong (CC BY-SA 2.0)
