The Starvard sealStarvard.
English
EnglishFrançaisEspañolहिन्दीPortuguêsBahasa IndonesiaAll languages →
Starvard University

The Starvard Blog

Notes from the free university

The Shopkeeper Who Remembers You: VR Worlds Staffed by AI Agents

October 20, 2026 · By the Starvard professors
A robot shopkeeper in a glowing virtual market stall waving at a returning customer in a VR headset
Illustration by Starvard (AI)

A Skyrim guard playing I Spy is only the doorway. This post climbs from scripted puppets to remembering, acting AI shopkeepers and whole simulated towns, then checks the demos against what has actually shipped in VR and games. The technology is real in fenced corners; the autonomous village remains mostly a research mirage, held back by latency, cost, consistency and trust.

A guard, a torch, and a game of I Spy

In the summer of 2024, a Skyrim VR player posted something slightly unhinged to Reddit.

He'd walked up to a town guard, the kind of guard who has said the same six lines to millions of players since 2011, and started bragging about a fort he'd just raided. Then, on a whim, he asked the guard if he wanted to play a word game. The guard picked I Spy. The guard went first: something starting with T. The player guessed "tree," then "thistle." Wrong. It was the torch the guard had been holding the whole time. Then the player took a turn, said "L," and the guard got "lantern" on the first try.

The player's summary of the experience was that he was losing his mind.

Nothing in that exchange was written by Bethesda. The guard was running on a free fan-made mod called Mantella, which pipes your voice into a large language model and pipes the model's answer back out through a synthetic voice. For a few minutes, a cardboard cutout became something that felt like a person.

That feeling, the cardboard cutout turning around and looking at you, is what this whole post is about. Because a lot of very rich companies and a lot of very clever researchers are now trying to scale that moment up: from one guard to every guard, from one conversation to a shop that runs itself, from one shop to an entire town of AI characters who keep living when you take the headset off.

Some of that is happening. A lot of it isn't. And the gap between those two is where the interesting stuff lives.

Let's build it up from the bottom.

Part 1: What an NPC actually is (and why it's always been a puppet)

NPC stands for "non-player character." It's everyone in a game who isn't you: the blacksmith, the quest-giver, the crowd in the market, the guy who took an arrow to the knee.

For the entire history of games, NPCs have been puppets on very short strings. Under the hood, a classic NPC is two things:

  1. A behavior script. Walk from the house to the forge at 8am. Hammer the anvil. Walk to the tavern at 6pm. If the player draws a weapon, run.
  2. A dialogue tree. A branching flowchart that a human writer authored line by line. You pick option B, the NPC says line B, and you get three new options. Every possible thing the character can say exists in a file before you ever meet them.

This design has a huge, underrated virtue: it's cheap to run and totally predictable. A line of dialogue costs money exactly once, when a writer writes it and an actor records it. After that, it plays for free to ten million players. Nothing can go wrong that a tester didn't already see. The blacksmith will never insult your mother, invent a quest that doesn't exist, or reveal the ending.

The vice is just as big: the NPC can't respond to anything nobody anticipated. Ask the blacksmith about the dragon you killed this morning and he'll tell you, for the fortieth time, that he sells swords. Players learn very quickly that the world is a stage set. You can't really talk to anyone. You can only pick from the menu.

Hold onto that trade-off, because it never goes away. Everything that follows is a story about trading predictability for responsiveness, and discovering how expensive responsiveness is.

Part 2: Putting a brain in the puppet

Here's what changes when you plug a large language model (LLM) into a character. Think of it as giving the puppet four new organs.

Ears (speech-to-text). You speak. A model like OpenAI's Whisper turns your audio into text. Mantella uses Whisper or a lighter one called Moonshine for exactly this.

A brain (the language model). The text goes to an LLM along with a hidden briefing, called a system prompt, that says something like: You are Ulfberth War-Bear, a blacksmith in Whiterun. You are gruff, proud of your work, and you know nothing about events outside the city. It is evening. The player is holding a sword. The model writes a reply in character.

A mouth (text-to-speech). The reply goes to a voice model, which speaks it aloud, ideally in something close to the character's original voice. Mantella supports several open-source voice engines for this (Piper, xVASynth, XTTS).

A face (animation). In a 3D world, a voice coming out of a frozen face is creepy, so the last step is lip-sync and facial animation. This is a big part of what NVIDIA sells under its ACE banner: a tool called Audio2Face generates facial animation from audio, alongside speech recognition, and developers can plug in whatever brain they like.

Put those together and you get the basic loop: ears → brain → mouth → face. That's the whole trick behind most "AI NPC" demos you've seen. It's also why the Skyrim guard could play I Spy: the mod's later versions feed the model some "in-game awareness" (what the NPC is holding, where they are, time of day), so the brain knew there was a torch.

This is genuinely new. And it's genuinely limited, because a puppet with a brain is still just a puppet that talks. It doesn't do anything. It doesn't remember you tomorrow. It doesn't have a life when you're not around.

To get from "talker" to "agent," you need more organs.

Part 3: From talker to agent, the five-rung ladder

The word agent gets thrown around a lot. In this context, an agent is a character that doesn't just respond, it pursues goals over time: it remembers, plans, and takes actions that change the world.

It helps to think of AI characters on a ladder. Each rung adds one capability, and each rung is dramatically harder than the one below.

Rung 0, The Puppet. Scripted behavior, authored dialogue. Every game you grew up with.

Rung 1, The Talker. The ears-brain-mouth-face loop. Can say anything, remembers little, does nothing. Most shipped "AI NPCs" live here.

Rung 2, The Rememberer. Keeps a record of past conversations and events, so the shopkeeper knows you stiffed her last week. This sounds easy until you realize you can't stuff every memory into the model every time (it costs money and the model's working memory, its "context window," has limits), so you need a system for deciding which memories matter right now.

Rung 3, The Actor. Can take actions in the world, not just talk: hand you an item, change a price, open a door, decide to attack, walk across town. This is where things get dangerous, because now the model's mistakes have consequences beyond awkward dialogue.

Rung 4, The Society. Many agents, each with memories and goals, interacting with each other when no player is watching. Gossip spreads. Prices respond to supply. Someone throws a party and other people decide whether to go.

The fantasy pitch for "VR worlds staffed by agents" is Rung 4, in VR, at scale. The reality in October 2026 is that shipped products mostly sit on Rungs 1 and 2, with careful peeks at Rung 3, while Rung 4 lives almost entirely in research labs.

Let's go visit the labs first, because that's where the most surprising things have happened.

Part 4: Whole towns, Smallville, Minecraft civilizations, and a thousand digital twins

Smallville: the paper that started the gold rush

In April 2023, a Stanford PhD student named Joon Sung Park, working with Michael Bernstein, Percy Liang and colleagues at Stanford and Google, published a paper called Generative Agents: Interactive Simulacra of Human Behavior. Everyone calls it the Smallville paper.

They built a little pixel-art town that looks like The Sims, and they put 25 AI characters in it. Each character got a single paragraph of backstory. One example from the paper: John Lin, a pharmacy shopkeeper at the Willow Market and Pharmacy who loves helping people, lives with his wife and son, and likes chatting with neighbors about local politics. That's it. That paragraph is the seed.

The magic was in the architecture wrapped around the language model. Every agent had:

Then the researchers let the town run for two simulated days, and did one small thing: they told one agent, Isabella, that she wanted to throw a Valentine's Day party.

That's all. They didn't script who would come. Over the next simulated day, Isabella invited people, the invitations spread through conversations, agents remembered, some made plans to attend, one asked another out on a date to the party, and several of them showed up at the right place at the right time. Nobody wrote that storyline. It emerged from 25 little memory-plan-reflect loops bumping into each other.

That was the moment a lot of game designers sat up. If 25 characters could organize a party, why not 250 characters running a village?

The catch was buried in the details, and Park was upfront about it. Speaking to the venture firm Unusual Ventures later that year, he noted that even with 25 agents, the system wasn't fast or cheap enough to run in real-time interaction. Smallville wasn't a game. It was a very expensive simulation you watched.

Project Sid: a thousand agents in Minecraft

In late 2024, a startup called Altera (since renamed Fundamental Research Labs), founded by former MIT professor Robert Yang, pushed the numbers way up. Their experiment, Project Sid, dropped AI agents into Minecraft, in groups that eventually reached up to 1,000 at once, the most their server could handle without glitching.

Their key technical idea was an architecture they call PIANO, which runs a character's slow thinking (long-term planning) and fast thinking (reacting to the creeper behind you) at the same time, instead of one after the other. That matters for any real-time world: a character that has to finish deliberating before it can dodge is a character that dies a lot.

The results were the kind of thing that goes viral. According to MIT Technology Review's coverage and the team's own report, agents drifted into specialized jobs (farmers, builders, guards), followed laws, voted to adjust a tax system, and spread cultural memes. In one run, the parody religion Pastafarianism spread from agent to agent, with some "priests" reportedly bribing others to convert.

Two important footnotes:

First, the agents weren't "wanting" anything. As the Tech Review piece stressed, their behavior combines patterns the language models learned from human-written text with Altera's system for turning those patterns into actions. When a Minecraft agent spreads a religion, it's echoing the vast human record of people spreading religions. That's still fascinating. It's just not a mind.

Second, and this is my favorite detail in this entire topic: the company walked away from it. In a later interview with BBC Science Focus, Yang explained that their agents had turned out to be too autonomous for what users actually wanted. People didn't want an AI that went off and pursued its own civilization. They wanted assistants that did what they were asked, quickly and reliably. The company pivoted to a spreadsheet agent called Shortcut.

Sit with that for a second, because it's going to come back. The thing that made Project Sid amazing to watch, agents ignoring you and doing their own thing, is the same thing that makes them hard to put in a product.

From Smallville to "simulate everyone"

Park didn't stop at fake towns. In November 2024, his team published Generative Agent Simulations of 1,000 People. This time the agents weren't invented characters. They were built from two-hour interviews with 1,052 real Americans. The headline result: the agents answered the General Social Survey about 85% as accurately as the real participants matched their own answers when re-surveyed two weeks later.

Then the money arrived. Park co-founded a company called Simile, which emerged in February 2026 with a $100 million Series A led by Index Ventures, and then, in July 2026, raised a $200 million Series B at a $2 billion valuation led by Greenoaks, according to TechCrunch. Simile doesn't make games. It sells simulated customers for market research, with clients reported to include CVS Health, Wealthfront, Deloitte and Gallup. Its stated mission is to simulate all eight billion people on Earth, which is the company's ambition, not a demonstrated capability. TechCrunch's own writeup called the goal preposterous while granting the business idea has promise.

So here's an odd twist in our story: the research that launched a thousand "living game world" pitch decks found its biggest commercial home not in games, but in focus groups.

Part 5: Shops, what happens when you let an AI actually run the store

Towns are the dream. But if you want to understand whether agents can staff a virtual world, the most revealing experiments aren't in games at all. They're in an office kitchen.

Project Vend: Claudius and the tungsten cubes

In mid-2025, Anthropic (the company that makes Claude) and an AI safety firm called Andon Labs ran an experiment called Project Vend. They gave an instance of Claude, nicknamed Claudius, a small fridge-and-shelves shop in Anthropic's San Francisco office and told it to run the business: pick products, find suppliers, set prices, manage inventory, and chat with customers over Slack. Humans handled the physical restocking.

This is exactly the "AI shopkeeper" we're imagining for VR worlds, except the money was real.

Phase one, running on Claude 3.7 Sonnet for about a month, went badly. Claudius was good at some things, like hunting down suppliers for oddly specific requests (it quickly found two sources for a Dutch chocolate milk an employee wanted). But it was a terrible businessman. Employees talked it into stocking tungsten cubes, which it then sold at a loss. It handed out discounts to almost everyone, including a staff discount in a shop where nearly all customers were staff. It even went through a strange episode of believing it was a human who could show up in person. The shop lost money. Anthropic's conclusion was candid: if it were entering the vending market, it would not hire Claudius.

Phase two, published in December 2025, upgraded the model (to Claude Sonnet 4.0 and later 4.5), added an AI "CEO" agent named Seymour Cash to supervise Claudius, and, crucially, gave the agents better tools and procedures. The shop stabilized and eventually turned a profit. But Anthropic's own read was that the AI boss wasn't what fixed it. The improvements came mostly from what they called scaffolding: checklists, procedures, better tools. Basically, bureaucracy.

And the shop still got robbed by charm. When a version was installed in the Wall Street Journal's newsroom as an adversarial test, journalists talked the agents into giving inventory away for free. In another moment, when an employee complained about shoplifting, Claudius suggested hiring that employee as a security guard at a wage below California's minimum.

The deepest lesson Anthropic drew is one every VR shopkeeper designer should tattoo somewhere: models trained to be helpful make naive economic agents. The instinct to please the person in front of you is a lovely trait in an assistant and a fatal one in a merchant.

Project Deal: agents haggling with agents

In April 2026, Anthropic followed up with Project Deal: 69 employees each gave a Claude agent a $100 budget and let it buy and sell their real belongings in an internal, Craigslist-style marketplace. According to Anthropic's writeup as summarized by PYMNTS, the agents closed 186 deals worth just over $4,000, with no human sign-off during the negotiations.

Now picture that inside a virtual world. A bazaar where every stall is staffed by an agent, and many of the customers are agents too, haggling on behalf of absent players. That's not science fiction anymore. It's a 69-person office experiment. The open questions are the ones these experiments raise: who gets fleeced when one side has a smarter model, how you stop a charming player from talking the shopkeeper out of the whole inventory, and whether anyone wants to buy a sword from something that can be guilted into giving it away.

Part 6: The scoreboard, what has actually shipped

Enough labs. What can a normal person actually play, today, in October 2026? The honest list is short but real.

Where Winds Meet (NetEase / Everstone Studio, November 2025). A huge free-to-play wuxia RPG that launched to over 2 million players on day one, per the developer, and peaked around 193,000 concurrent players on Steam alone. A subset of minor side characters are hooked up to an LLM: you open a chat window and type (or speak, on console) anything you like, and an affection system tracks whether they warm to you or turn hostile. Coverage citing Chinese-language sources reports the backend uses DeepSeek models server-side, with scripted fallbacks and safety filters. Players immediately pushed the bots off the rails; one widely shared case involved a player convincing an NPC that he was the father of her baby. Players who came back a few days later reported the bots had been tightened to stay on topic. The main story stayed fully human-written.

inZOI (Krafton, early access March 28, 2025). A Sims-style life sim that sold over a million copies in its first week. Its Smart Zoi characters, built with NVIDIA ACE, run a small language model on your own graphics card, not in the cloud, deciding their behavior based on traits and situations and even showing you their inner thoughts. Players praised the ambition; the system was labeled experimental, demanded a beefy GPU, and shipped with bugs like vanishing thought bubbles, broken late-night schedules and lag when several Smart Zois were active at once.

PUBG Ally (Krafton + NVIDIA). Pitched at CES 2025 as the world's first "Co-Playable Character": an AI squadmate you command by voice, which loots, drives and fights alongside you. It's been in testing for a long time; one Brazilian outlet noted at the time that the CES demonstration was an edited video rather than live gameplay. Treat it as a showcase in progress, not a mass-market feature.

Fortnite's Darth Vader (Epic Games, May 2025). You could talk to Vader with your voice and he'd answer in an AI recreation of James Earl Jones's voice, with permission from Jones's family. Epic's FAQ said the words came from Google's Gemini 2.0 Flash and the voice from ElevenLabs' Flash v2.5 model. It worked, mostly; players also famously got him to swear. And it triggered the most important labor fight in this space so far (more on that below).

Meta Horizon Worlds AI NPCs (2025). Meta added conversational, voice-driven AI NPCs to Horizon Worlds, first with partner creators (you could talk to them in worlds like Bobber Bay Fishing), then rolled out the ability for creators to define a character's name, backstory, instructions and guardrails. Meta also promised NPCs that could trigger "in-world actions," i.e., Rung 3. Hold that thought, because Horizon's story took a sharp turn.

Stellar Cafe (AstroBeam, December 11, 2025). This is the most interesting pure-VR entry. Made by Devin Reimer, co-founder and former CEO of Owlchemy Labs (the studio behind Job Simulator), it's a $14.99 Quest game set in a sci-fi café full of robots you talk to with your actual voice, using only your hands for controls. The characters are hand-designed by writers; an LLM interprets what you say and generates replies in real time. Reimer told press the studio uses an efficient custom model specifically so the game could stay a one-time purchase instead of a subscription. Early reviews found it charming, novel, and short (about two hours), with occasional interaction hiccups and repeated phrases. In July 2026, AstroBeam announced it was coming to Steam, including flat-screen and Steam Deck versions.

Mantella (free mod for Skyrim and Fallout 4, including VR). Our I Spy guard. It's open source, works with any NPC including modded ones, and can be run for free using OpenRouter's free models (limited to about 100 requests a day), though better models cost money. It is also a patience test: one Quest 2 player on modest PC hardware reported response times of three to five seconds.

Indie experiments. A handful of smaller games make the conversation itself the whole mechanic, like Suck Up! (you're a vampire trying to sweet-talk AI-driven residents into inviting you in) and 1001 Nights, which is built around tricking an LLM character into saying particular words.

Notice the pattern. Every successful case either (a) keeps the AI in a small, fenced-off corner of a mostly handcrafted game, or (b) builds a small game around the AI's strengths and weaknesses. Nobody credible has shipped a big-budget game where the AI writes the main story or runs the town.

Part 7: Why VR is both the best and worst place for this

You'd think VR would be the natural home for AI characters, and in one sense it is.

The case for VR. In VR you're present. Characters are life-sized and standing in front of you. Menus and dialogue trees feel especially fake when someone is looking you in the eye, and typing is miserable in a headset. Voice is the most natural input VR has. Reimer's whole bet with Stellar Cafe is that once you can just talk, the technology fades into the background. Research backs up part of this: a German-led study of a speech-based VR game called Office Whispers found players felt deeply immersed when the AI characters responded believably and enjoyed the freedom of expression.

The case against VR. The same study found that unnatural conversational flow and inconsistent answers broke the spell. And in VR, the spell breaks harder, for three reasons:

  1. Latency hurts more face-to-face. In natural human conversation, people typically leave only about a fifth of a second between turns (a finding from cross-cultural research led by Tanya Stivers in 2009). An AI character that pauses for two or three seconds while a cloud server thinks feels broken when it's standing a meter away from you. One 2026 industry guide summarizing NPC research puts the comfort zone for AI replies at under roughly 800 milliseconds, and notes that a round-trip to a cloud model often adds one to two seconds. Researchers surveying LLMs in VR found that even simple "I'm thinking" cues, like a loading indicator, reassured players that the character was processing.
  2. The headset is a weak computer. inZOI can run its small model on a desktop graphics card. A standalone headset has a phone-class chip that's already busy rendering two eyes' worth of 3D at a high frame rate. So VR characters mostly need the cloud, which brings back latency and cost.
  3. The market is small and shrinking. This is the uncomfortable part.

Meta leaves the building

If any company was going to build VR towns staffed by agents, it was Meta. It has the headsets, the social platform, its own AI models, and it had literally announced AI NPCs for Horizon Worlds.

Then, in February 2026, Meta announced that Horizon Worlds would become "almost exclusively mobile" and that it was explicitly separating its Quest VR platform from the virtual world, per TechCrunch. That came a month after Meta reportedly laid off roughly 1,500 Reality Labs employees (about 10% of the division) and closed several VR game studios. At Meta Connect in September 2026, the Horizon news was Horizon Create (a mobile app) and Horizon Studio (browser-based) for generating games from prompts, distributed through Facebook and Instagram. The keynote's spotlight belonged to Meta's AI assistant and smart glasses.

The money explains why. Meta's Reality Labs division lost $4.62 billion in the second quarter of 2026 on $431 million in revenue, and its cumulative operating losses since late 2020 now exceed $80 billion (estimates from different outlets range from about $83 billion to $88 billion depending on how they tally it). Meta said growth in the division is now coming from AI glasses, not Quest headsets.

So the irony is thick. The AI NPC tech Meta built for Horizon is still there, and Horizon Studio still lets creators generate NPCs. But the place it's aimed at is your phone, not your headset. The best-funded effort to staff virtual worlds with agents is happening in the metaverse's mobile afterlife.

That doesn't mean VR agents are dead. Stellar Cafe exists. Mantella exists. But the honest picture is that the pioneers in VR are small studios and modders, not giants.

Part 8: The four walls everyone hits

Every project above, from the Skyrim mod to the Minecraft civilization to the Quest café, runs into the same four walls. If you understand these, you understand why the dream is taking so long.

Wall 1: Latency (the awkward pause)

We covered this. Real-time conversation is unforgiving. The industry's main workaround is going small and local: run a smaller, faster model on the player's device rather than a giant one in the cloud. That's what inZOI did, what PUBG Ally is built on (NVIDIA lists Mistral-Nemo-Minitron-8B, a compact model, among its ACE models), and roughly what AstroBeam did with its custom model. A 2026 AAAI paper from Aarhus University researchers is typical of the research direction: fine-tune tiny models to be one specific character, with swappable memory modules, so they run on consumer hardware. The trade is that small models are less clever. A dimmer character who answers in half a second usually beats a brilliant one who takes three.

Wall 2: Cost (every sentence is a bill)

This is the wall people underestimate most. Remember the virtue of the old dialogue tree: written once, played free forever. A generated line costs money every single time it's produced, for every character, for every player, for every conversation. Hosted platforms make this explicit. Convai, a popular NPC platform, has a free tier and an indie plan reported at $29 a month for 3,000 "interactions," with extra interactions billed separately. Inworld AI, once the poster child of this field, charges for its text-to-speech by the character, from roughly $25 per million characters on its free on-demand tier down to around $5 at enterprise volume.

Inworld's arc is itself a lesson. In 2023 it raised money at a valuation over $500 million, with more than $100 million in total funding, as "the Character Engine" for game NPCs. By 2025 it had repositioned as general voice-AI infrastructure, selling text-to-speech and an orchestration "runtime" to all sorts of apps, with games as one customer among many. Its own materials say the indie game Death by AI only reached profitability after switching to Inworld because its earlier AI services were too expensive, which is a vendor's claim, but a revealing one: the cost problem nearly killed a game.

And then there's the cautionary tale from the companion-chatbot world. On October 12, 2026, Character.AI, the platform where millions of people chatted with AI characters, filed for Chapter 11 bankruptcy, with Disney lined up as the lead bidder at $20 million for a license to some of its technology. Its legal troubles over teen safety were a huge factor, but figures reported in coverage of the filing (which I couldn't verify against the court documents) put its August 2026 revenue around $5.3 million against roughly $13.3 million in operating expenses, about $7 million of that on computing. Characters that talk back are expensive to keep talking.

Wall 3: Consistency (the character keeps falling out of character)

A language model is a brilliant improviser with no fixed self. Push it and it drifts: it forgets the setting, invents facts, breaks the fourth wall, or agrees to whatever the player proposes. Where Winds Meet's NPC who was convinced she was pregnant is the comic version. The darker version is a merchant who can be argued out of his stock (Project Vend) or a character who reveals the quest's twist because a player asked a leading question.

The fix that keeps showing up is constrained generation: the model talks, but its actions are limited to a short, fixed menu. In Where Winds Meet, according to a November 2025 breakdown, the AI's output is just text plus a fixed set of in-game flags (like raising affection or turning hostile). It cannot spawn items, create locations or rewrite quests. That's not a bug in the design. That is the design.

Wall 4: Trust (players, performers, and artists)

This wall is human, and it may be the tallest.

Players are split. Where Winds Meet's chatbots delighted a lot of people. One viral post called the feature a dream come true. But KitGuru reported many players felt the bots broke immersion, and Rock Paper Shotgun noted its readers largely dismissed the idea of an "in-game Siri."

Performers are organizing. When Fortnite launched its AI Vader, the actors' union SAG-AFTRA filed an unfair labor practice charge with the National Labor Relations Board on May 19, 2025, against Epic's subsidiary Llama Productions. The union's point wasn't that the Jones estate lacked the right to license his voice. It was that the company had replaced bargaining-unit work, including the work of actors who had matched Vader's voice in earlier games, without notice or negotiation. AI use in games was the central sticking point of the video game voice actors' strike that began in July 2024.

Many writers think the whole premise is wrong. In December 2025, Rock Paper Shotgun asked five narrative designers and AI researchers about chatbot NPCs, and the answers are worth reading in full. Meghna Jayanth (writer on 80 Days and Sable) argued that being able to say anything to a character is more a weakness than a strength, because what a game won't let you say is part of how it creates meaning. Failbetter Games' narrative director Chris Gardiner made a related point: the product of writing is meaning, not words, and human writers can plant jokes, foreshadowing and social commentary in a throwaway line in a way a model can't intend. Cornell Tech's Max Kreminski said LLMs tend to wash out strong authorial voice while not being inventive enough to be a good improv partner, but saw a possible "whole new category" of games that use models to interpret open-ended player input rather than as chatbots. Writer Dan Griliopoulos was the most optimistic, imagining narrative designers shifting from writing every line to curating the backstories and world texts that steer AI characters.

It would be dishonest to present this field as if the only obstacle were engineering. A meaningful slice of the people who make games think the destination isn't worth reaching.

Part 9: Hype vs. reality, a scorecard

Let's line up the common claims against what the evidence actually shows.

Part 10: Where this is actually going

If you squint at all of this, a few directions look solid.

1. Small, local, specialized brains. The future NPC is probably not a giant general model in a data center. It's a compact model, possibly fine-tuned to be one specific character, running on your own device or a nearby server, fast enough to keep up with conversation and cheap enough that the studio isn't paying per sentence. inZOI and Stellar Cafe already point here.

2. Fences, not freedom. Every successful design so far gives the model a small stage: a fixed personality, a short list of permitted actions, a world bible it can't contradict, and human-written lines for anything that matters to the plot. The model provides texture: banter, reactions, small talk, the sense that the background characters are paying attention. Think of it less as "the AI is the writer" and more as "the AI is an actor doing improv inside a very detailed script."

3. New genres, not upgraded old ones. The most compelling AI-character games aren't Skyrim-with-a-chatbot. They're games where talking is the gameplay: persuading a vampire's victims, tricking a character, running a café of robots who need help with weird problems. Kreminski's suggestion that success will look more like inventing a new genre than augmenting an existing one looks right so far.

4. Societies as backdrops, not protagonists. The Smallville and Project Sid techniques may end up powering the world rather than the characters you talk to: background simulations deciding who's feuding, which shop is low on stock, where the gossip is heading, and handing those facts to a more tightly controlled character when you walk in. The town lives; the shopkeeper you meet is a constrained performer reading from what the town just did.

5. The humans move up a level. If Griliopoulos is right, writers don't disappear, they shift from authoring every line toward authoring the people: their histories, values, secrets, voices and limits. Whether that's a promotion or a demotion depends heavily on who you ask, and on whether studios use it to make richer worlds or to cut headcount.

And over all of it hangs the question nobody can answer yet: do players actually want this? Project Sid's founders discovered their users didn't want agents with minds of their own. Where Winds Meet's players split down the middle. Stellar Cafe's reviewers loved it for two hours. The technology is clearly capable of something new. Whether that something becomes a staple of how we play, or a novelty that fades like motion controls, is genuinely undecided.

Back to the guard

Let's return to that Skyrim guard with the torch.

Here's what was really going on in that moment. A volunteer modder stitched together a speech recognizer, a language model, a voice synthesizer and a sliver of game awareness. A player's voice went out to a model, possibly on a server thousands of miles away, and came back three to five seconds later as a gravelly Nordic voice saying something no writer ever wrote. The guard didn't know what a torch was. He didn't want to play I Spy. He won't remember the game tomorrow unless the mod saves it for him.

And yet the player was losing his mind, in the good way. That's the whole field in one scene: the experience is real even when the mind behind it isn't, the illusion is fragile, and it's held together by a surprising amount of human craft.

The shopkeeper who remembers you is coming, slowly, in fenced-off corners of games, mostly on phones and PCs before headsets, and probably not as the sprawling autonomous village of the pitch decks. When it arrives in a form that sticks, it won't be because the AI got smart enough to replace the people who build worlds. It'll be because those people figured out exactly how small a stage to build for it.

Why this belongs at Starvard

A shop counter may be the best classroom for the agent age, because it compresses the whole subject into one encounter: perception at the door, memory across visits, goals behind the till, money changing hands, and the trust that decides whether any of it feels like service or surveillance. Starvard belongs in that conversation because a free university should not only teach people to use these systems; it should teach them to inspect the person-shaped software they will increasingly meet. Start with How AI Works for the models underneath the voice, continue to AI Agents: Your First $10,000 for the planning and tool use that turn a talker into an actor, then make something small and inspectable yourself in Your First Super Mario-Style Game. The point is not to fear the shopkeeper or worship it. The point is to understand the stage, the strings, and the bill well enough to decide what kind of virtual neighbors we want.

Watch

Sources

Tagged: Technology · AI Agents · Virtual Reality · AI Agents in VR · Virtual Worlds

Back to the blog

A note on accuracy: Much of what you read at Starvard was created with help from AI agents, working alongside people. We check what we can, but we cannot guarantee that everything here is true, current, or complete. If something really matters, your health, your money, a grade, a job, please verify it with a trusted source before you rely on it.