AI Agents in VR: Virtual Worlds With a Real Workforce

For forty years, the people inside virtual worlds came in exactly two kinds: real players, and puppets reading scripts. A third kind of resident has now moved in. AI agents that can see the world around them, remember what you said yesterday, hold down jobs as guides, teammates, shopkeepers and tutors, and clock in for training shifts inside worlds built purely for practice. This post maps who is building this workforce, what it charges by the minute, what has actually shipped, and what is still a stage demo wearing a confident smile.

Why an agent wants a body

Start with a simple question: why put an AI agent inside a virtual world at all? The chat window already works. You type, it answers, everyone goes home.

The chat window works for questions. It fails at presence. A chat agent has no place, so it cannot meet you anywhere. It has no senses, so it cannot notice the thing you are looking at. It has no shared situation with you, so every conversation starts from zero. And it cannot do anything except talk, which is why chat agents keep getting described with the slightly sad word "assistant," like someone who hovers near the work instead of doing it.

A world fixes all four problems at once. Give an agent a virtual body and suddenly it has perception: cameras and microphones pointed at the same scene you see, plus a direct feed of the world's state, who is standing where, what just exploded. It gets memory with an address: the shop on the corner, the player who was rude last Tuesday. It gets actions with consequences: it can walk over, pick the thing up, drive the car, revive the teammate. And it gets other people. A world is the one place where an agent and a human can stand in the same situation and point at the same object and mean the same thing by "that one."

Game studios got here first, because games have been simulating residents for decades. The classic NPC, the non-player character, is a puppet: a character model wrapped around a script. Talk to the innkeeper in a 1990s role-playing game and he says his one line. Talk to him a thousand times and he says it a thousand times, with the serene confidence of a man who has never once been listened to. Dialogue trees made the puppet branching, not alive. The new claim is that the puppet can finally be cut from its strings: plug in a language model for a brain, voice models for ears and a mouth, and the innkeeper can hold an actual conversation, remember you, and decide things.

NVIDIA describes the anatomy plainly: an autonomous character needs perception, cognition, and action, looping fast enough to feel alive. In its January 2025 announcement of ACE autonomous characters, the company cited its esports research showing that skilled gamers make roughly 8 to 13 micro decisions per second, things like correcting aim or deciding when to fire a skill. A character that thinks once every few seconds will feel like a statue that occasionally talks. A character that perceives and decides at human tempo starts to feel like a colleague.

That is the deep idea of this whole post. The story of AI agents so far has been the story of brains looking for work. Virtual worlds are where the brains get bodies, shifts, and coworkers. Everything below is the current state of that labor market.

The character factories: Inworld and Convai

If you want an agent resident in your world today, you probably do not build the brain yourself. You rent one from a character factory: a platform where you describe a person, backstory, personality, voice, what they know, and the platform supplies the live conversation, streaming your character's words into Unity or Unreal Engine. The two best known factories are Inworld AI and Convai, and their price lists tell you a lot about how this industry sees a character: not as a product you buy once, but as a utility you meter.

Inworld started as the character company, and it landed the demo that defined the category: at the Game Developers Conference in March 2024, Ubisoft showed NEO NPCs, experimental characters built with Inworld and NVIDIA, including a resistance member named Bloom who could discuss a heist plan with you, react to events on a screen in the room, and warm up or cool down depending on how you treated her. Hands-on reports, including The Verge's, noted the seams: pauses before replies, occasional stutters, and Ubisoft repeating, in every direction, that this was a research prototype, not a promise to ship it in a game.

By 2026, the storefront looks different. Its pricing page now leads with realtime speech and inference: text-to-speech at $25 per million characters on the pay as you go tier, falling to as low as $5 per million characters at enterprise scale; speech-to-text at $0.15 per hour, down to $0.10; and LLM inference through its Router starting at $0.07 per million input tokens. Subscription plans run $25, $100, $300, and $1,500 per month, mostly as bundles of usage credits with better rates. The company that set out to sell personalities now sells the voice and the thinking by the sip. The character became infrastructure.

Convai kept the character front and center. Its pitch is a studio where you design an embodied character, give it a backstory and a knowledge bank of documents it is allowed to know, then deploy it through plugins for Unity and Unreal, with perception of its surroundings and the ability to perform actions in the scene. Convai prices by the interaction, where one interaction is one exchange: you say something, the character responds. A current listing of its plans shows a free tier for testing; Indie Dev at $22 per month with a monthly quota in the low thousands of interactions; Professional at $69; Scale at $299; and Business at $499. Two details in Convai's own documentation matter. First, flagship language models have their own smaller interaction caps inside the big quota, so the smartest version of your character is rationed. Second, the binding constraint is concurrency: how many humans can talk to your cast at the same moment. A world with one beloved shopkeeper and ten thousand visitors does not need a smarter shopkeeper. It needs a thousand shopkeepers, and the invoice knows it.

Between them, the factories power a genuinely large amount of make-believe, including two stories we will return to below: Little Umbrella's Death by AI, whose AI game master reached 20 million players in its first few months, and Talkpal's tutors, with more than 10 million learners (both company-reported figures, via Inworld).

NVIDIA ACE, and the first agents with in-game jobs

While the factories rent brains from the cloud, NVIDIA is pushing the opposite architecture: shrink the brain until it fits inside the player's own machine. ACE began as a way to make game characters conversational. At CES in January 2025, NVIDIA expanded it into ACE autonomous game characters: small language models, some small enough to fit in 1.5 gigabytes of video memory, that let a character perceive the game through vision and audio models, plan with a language model, remember relevant facts with retrieval, and act, all on the player's own graphics card.

Three announced residents matter, covering the three jobs agents are actually getting in games: teammate, citizen, and boss.

The teammate is PUBG Ally, built by KRAFTON for PUBG: BATTLEGROUNDS. NVIDIA and KRAFTON call Ally the world's first Co-Playable Character, a deliberate step beyond NPC: not a bystander you talk to, but a squadmate who shares your shift. Ally runs a small on-device model (NVIDIA names it as Mistral-Nemo-Minitron-8B) and is designed to speak in the game's own slang, call out strategy in real time, find and share loot, drive vehicles, fight, and revive you when you are down. The honesty label: Ally is in the playtest chapter of its life, not the launch chapter. Coverage through 2026 describes limited-time tests in PUBG's Arcade mode in the first half of 2026, with KRAFTON showing off a long term memory upgrade at CES 2026, a partner that remembers your past matches. The famous reveal video was, as skeptical outlets noted at the time, heavily edited: real technology, real playtests, not yet a shipped feature you can queue up with tonight.

The citizen has shipped. KRAFTON's life simulator inZOI launched in early access on March 28, 2025 at $39.99, and sold more than one million copies in its first week, a company-reported figure that made it KRAFTON's fastest game ever to that mark. Inside the game sits an experimental setting called Smart Zoi. Flip it on and the city's residents stop being puppets: ACE gives each Zoi goals, reactions, and a social life that simmers along whether or not you are watching. NVIDIA's description of the launch build says the trick runs on a remarkably small on-device model, about half a billion parameters, distilled for home graphics cards. A whole city of autonomous residents, running locally, in a game you can buy: that is the strongest "this is real now" exhibit in the entire agent workforce story, and it arrived with almost no fanfare, as a toggle in a settings menu.

The third job is the boss. Wemade Next's upcoming MMORPG MIR5 is using ACE for boss enemies that learn from previous fights, so that farming the same boss twice produces two different fights. NetEase, meanwhile, put ACE teammates into NARAKA: BLADEPOINT MOBILE PC VERSION in March 2025, companions who fight beside you and fetch items, on local inference. The pattern: every working or near-working example here runs the brain on the device. No server bill per conversation. No network round trip. Remember who escaped the meter, and how.

Meta opened the doors, then moved the building

The biggest single stage for agent residents was supposed to be Meta's. Horizon Worlds, Meta's social VR platform, spent years with the opposite problem from most products: a functioning world, and a chronic shortage of anyone interesting to meet inside it. AI residents were the obvious patch, and Meta teased them at Connect 2024: characters you could actually talk to, for worlds that badly needed inhabitants.

Non-embodied conversational NPCs reached select partnered creators early in the year. In September 2025, as UploadVR reported, the tools opened to all Horizon creators: any world builder could add scriptable AI NPCs that United States adults could talk to naturally, powered by Meta's Llama 4 model. A Character Builder let creators set each NPC's name, backstory, instructions, and guardrails; there were dozens of synthetic voices; and TypeScript scripting could feed the NPC live knowledge of the world, and even motivate it with what Meta calls elicitations, stage directions such as "the laser blast pierced your armor and you are in terrible pain," which the character then performs in its own words. Fully embodied NPCs, characters with a visible body standing in the world with you, were previewed as the next step. It was the most complete version yet of the agent resident idea: no code required, no API bill for the creator, population as a feature.

Then the platform itself wobbled, which is the part of the story nobody puts in the keynote. In March 2026, Meta announced that Horizon Worlds would leave VR: the app was removed from the Quest Store on March 31, 2026, and the VR version shut down on June 15, 2026, with Worlds continuing on mobile. Community backlash won a partial reversal within days, with Meta clarifying that only worlds built on its newer Horizon Engine would be deprecated on headsets. By the autumn of 2026, UploadVR was reporting that Meta is preparing to spin Worlds out of the Horizon mobile app into a separate, games-focused app, internally codenamed Eclipse, where AI tools help you remix worlds from a prompt.

The lesson is not that Meta's NPCs failed. By most accounts they worked about as well as the state of the art allowed, and Meta could afford to give them away. The lesson is that an agent resident has two employers: the technology that animates it and the platform that houses it. The first employer is getting better every quarter. The second can move the building to a different medium, on a quarter's notice, for reasons that have nothing to do with how delightful your virtual bartender was.

Agents already holding jobs in virtual places

Step away from the big announcements and a quieter pattern appears: agents have started taking the classic front-of-house jobs. Greeter. Guide. Concierge. Companion. Tutor. These are jobs where the core skill is being present, friendly, and roughly right, and where the alternative is not a brilliant human but nobody at all.

The most instructive deployment lives in the oldest world still standing. Second Life, Linden Lab's user-built universe, has been running since 2003. At GDC in March 2024, Convai announced a partnership with Linden Lab to bring conversational AI characters into Second Life, and Convai's bots are live there as companions and greeters. They can talk with newcomers, answer questions from a knowledge bank the world's owners control (which keeps the character from inventing facts), and act on the conversation, including teleporting a visitor to a place they ask about. They keep a long term memory of the people they meet. Linden Lab's chairman, Brad Oberwager, framed the job description at the announcement: the characters can ease the learning curve for new users, give tours, carry out actions based on conversation, and connect people with each other. The first job of the agent resident is not to dazzle. It is to make sure a new arrival in a strange world meets someone helpful in the first sixty seconds, because in virtual worlds, the first sixty seconds decide whether anyone stays.

The companion job got its own flagship product in Rec Room, the social VR platform. In June 2025, Rec Room shipped Roomie, a small robot buddy who roams the world with you: exploring alongside you, chatting, answering questions about the room you are in, helping you change your style. Roomie was exclusive to the Rec Room Plus subscription, and the detail that makes Roomie historically important is its meter. Talking to Roomie burned a currency called Roomie Energy. Subscribers received 8,000 energy per month with their plan, and extra energy could be bought at about $1 per 1,000, as documented in the platform's notes and Wikipedia's summary. In other words, Roomie was the first mainstream attempt to sell an agent companion the way arcades sold lives: in small, friendly, forgettable sips. The market test was genuine. Then the platform test failed: Rec Room shut down entirely on June 1, 2026, a little under a year after Roomie arrived. Roomie did nothing wrong. Its world died under it. The sentence to remember: residents are only as durable as their worlds.

The tutor job is thriving one layer over, in voice rather than in headsets. Talkpal's language tutors, running on Inworld's voice infrastructure, have served more than 10 million learners, holding spoken practice conversations no human tutor market could staff around the clock. Convai markets the same pattern into training and XR: role-play characters for rehearsing difficult conversations, tutors who can see what you are doing in the scene. The throughline across all of these jobs is that the agent's product is not information. It is attendance. A world, a lobby, a practice room that is never empty, staffed by someone who always picks up, always remembers your name, and never has a shift end.

Virtual worlds are also where agents go to school

Now flip the direction of travel. Virtual worlds are also where agents are manufactured: trained, tested, and graduated into jobs, including jobs in the physical world.

The clearest exhibit: Google DeepMind's SIMA 2, announced on November 13, 2025. SIMA, short for Scalable Instructable Multiworld Agent, lives inside 3D games the way a person does: it watches the screen and works a virtual keyboard and mouse, never touching the game's code. The first SIMA, in 2024, could follow simple instructions like "climb the ladder" across many games, completing about 31 percent of tasks in DeepMind's test suite. SIMA 2 puts a Gemini model at the center, and the behavior jumps category: the agent can pursue high level goals, explain what it is doing and why, follow instructions delivered as speech, sketches, or even emojis. DeepMind reported a task completion rate of 65 percent, against 31 percent for SIMA 1 and 71 percent for human players. It can also transfer concepts between worlds: learn what "mining" means in one game, then apply it to "harvesting" in another.

The part that matters for our story is how SIMA 2 improves. After an initial education from human gameplay videos, it can switch to self-directed practice: another Gemini model invents tasks and grades the attempts, and the agent banks that experience to train its next, better version, no new human demonstrations required. And the practice fields themselves can be generated. DeepMind paired SIMA 2 with Genie 3, a world model that builds interactive 3D environments from a text prompt or an image, and the agent orients itself and works inside worlds that were invented minutes earlier. The metaphor writes itself: Genie dreams up an endless supply of new schools, Gemini grades the homework, and SIMA is the student who never sleeps and never graduates, because there is always another world. DeepMind is explicit that this is a research system in limited preview, and equally explicit about the destination: the reasoning layer for robots.

The robots are already enrolled. At GTC in March 2025, NVIDIA announced Isaac GR00T N1, an open foundation model for humanoid robots, and its training recipe is the industrial version of the same idea. Its blueprint expands a small set of human demonstrations into synthetic practice inside simulation: NVIDIA reported generating 780,000 synthetic robot trajectories in 11 hours, the equivalent of about 6,500 hours, or nine continuous months, of human demonstration data. Mixing that synthetic schooling with real data improved the robot model's performance by 40 percent compared with real data alone, by NVIDIA's measurement. That is the economic engine of this entire post in one sentence: in a virtual world, experience is nearly free, infinitely varied, and nobody gets hurt. A warehouse robot that will one day work beside people first clocks its practice hours in a simulated warehouse, falling over safely ten thousand times.

The two walls: 200 milliseconds, and the meter

Two numbers govern everything above, and every builder in this space is quietly obsessed with both.

The first is latency, the gap between you finishing a sentence and the character answering. Conversation researchers have measured the human version of this gap for decades: in natural talk, people typically respond within about 200 milliseconds of each other (a figure from Stivers and colleagues, published in PNAS in 2009). Inworld's guidance for companion builders states the practical rule: below about 200 milliseconds total, a conversation feels natural; above about 300 milliseconds, pauses start to feel like lag. Transcription, thinking, voicing, and a network round trip all have to fit inside that budget, which is why the serious platforms overlap the stages, streaming the first sounds of the answer while the model is still composing the rest. Inworld reports its TTS-2 voice model producing first audio in under 100 milliseconds at the 99th percentile, server side, and its Flash variant under 25 milliseconds. That is one layer of the sandwich, measured on its home turf. NVIDIA's on-device ACE characters sidestep the worst of it by never leaving your computer, and it is not a coincidence that the shipped examples cluster there.

The second wall is cost, and it is the reason so much of this field is still subsidized. A resident who talks all day bills all day. Inworld's own cost guide works a scenario out loud: a companion app with 100,000 daily users, each talking by voice for 30 minutes a day, generates roughly 900 million characters of speech per month. At premium voice rates of about $103 to $206 per million characters (Inworld's comparison uses ElevenLabs list prices), that is $93,000 to $185,000 per month, for the voice alone, before the brain's tokens are counted. That is a vendor's math, from a vendor selling the cheaper option, so hold it at arm's length. But it explains the casualties. Little Umbrella's Death by AI was a hit so fast that, in the studio technical director's words as quoted by Inworld, "our cloud API costs went from $5K to $250K in two weeks," and the team had to throttle new signups until they re-architected onto cheaper infrastructure. Wishroll's Status app reportedly saw total AI costs of $12 to $15 per user per day before a similar rebuild cut them by about 95 percent (customer figures as reported by Inworld). Success, in this business, arrives holding an invoice.

The good news is that the meter is spinning slower every quarter. Inworld cut its prices by half or more in its June 2026 pricing update, with top tier voice rates heading toward $5 to $10 per million characters, speech-to-text down to about $0.10 per hour, and dedicated GPUs offered from $5 per GPU-hour. Small on-device models keep improving, which moves characters off the meter entirely. And Rec Room's Roomie Energy experiment proved consumers will accept a friendly meter if the companion is charming enough. The wall is real. It is also visibly cracking.

The honest scorecard, and what happens next

Let us total it up, with the labels this industry works so hard to blur.

Live now, usable today: inZOI's Smart Zoi, a shipped game with on-device resident AI. NARAKA: BLADEPOINT's ACE teammates, shipped in 2025. Second Life's Convai-powered companions and greeters. Meta's Horizon NPC tools, opened to all creators in September 2025, on a platform that has since left VR for mobile. Plus a long tail of indie characters rented by the interaction.

Playtests and research projects: PUBG Ally, in limited Arcade tests through 2026, promising and unlaunched. Ubisoft's NEO NPCs and Teammates demo, playable research with AI squadmates who take voice orders. MIR5's learning bosses, announced and in development. Stellar Cafe, an announced VR game from AstroBeam where you run a cafe of robots you are expected to actually talk to.

Gone, and worth mourning precisely because it worked: Roomie, switched off with Rec Room on June 1, 2026.

As latency budgets get met by streaming pipelines and small local models, and as the per-minute price of a character keeps falling by halves, the economics flip from "one subsidized showcase character per platform" to "a population." That is the real threshold. A virtual town does not need one brilliant resident. It needs a hundred adequate ones: the guide at the gate, the keeper at the shop, the coach at the range, the friend who remembers your name, the trainee in the back room practicing on simulated customers before she meets real ones. The platforms that win will be the ones where a small team can afford a population, and where the world itself stays open long enough for the population to matter. The metaverse spent a decade waiting for inhabitants who would make its empty plazas feel alive. It turns out the inhabitants were never going to move in as tenants. They were going to be hired.

Why this belongs at Starvard

Agents that live in worlds sit at the crossing of three Starvard subjects: how AI actually reasons, how learning systems improve from experience, and the code underneath both. If this post left you curious about the machinery, start with How AI Works, our plain-English tour of what a model is and does. Go deeper with Machine Learning 101, which covers how systems learn from data, the same loop SIMA 2 runs inside its practice worlds. And if you would rather build a small character of your own, Python 101 is the friendliest first step toward the APIs every factory in this post exposes. All three are free, like everything at Starvard.

Watch

• NVIDIA GeForce, "NVIDIA ACE | Introducing PUBG Ally - First Co-Playable Character": youtube.com/watch?v=wEKUSMqrbzQ. Two minutes that show the teammate job better than any spec sheet: Ally fetching loot, driving, and calling plays like a squadmate, not a menu.
• Google DeepMind, "Introducing SIMA 2, the next milestone in our research creating general and helpful AI agents.": youtube.com/watch?v=Zphax4f6Rls. Watch the student side of this story: an agent thinking out loud in No Man's Sky, then practicing inside worlds that Genie 3 invented on the spot.

Sources

• UploadVR, "Horizon Worlds Creators Can Now Add Conversational AI NPCs", September 10, 2025. Meta's NPC rollout, Llama 4, Character Builder, and the per-token economics.
• Inworld AI, pricing page, checked October 2026. Speech and inference rates, and the $25 to $1,500 plan tiers.
• Inworld AI, "Voice AI for AI Companions", updated April 5, 2026. Latency targets, the 200 millisecond human response gap, and the 100,000 user cost scenario.
• Inworld AI, "Cost is the wall in front of consumer AI", June 2026. The price cuts, and the Status and Death by AI cost stories, with customer figures as reported by Inworld.
• Inworld AI and GamesBeat, "Inworld AI showcases AI case studies as they move to production", 2025. Death by AI reaching 20 million players, and the quote on API costs rising from $5K to $250K in two weeks.
• Toolify, Convai listing, updated October 7, 2026. Convai plan prices, interaction quotas, and concurrency caps.
• Wccftech, "Convai Shows Off Unity AI NPC Demo, Announces New Features and Partnerships with Second Life and Stormgate", March 2024. The Linden Lab partnership and the Oberwager quote.
• NVIDIA, "NVIDIA Redefines Game AI With ACE Autonomous Game Characters", January 6, 2025. ACE's stack; PUBG Ally; Smart Zoi; NARAKA teammates; MIR5 bosses; the 8 to 13 micro decisions figure.
• WorthPlaying, summarizing NVIDIA's Game Ready driver notes, March 2025. Smart Zoi's on-device model size and launch.
• Gadgets 360, "Krafton's inZOI Crosses 1 Million Copies Sold in a Week After Steam Early Access Launch", April 2025. The one million copies figure, as announced by KRAFTON.
• GameGPU, "PUBG BATTLEGROUNDS gets AI companion PUBG Ally with NVIDIA ACE-based memory", January 2026. The CES 2026 memory demo and Arcade test window.
• UC Today, "Meta Shuts Down Horizon Worlds VR, Mobile Version Remains", 2026. The March 2026 announcement and the June 15, 2026 VR shutdown.
• UploadVR, "Meta Is Working To Spin Off Worlds From The Horizon Mobile App", 2026. The partial reversal of the VR decision and the planned Horizon Create spin-off.
• Rec Room, "Rec Room UPDATE: the Welcome, Roomie! Edition", June 16, 2025. Roomie's launch as a Rec Room Plus feature.
• Wikipedia, "Rec Room (video game)", checked October 2026. Roomie Energy amounts and pricing, and Rec Room's June 1, 2026 shutdown.
• UploadVR, coverage of AstroBeam's Stellar Cafe. The announced VR cafe staffed by conversational robots.
• SiliconANGLE, "Google DeepMind's SIMA 2 agent learns to think and act inside virtual worlds", November 13, 2025. SIMA 2's 65 percent task completion rate, and the Genie 3 tests.
• NVIDIA press release, "NVIDIA Announces Isaac GR00T N1, the World's First Open Humanoid Robot Foundation Model", March 2025. 780,000 synthetic trajectories in 11 hours, and the 40 percent gain from synthetic data.
• The Verge, "Ubisoft let me actually speak with its new AI-powered NPCs", March 2024. Hands-on with the NEO NPC prototype, and its research project status.

Tagged: Technology · AI Agents · Virtual Reality · AI Agents in VR · Virtual Worlds