The average citizen of Novigrad, Night City, or Los Santos isn’t very smart. They repeat the same few phrases, do the same things, and react the same way when you cause a ruckus around them—usually running in fear until turning a corner and then returning as if nothing happened. They’re vapid, predictable, and their responses are only skin-deep. I’ve been speaking to the CEO of a company hoping to change that by harnessing AI.
“Let’s say you want a background character that doesn’t have any impact to the story… Today, these characters are given one line and repeat that one line over and over again, maybe two lines at best,” says Purnendu Mukherjee, CEO of Convai.
“However, even if you give, in the backstory, a couple of lines to these characters, now they can extrapolate from that, stay in character, stay in the story.”
Convai is a platform for creating these sorts of NPCs. You might have seen some of the work produced with the tool already, in collaboration with Nvidia’s ACE platform. I gave it a try myself over at CES, chatting to Jai about ramen, ordering a bowl, and even sending a few back for being cold. The character reacts, in real-time, with appropriate answers, and is even able to perform actions with objects in the game world. It can also be used to allow NPCs to talk to one another, as a video released earlier this year shows.
What’s actually going on behind the scenes is the Convai client communicates with cloud servers, which are running a range of models, including large language, text-to-speech, speech-to-text, animation generation, and retrieval augmented generation.
“And all of that executes in less than a second.”
The game engine has also been pre-programmed using a Convai plugin to tag various objects. Once properly tagged, this allows an NPC to “see” objects it can interact with. I’m told the plugin for the major game engines takes care of most of that work, and reportedly there are already quite a few developers and studios working with the tool today, though Mukherjee can’t yet tell me who.
Yet the question on my mind is: will this sort of integration devolve into mindless chatbots just repeating Google searches at me? We’ve been there before with ChatGPT plugins for Skyrim that spit out incorrect answers to simple puzzles.
“One thing is giving it a backstory, and giving it this thing that it can adhere to, and stay and talk to you in an engaging way,” Mukherjee says, “It’s a whole other thing where you are designing their intent.”
In theory you could spend hours talking to an AI-powered NPC for absolutely no reason. In my mind, that could be worse than an NPC with limited conversation options—at least the stupid NPC is not wasting my time and running in circles. But Mukherjee says the company is well aware of the concerns that gamers have for AI-driven NPCs and endless conversations going nowhere.
@pcgamer_mag
♬ original sound – PC Gamer
“One of the criticisms is that they need intent, they need narrative, they need guidance. Of course, we know that, and we are building towards it,” Mukherjee says.
“For the main characters, we expect the narrative designer to write significantly for the backstory for the knowledge bank … But do they have a particular target and a goal and objective and purpose that they are trying to drive you towards? In which case, you may even have great dialogue that you as an additive writer have already written that they would want to say, so you can actually do that.”
Mukherjee shows me the dashboard of the app, and there are quite a few knobs and dials to poke and prod in Convai to get a character acting as intended.
“This is the emotion wheel that you can see how the character is feeling. You can also set it up how they will react. This is their personality and style, the core personalities. This is where you set the guardrails, you know where they can go and whatnot.”
It all sounds like a lot of work, and that’s a topic that Mukherjee touches on at multiple points during the conversation, noting he feels: “We need more high quality writers and artists, not less” in tandem with AI NPC development. The guardrails required to ensure an NPC says the right thing, or more importantly doesn’t say the wrong thing.
“The writers can now talk to these characters over and over, asking them those corner case questions.”
Though the tech does naturally exclude voice actors, which has already seen a backlash against these sorts of text-to-speech AI tools and their use in game development.
Personally, I’m less immediately excited in the uses of AI NPCs for a game’s main performances. That’s not because the tech isn’t impressive, it is, it’s that when I play Cyberpunk 2077, I’m plenty engaged with the main characters and their storylines already. As I see it, it’s not a problem that needs solving—leave it to the voice actors, writers and artists. But the average Night City NPC is an idiot, and if there is a way to harness AI to make those incessant small interactions, if not more exciting, at least more varied? Maybe even more relevant to the location and events playing out in the game at that time? Yeah, that would be something. That said, there are those games that have created living worlds with rich NPCs just wandering about the place—Red Dead Redemption 2 springs to mind.
Mukherjee has an interesting take on AI and its impact on game development, as someone running an AI biz.
“The risks are legit. Those are not something that should be brushed off, those should be listened to and understood and talked about and addressed.
“Folks are concerned that [AI] is gonna make games worse… people are still in control, people still make pretty bad games.”
He also notes, “There are more terrible games than ever before without AI. There are more amazing games than ever before without AI. And it’s still going to be the same thing.”
Perhaps with the right direction, and if the tech works well on a larger scale than the demo I’ve experienced, there’s a time and place where Convai could improve things. For now, I’m just waiting to see how those first few games actually use the tech. What does that actually look like? Incentivised NPCs running around an open-world won’t be the only way to utilise the tech, nor likely the best, but it’ll be important to bring something genuinely new to the table—not all ‘AI’ features do. Convai does appear to offer new levers and pullies to allow for something fresh, anyways.
