Behind the Scenes of Running a Conversational Character in a 3D Scene

While immersed in a game, it feels as if you're in a real 3D scene, and there's a realistic non-player character (NPC) that feels lifelike. The NPC is precisely as the game designer or author intended. It follows the game narrative, and behaves and acts naturally. Learn how this is possible using Convai's end-to-end character creation platform with 3D plugins. We'll discuss how we achieve low latency of the different AI models powering the interaction, as well as our voice system, which leverages Riva, our use of large language models (LLM), and our integration with game engines. Within the game engine itself, we'll dive into how you power the face and gestures of a character using Audio2Face within Omniverse. Lastly, we'll show how a hybrid LLM and our system combine to power a controlled narrative, allowing game designers to make their target experience in the 3D space. We'll conclude by demonstrating a narrative playing in a 3D scene.