Optimizing AI-powered NPCs Cost-Efficiency Using TRT-LLM Without Sacrificing Quality

The future of gaming will be powered by AI. Large language models allowed to enable a new era of immersive gaming experiences and to make sure that thousands of players can enjoy it one needs to guarantee that LLMs can be served as efficiently as possible. This session will shed more light on how to approach this problem on a practical surface and how the TRT-LLM framework allows to achieve SOTA performance.