Deep Dive into Training and Inferencing Large Language Models on Azure (Presented by Microsoft)

In the coming years, the foundational models in artificial intelligence will continue to rapidly evolve towards multi-modality and targeted small language models in addition to improving the capabilities of existing large language models. As developers of AI, you need an infrastructure that can scale seamlessly and be flexible enough to cater to the demands of the AI model that you want to develop. In this session, we will explore the advancements we have made in accelerating training and inferencing of Large Language Models (LLMs) using Azure's cutting-edge infrastructure powered by NVIDIA H100 SXM Tensor Core GPUs and NVIDIA H100 NVL 94GB PCIe Tensor Core GPUs. We will dive into the performance, scaling, and readiness of our AI infrastructure and will extend our vision in deploying inferencing in a cost-effective way on Azure's NC H100 v5-series.