In this session, we will explore how to achieve state-of-the-art inference using TensorRT LLM and NIM. Attendees will dive into advanced TensorRT LLM features, such as in-flight batching and KV caching, designed to accelerate large-scale LLM production systems. We’ll also review unique inference challenges, including high computational demands, latency, and throughput. Discover how TensorRT LLM efficiently optimizes LLM performance on NVIDIA GPUs and integrates seamlessly with NIMs, offering an easy-to-use inference microservice that accelerates the deployment of foundation models across any cloud platform.