GPU Detective Stories: PyTorch Recompiles & Hidden Bottlenecks

AI Performance Engineering Meetup (Munich2) Monday, November 16, 2026 · 9:00–10:00 AM PST · Online Zoom link: https://us02web.zoom.us/j/82308186562 Joinus for two deep dives into hidden AI performance costs: surprise PyTorch recompilations during inference and compute, memory, and communication bottlenecks in Jev training. Talk #0: Introductions and Meetup Updates by Chris Fregly and Antje Barth Talk #1: The Recompile Ambush: TamingPyTorch Inference Spikes by Chaim Rand torch.compile can acceleratesteady-state inference, yet a guard failure on a new input shape or layout can trigger an expensive recompilation in the request path. Drawing on his latest post, Chaim will compare six approaches using a dynamic graph-neural-network example: cache preloading, bounded recompilation with eager fallback, Dynamo and input-layout tuning, warmup seeding, background compilation, and ahead-of-time compilation with torch.export and AOTInductor. He’ll explain their tradeoffs and how the right choice depends onthe model, input distribution, runtime environment, and latency requirements. Bio: Chaim Rand is a machine-learning algorithm developer whowrites about PyTorch and practical AI/ML performance optimization. Based on: Overcoming the PyTorch Recompilation Dilemma in Real-Time Inference Workloads Talk #2: Follow the Profile: Hunting Jev’s GPU Bottlenecks by Israel Ogbole, Co-founder and CEO of Zymtrace Israel will explore the latest Zymtrace/TypeSafe case study on training and serving Jev. He’ll discuss how continuous CPU/GPU profiling and MCP-accessible profile data supported an iterative optimization loop across compute, memory, communication, and inference synchronization. The published case study reports approximately 46% higher training throughput on the same hardware after three successive changes, plus approximately 10% better inference performance after removing a hidden device-to-host synchronization. The talk will focus on how the team found each bottleneck, measured each change, and identified the next constraint. Bio: Israel Ogbole is co-founder and CEO of Zymtrace, whichdevelops continuous CPU/GPU profiling tools for performance optimization. Based on: How TypeSafe achieved 46% higher GPU throughput training Jev — with the help of Zymtrace Zoom link: https://us02web.zoom.us/j/82308186562 Related Links Github Repo…