How Much Energy Do Pruned LLMs Actually Save?

PyData St. Louis Join us at PyData St. Louis for a talk and community discussion on the energy efficiency of Large Language Models (LLMs) and what happens when we try to make them smaller through model pruning. As AI models continue to grow in size and capability, concerns about their computational cost and energy consumption have become increasingly important. One popular approach to reducing these costs is pruning, a technique that removes less important components of a neural network to create smaller and potentially more efficient models. But does making a model smaller always reduce the energy it consumes? Pizza and networking will begin at 5:30 PM, and the talk will start at 6:15 PM. In this session, we'll cover: * What model pruning is and why it is used in modern AI systems * The computational and energy challenges posed by large language models * How structured pruning changes model size, performance, and efficiency * Measuring energy consumption across GPUs, CPUs, and memory during inference * A case study using the Llama 3.1 and 3.2 model families * Evaluating code-generation performance using the HumanEval benchmark * The trade-offs between model compression, accuracy, and energy savings * Why aggressively pruning a model can sometimes increase energy consumption rather than reduce it * Practical lessons for building more efficient and sustainable AI systems * The talk is beginner friendly and open to anyone interested in machine learning, artificial intelligence, data science, software engineering, or high-performance computing. After the talk, we'll leave time for questions, discussion, and networking from 6:50 PM to 7:30 PM with others in the local data and AI community. Pizza will be provided. Who should attend? Anyone curious about large language models, machine learning efficiency, AI infrastructure, sustainable computing, or modern data science, including students, professionals, researchers, hobbyists, and beginners. Special thanks to Spark Coworking for providing the venue and supporting the local data science community. Come learn, connect, and be part of the PyData St. Louis community! PyData St. Louis is part of the global PyData community. PyData is an educational program of NumFOCUS, a nonprofit organization that promotes open practices in research, data, and scientific computing.