KAIST Study Shows the High Energy Cost of AI Agents

As technology companies shift more attention from chatbots such as ChatGPT, Gemini, and Grok to AI agents, a new KAIST study highlights the energy cost of that change. AI agents are systems that can carry out tasks on their own by planning multiple steps, searching the web, gathering information, running calculations, and using other tools. In tests on a large language model with 70 billion parameters, a complex agent query averaged 348.41 watt-hours of electricity. The Reflexion agent framework, running on Meta's Llama-3.1-Instruct 70B model, used 136.5 times more GPU energy per query than a standard chatbot, while the LATS system reached 62.1 times that level. The study also found that agent responses could take up to 153.7 times longer. Researchers said GPUs spent more than half their time waiting for outside websites or software tools, even though the system was still using power. They estimated that if daily use of AI agents reached about 13.7 billion requests worldwide, data centers could need around 199 gigawatts of power, which they compare with about half of average electricity use in the United States. Professor Rhu Min-soo said the problem may not be solved by software improvements alone, and that models, chips, and data center power systems may all need redesigning.