The SIMPLE Law for Energy Cost of LLM Inference
A new analytical model, the SIMPLE Law, addresses the non-linear energy cost of LLM inference by accounting for how each token entering the KV-cache raises the cost of subsequent tokens, according to the article. The art…