runarberg
today at 6:25 PM
In my go example we have a human and an AI model competing at the same task. A good AI model will perform much much much better and probably win the game, but if we measure the energy input into either player the AI model will consume a lot more energy. However a game of go is not automation, it wonāt save us any time. The benefits of the AI model is it helps human go players improve their own game, finding new moves, new patterns, new proverbs, etc. Because of go playing AI models human go players now play their games better, but nor more efficiently, nor faster.
In your LLM coding example you have a human and an AI model collaborating on a single task, both spend some amount of energy (taking your assumptions at face value, compatible amount of energy) and produce a single outcome. In the go example it is easy to compare energy usage and the quality of the outcome is also easy to measure (simply who won the game). In your coding example the quality of the outcome is impossible to measure, and because the effort is collaborative, splitting the energy usage is complected.
When talking about automation my game of go example falls apart. A much better examples would be something like a loom, or a digital calculator. These tools help the human arrive at a particular outcome much faster and with much less effort then a human performing the task without the help of the machines. The time saved by using these tools are measured in several orders of magnitudes, and the energy spent is at par with a human. It is easy to see how a loom or a digital calculator are more efficient then a human.
I guess if we take into account the training cost of an LLM model we should also take into account the production costs of looms and digital calculators. I donāt know how to do that, but I canāt imagine it would be anywhere close to that of an LLM model.
And we have an LLM model we have increased the productivity of, not 5000x[1], but by 5%-30%. To me this does not sound like a revolutionary technology. But I have my doubts of even the 5%-30% figure. We have preliminary research ranging anywhere from negative productivity increase to your cited 5%-30%. We will have to wait for more research, and possibly some meta-analysis before we can accurately assess the productivity boost of LLMs. But we will have to do a whole lot better then 5%-30% to sufficiently justify the huge energy consumption of AI[2].
Personally, I am not convinced by your back of the envelope calculations. It fails my sniff test that 9 Wh of matrix multiplication will consistently save you an hour of using your brain to perform the same task adequately. I know our brains are not super good at the logic required for coding (but neither are LLMs), but I know for a fact they are very efficient at it.
That said I refuse to accept your framing that we can simply ignore the energy used in training, on the bases that it is equally invalid as considering the energy used for evolving into our species, or that we can simply stop training new models and use the models we do have. That is simply not how things work. New models will get trained (unless the AI bubble bursts and the market looses interest) and the energy consumed by training is the bulk of the energy cost. And omitting it makes the case for AI comically easy to justify. I reject this framing.
Instead of calculating, instead Iām gonna do a thought experiment. Imagine a late 19th century where iron and steel production took an entire 2% of worldās energy consumption[3] (maybe an alternative reality where Iron working is simply that challenging and requires much higher temperatures to work). But the steam train could only carry the same load as a 20 mule team, and would only do it 5%-30% faster on average then the state of the art cargo carriages at the time without steam power. Would you accept the argument that we should simply ignore the fact that rail production takes a whopping 2% of global energy consumption, when factoring the energy consumption of the steam train, even when it only provides you with 5%-30% productivity boost. I donāt think so.
---
1: I donāt know how much the loom has increased productivity, but this is what I would guess without any way of knowing how to even find out.
2: That is, if you are only interested in the increased productivity. If you are interested in the LLM models for some other reason, those reason will have to be measured differently.
3: https://www.allaboutai.com/resources/ai-statistics/ai-enviro...