Just something I am thinking about, wondering if we can even know. If we completely replaced and/or augmented, e.g., web search with GPT(-like) models, would there be substantially more energy needed in order to sustain this? (Or less, or about the same?)
Very interesting! What about like web search/general-use task prompts? I feel like that scales differently, but feel like it could go either way? Like how many pages do I have to visit and render to match the energy used by GPT answering the same question?
Estimates of anything that involve the internet are often shockingly high and unreliable. See these two papers that come to opposite results (not just by a little but a lot)
You have to consider not just the server and client costs but how to attribute the cost of all the routers and networking parts between them that are invisible. Networking hardware can be particularly wasteful when it is sitting there waiting for a packet which has to be attributed somehow.
Nice paper, thank you for bringing it up. I think OP meant to compare LLM-driven operations with classical algorithms (they mention web search). It's not clear whether training + retrieval is cheaper than indexing + searching.
I think he’s talking about the energy being consumed by GPT systems as well GPT++ systems which combine GPT plus web search not so much as a competitor but as the next step.
As an example of the latter, I can answer questions out of my long term memory and GPT-4 does the same. I can answer more questions with more accurate results I’d I search for documents and use the capabilities encoded in my long term memory to evaluate, summarize and otherwise process those documents and no doubt that GPT-4 could be embedded in a system that does this as well and the large attention window would be a big help.
(E.g. a few papers a week are submitted to arXiv about this now)
One of the most fun things I did in grad school was when, for my A exam, I was asked to write a summary in the literature for giant magnetoresistance which involved looking at 50 or so papers. One would imagine it could take 50x the effort (queries, energy, etc.) for GPT-4 to do this than it would take to speak extemporaneously on the subject. (Funny I’d have a hard time talking about the physics w/o looking it up now but I could definitely talk about applications off the cuff.)
The amount of electricity Google uses in each search is bounded by how much money they get from the adds in that page. It's not a direct page by page calculation. Just an average value.
If someone in Google propose a new search method that cost more money (in average) than they money they get (in average), the proposal will be ignored.
Unless the new search method increased revenues. Perhaps by using the same technology to find even more relevant ads. You could have ads seemlessly integrated with the text response for example.
Microsoft would quickly go bankrupt if they tried to spend that much per search, it isn't like they have more resources to spend per query than Google does.
I will say it save more energy. Since intelligence is being accumulated inside the weights, we do not need to waste time on re-training students or robots again and again to do the same task.
Think of that training a teacher take more than 10 years at minimum, and still they do not perform well.
But still you train one human teacher to teach a class with many students, while the idea with this AI stuff is having a single tutor for every student.
In America, 10 years is quite alot of schooling if we are just talking about K-12 education.
And regardless, this is just one of the many ways we will be firing up these GPU arrays..
https://arxiv.org/abs/2303.06219
it will actually save energy.