A million people like you can do a million times more things that require intelligence†, and would consume a million times more Taco Bell double deckers. Ergo, in that limited sense, more energy == more intelligence.
"Over time" clearly means that he's talking about the far future, not the next few years when they're cleaning up the low-hanging fruit. Rather than "absurd ... fantastic and impossible" I would describe the Dyson-sphere outcome as inevitable, unless humanity goes extinct within a few centuries. Maybe you thought he meant next March?
In https://epoch.ai/gradient-updates/how-much-energy-does-chatg..., Josh You, Alex Erben, and Ege Erdil estimate that a typical GPT-4o query uses 0.3 watt hours, based on the estimates that it's a 400-billion-parameter MoE model in which ¼ of the parameters are activated for a given query, so each token requires 200 gigaflops (100 billion multiplies and 100 billion adds I guess), a typical query produces 500 output tokens (about a page of text), and they're running on 1-petaflop/s H100 GPUs, with the overall cluster consuming at peak 1500 watts per GPU, with a utilization rate of 10%, and that the cluster on average consumes 70% of peak power. This works out to 1.05 kilojoules, or about 0.25 kilocalories, the amount of food energy in one gram of carbohydrates.
So, a 320-kcal Double DeckerⓇ Taco™ works out to 1280 GPT-4o queries answered, and a standard 2000kcal/day diet works out to 8000 GPT-4o queries per day, if we believe You, Erben, and Erdil's estimate. For someone who is looking for GPT-4o-quality output, if you are producing less than 8000 pages of writing per day, you are less energy-efficient than GPT-4o.
______
† although there's no guarantee they would, and it's not obvious that they would be a million times more intelligent, or even 10% more intelligent; they might just post a million times more poorly-thought-out sarcastic comments on forums due to their poor impulse control, resulting in long pointless arguments where they insult each other and dismiss obviously correct ideas as absurd because they come from people they hate
"Over time" clearly means that he's talking about the far future, not the next few years when they're cleaning up the low-hanging fruit. Rather than "absurd ... fantastic and impossible" I would describe the Dyson-sphere outcome as inevitable, unless humanity goes extinct within a few centuries. Maybe you thought he meant next March?
In https://epoch.ai/gradient-updates/how-much-energy-does-chatg..., Josh You, Alex Erben, and Ege Erdil estimate that a typical GPT-4o query uses 0.3 watt hours, based on the estimates that it's a 400-billion-parameter MoE model in which ¼ of the parameters are activated for a given query, so each token requires 200 gigaflops (100 billion multiplies and 100 billion adds I guess), a typical query produces 500 output tokens (about a page of text), and they're running on 1-petaflop/s H100 GPUs, with the overall cluster consuming at peak 1500 watts per GPU, with a utilization rate of 10%, and that the cluster on average consumes 70% of peak power. This works out to 1.05 kilojoules, or about 0.25 kilocalories, the amount of food energy in one gram of carbohydrates.
So, a 320-kcal Double DeckerⓇ Taco™ works out to 1280 GPT-4o queries answered, and a standard 2000kcal/day diet works out to 8000 GPT-4o queries per day, if we believe You, Erben, and Erdil's estimate. For someone who is looking for GPT-4o-quality output, if you are producing less than 8000 pages of writing per day, you are less energy-efficient than GPT-4o.
______
† although there's no guarantee they would, and it's not obvious that they would be a million times more intelligent, or even 10% more intelligent; they might just post a million times more poorly-thought-out sarcastic comments on forums due to their poor impulse control, resulting in long pointless arguments where they insult each other and dismiss obviously correct ideas as absurd because they come from people they hate