He is comparing energy spend during inference in humans with energy spend during training in LLM's.
Humans spend their lifetimes training their brain so one would have to sum up the total training time if you are going to compare it to the training time of LLM's.
At age 30 the total energy use of the brain sums up to about 5000 Wh, which is 1440 times more efficient.
But at age 30 we didn't learn good representations for most of the stuff on the internet so one could argue that given the knowledge learned, LLMs outperform the brain on energy consumption.
That said, LLM's have it easier as they are already learning from an abstract layer (language) that already has a lot of good representations while humans have to first learn to parse this through imagery.
Half the human brain is dedicated to processing imagery, so one could argue the human brain only spend 2500 Wh on equivalent tasks which makes it 3000x more efficient.
Liked the article though, didn't know about HNSW's.
Edit: made some quick comparisons for inference
Assuming a human spends 20 minutes answering in a well-thought out fashion.
Human watt-hours: 0.00646
GPT-4 watt-hours (openAI data): 0.833
That makes our brains still 128x more energy efficient but people spend a lot more time to generate the answer.
Edit: numbers are off by 1000 as I used calories instead of kilocalories to calculate brain energy expense.
Corrected:
human brains are 1.44x more efficient during training and 0.128x (or 8x less efficient) during inference.