Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude vs. Gemini: Testing on 1M Tokens of Context

every.to

Post reply on HN
Print view View original

1–10 of 49 posts

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#4

Post by sebastienbarre »

thefourthchime wrote: post #3
arnaudsm wrote: post #2

https://archive.is/sb7D5

Does anyone else have trouble with the archive rendering of that? It seemed to also have the pop up.

You can delete the div with id=subscribe-popup from the dev tools for a better view.

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#7

Post by bitpush »

irthomasthomas wrote: post #5

So sonnet-4 is faster than gemini-2.5-flash at long context. That is surprising. Especially since Gemini runs on those fast TPUS.

> Claude’s overall response was consistently around 500 words—Flash and Pro delivered 3,372 and 1,591 words by contrast.

It isnt clear from the article whether the time they quote is time-to-first-token or time to completion. If it is latter, then it makes sense why gemini* would take longer even with similar token throughput.

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#9

Post by curl-up »

irthomasthomas wrote: post #5

So sonnet-4 is faster than gemini-2.5-flash at long context. That is surprising. Especially since Gemini runs on those fast TPUS.

Note that (in the first test, the only one where output length is reported), Gemini Pro returned more than 3x the amount of text, at less than 2x the amount of time. From my experience with Gemini, that time was probably mainly spent on thinking, length of which is not reported here. So looking at pure TPS of output, Gemini is faster, but without clear info on the thinking time/length, it's impossible to judge.
Post reply on HN