Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude vs. Gemini: Testing on 1M Tokens of Context

every.to

Post reply on HN
Print view View original

41–49 of 49 posts

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#41

Post by thomastjeffery »

ratg13 wrote: post #38
thomastjeffery wrote: post #37

Earlier quoted context omitted.

How?

Read the text, click the links, let it sink in

I did that, and I assume GP did as well.

There is some information that you assume to have shared that we are not picking up on.

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#42

Post by llm_nerd »

koakuma-chan wrote: post #14
gcr wrote: post #10

Earlier quoted context omitted.

The entire HP series is about one million words.

Harry Potter and the Order of Phoenix alone is 400K tokens.

Curious, I found an epub, converted it to a txt, and dumped it into the Qwen3 tokenizer. It yielded 359,088 tokens, end to end.

Using the GPT-4 tokenizer (cl100k_base) yields 349,371 tokens.

Recent Google and Anthropic models do not have local tokenizers and ridiculously make you call their APIs to do it, so no idea about those.

Just thought that was interesting.

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#43

Post by bredren »

chrismustcode wrote: post #30
koakuma-chan wrote: post #21

Earlier quoted context omitted.

It's 2M tokens for Gemini.

That was previous iterations, 2.5 is 1 million context window https://ai.google.dev/gemini-api/docs/models (context window is details under model variant section with + signs) They were meant to crank 2.5 to 2 million at some point though, maybe waiting now till 3?

Maybe consuming the resources internally.

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#44

Post by matesz »

rat9988 wrote: post #29
matesz wrote: post #27

Earlier quoted context omitted.

Geminis free tier allows maybe 5 messages on average, for 2.5 pro at least and this is not usable. I’m using Claude Pro for daily driver and Gemini / ChatGPT free tiers.

> Geminis free tier allows maybe 5 messages on average, for 2.5 pro at least and this is not usable. Not on ai studio.

Oh my... I didn't know about Gemini Studio and didn't expect the possibility of it existing. Thanks for correcting!

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#45

Post by rafaelmn »

akomtu wrote: post #19

IMO, a good contest between LLMs would be data compression. Each LLM is given the same pile of text, and then asked to create compact notes that fit into N pages of text. Then the original text is replaced with their notes and they need to answer a bunch of questions about the original text using the notes alone.

Summarization ? I'm pretty sure there are benchmarks for this because people used summarization to build search indexes (at least a few years ago when I was working on this they did and there were benchmarks)

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#46

Post by what_ever »

thomastjeffery wrote: post #41
ratg13 wrote: post #38

Earlier quoted context omitted.

Read the text, click the links, let it sink in

I did that, and I assume GP did as well. There is some information that you assume to have shared that we are not picking up on.

May be ask your favorite AI about what you are missing. Or may be ask using AI studio as that won't rate limit you ;)

Re: Claude vs. Gemini: Testing on 1M Tokens of Context

#47

Post by irthomasthomas »

ancientworldnow wrote: post #32
irthomasthomas wrote: post #20

Earlier quoted context omitted.

Do they rent them from Google? Or are they a different brand?

Google provides them.

Ah cool I'll have to read up on that, I had thought that google was hoarding them.
Post reply on HN