Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can Gemini 1.5 read all the Harry Potter books at once?

twitter.com

Post reply on HN
Print view View original

31–40 of 40 posts

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#31

Post by chmod775 »

munchler wrote: post #14
viraptor wrote: post #13

Earlier quoted context omitted.

The training data may not be HP itself. It may be millions of pages summarising/discussing/dissecting HP, which already contain the relationships spelled out better than in the book itself.

That's true, but the model still analyzed all that disparate information and produced a very detailed graph of the relevant relationships. If anyone can show that the graph itself was in the training data, then I would agree that it's not a good test.

> disparate information

I wouldn't call it disparate when there's about a dozen wikis each spelling it out like this: https://harrypotter.fandom.com/wiki/Severus_Snape

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#32

Post by lukan »

vsnf wrote: post #20
munchler wrote: post #14

Earlier quoted context omitted.

That's true, but the model still analyzed all that disparate information and produced a very detailed graph of the relevant relationships. If anyone can show that the graph itself was in the training data, then I would agree that it's not a good test.

If eat my hat if multiple graphs almost exactly like this one weren’t in the training days. This is like fandoms 101.

The frustrating thing about all this speculations is, that we don't know what was in the training data, but I think we should know that, to have any meaningful discussion about it.

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#33

Post by actionfromafar »

lukan wrote: post #32
vsnf wrote: post #20

Earlier quoted context omitted.

If eat my hat if multiple graphs almost exactly like this one weren’t in the training days. This is like fandoms 101.

The frustrating thing about all this speculations is, that we don't know what was in the training data, but I think we should know that, to have any meaningful discussion about it.

We should. However in this case, isn't it a bit of a stretch to assume they didn't put just about everything in the training data?

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#34

Post by sinuhe69 »

It would be more impressive (and cleaner, btw) if it was fed with fan-fiction books and not the original books. Then we can see what it can make out of the context and what it "borrows" from the training data.

Why fan-fiction? Well, fan-fictions are not famous enough to be included in any training corpus, I believe. But fan-fictions of Harry Potter are numerous enough to test the context limit. There are also similarities and distinctions from the originals, which require correct recall to distinguish between them. That would be a good test, isn't it?

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#35

Post by hhh »

sinuhe69 wrote: post #34

It would be more impressive (and cleaner, btw) if it was fed with fan-fiction books and not the original books. Then we can see what it can make out of the context and what it "borrows" from the training data. Why fan-fiction? Well, fan-fictions are not famous enough to be included in any training corpus, I believe. But fan-fictions of Harry Potter are numerous enough to test the context limit. There are also similar…

Why are fanfictions not famous enough to be included? There are huge archives of them online, which make for great sources of information. Archive of our own for example lists over 12 million works on their site.

It’s cheap to gather, unlikely to have any recourse, and has a huge range of quality.

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#36

Post by julianpye »

So, according to Gemini pricing, the call would cost approx. $11. Now, hopefully all goes to plan and the input correct and the result is what you wished for. If not, how many $11 calls do you need? Sure, pricing will go down, but my observation is that people just ignore the cost of context. When it's all about tech it's fine, but not if it's about efficiency.

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#38

Post by Yiin »

julianpye wrote: post #36

So, according to Gemini pricing, the call would cost approx. $11. Now, hopefully all goes to plan and the input correct and the result is what you wished for. If not, how many $11 calls do you need? Sure, pricing will go down, but my observation is that people just ignore the cost of context. When it's all about tech it's fine, but not if it's about efficiency.

If you're a business wanting to process highly technical training material into shorter handbooks, paying $11/each is practically free.

Re: Can Gemini 1.5 read all the Harry Potter books at once?

#39

Post by joshspankit »

munchler wrote: post #14
viraptor wrote: post #13

Earlier quoted context omitted.

The training data may not be HP itself. It may be millions of pages summarising/discussing/dissecting HP, which already contain the relationships spelled out better than in the book itself.

That's true, but the model still analyzed all that disparate information and produced a very detailed graph of the relevant relationships. If anyone can show that the graph itself was in the training data, then I would agree that it's not a good test.

It would have been fairly trivial to AB test this where the other side is to ask the same question but without all the books in-window.
Post reply on HN