Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimeCapsuleLLM: LLM trained only on data from 1800-1875

github.com

Post reply on HN
Print view View original

191–200 of 334 posts

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#191

Post by bigfudge »

Shorel wrote: post #162
bazzargh wrote: post #31

Earlier quoted context omitted.

You would find things in there that were already close to QM and relativity. The Michelson-Morley experiment was 1887 and Lorentz transformations came along in 1889. The photoelectric effect (which Einstein explained in terms of photons in 1905) was also discovered in 1887. William Clifford (who _died_ in 1889) had notions that foreshadowed general relativity: "Riemann, and more specifically Clifford, conjectured tha…

Then that experiment is even more interesting, and should be done. My own prediction is that the LLMs would totally fail at connecting the dots, but a small group of very smart humans can. Things don't happen all of a sudden, but they also don't happen everywhere. Most people in most parts of the world would never connect the dots. Scientific curiosity is something valuable and fragile, that we just take for granted.

One of the reasons they don’t happen everywhere is because there are just a few places at any given point in time where there are enough well connected and educated individuals who are in a position to even see all the dots let alone connect them. This doesn’t discount the achievement of an LLM also manages to, but I think it’s important to recognise that having enough giants in sight is an important prerequisite to standing on their shoulders

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#192

Post by PaulDavisThe1st »

viccis wrote: post #190
PaulDavisThe1st wrote: post #186

Earlier quoted context omitted.

My point is that we know very little about the sort of "thought" that we are capable of either. I agree that LLMs cannot do what we typical refer to as "thought", but I thnk it is possible that we do a LOT less of that than we think when we are "thinking" (or more precisely, having the experience of thinking).

How does this worldview reconcile the fact that thought demonstrably exists independent of either language or vision/audio sense?

I don't see a need to reconcile them.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#193

Post by bowmessage »

kristopolous wrote: post #179
dogma1138 wrote: post #2

Would be interesting to train a cutting edge model with a cut off date of say 1900 and then prompt it about QM and relativity with some added context. If the model comes up with anything even remotely correct it would be quite a strong evidence that LLMs are a path to something bigger if not then I think it is time to go back to the drawing board.

It's going to be divining tea leaves. It will be 99% wrong and then someone will say 'oh but look at this tea leaf over here! It's almost correct"'

Look! It made another TODO-list app on the first try!

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#194

Post by johnsmith1840 »

SoftTalker wrote: post #189
johnsmith1840 wrote: post #63

Earlier quoted context omitted.

There is some ability for it to make novel connections but it's pretty small. You can see this yourself having it build novel systems. It largely cannot imaginr anything beyond the usual but there is a small part that it can. This is similar to in context learning, it's weak but it is there. It would be incredible if meta learning/continual learning found a way to train exactly for novel learning path. But that's lit…

Is this so different from what we see in humans? Most people do not think very creatively. They apply what they know in situations they are familiar with. In unfamiliar situations they don't know what to do and often fail to come up with novel solutions. Or maybe in areas where they are very experienced they will come up with something incrementally better than before. But occasionally a very exceptional person makes…

Sure we make small steps at the time but we compound these unlike AI.

AI cannot compound their learnings for the foreseeable future

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#195

Post by fluoridation »

radarsat1 wrote: post #155

Heh, at least this wouldn't spread emojis all over my readmes. Hm, come to think of it I wonder how much tokenization is affected. Another thought, just occurred when thinking about readmes and coding LLMs: obviously this model wouldn't have any coding knowledge, but I wonder if it could be possible to combine this somehow with a modern LLM in such a way that it does have coding knowledge, but it renders out all the…

Just have the models converse with each other?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#196

Post by fluoridation »

patcon wrote: post #120

> OCR noise (“Digitized by Google”) still present in outputs This feels like a neat sci-fi short story hook to explain the continuous emergence of God as an artifact of a simulation

I'm reminded of SD models that put vaguely-shaped Patreon logos in the corner.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#197

Post by viccis »

PaulDavisThe1st wrote: post #192
viccis wrote: post #190

Earlier quoted context omitted.

How does this worldview reconcile the fact that thought demonstrably exists independent of either language or vision/audio sense?

I don't see a need to reconcile them.

Which is why it's incoherent!

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#199

Post by PaulDavisThe1st »

viccis wrote: post #197
PaulDavisThe1st wrote: post #192

Earlier quoted context omitted.

I don't see a need to reconcile them.

Which is why it's incoherent!

I'm not clear that it has to be coherent at this point in the history of our understanding of cognition. We barely know what we're even talking about most of the time ...

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#200

Post by SV_BubbleTime »

radiothomp wrote: post #105

A LLM trained only on data from certain time periods to ~reduce modern bias~ enhance past bias

Doesn’t that seem useful though? Isn’t that why I’m forced to read “This movie was made a time when racial stereotypes were different and not well considered” or whatever on old movies?

I think talking to a legit trained LLM from a different era would be rad. But… this seems the opposite of Gemini making black lady popes and Native American Nazis… that these views wouldn’t really be “allowed” (published by anyone that wants AI funding money).

Post reply on HN