Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimeCapsuleLLM: LLM trained only on data from 1800-1875

github.com

Post reply on HN
Print view View original

291–300 of 334 posts

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#291

Post by swalsh »

dogma1138 wrote: post #2

Would be interesting to train a cutting edge model with a cut off date of say 1900 and then prompt it about QM and relativity with some added context. If the model comes up with anything even remotely correct it would be quite a strong evidence that LLMs are a path to something bigger if not then I think it is time to go back to the drawing board.

Could be an interesting experiment, but its not conclusive proof one way or another. So much of what makes LLMs so great today (vs gpt 3.5) would not be in that dataset. The training to turn these models into coding savants has generalized to other areas just as one example.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#292

Post by redman25 »

dogma1138 wrote: post #2

Would be interesting to train a cutting edge model with a cut off date of say 1900 and then prompt it about QM and relativity with some added context. If the model comes up with anything even remotely correct it would be quite a strong evidence that LLMs are a path to something bigger if not then I think it is time to go back to the drawing board.

It's a base model. It hasn't been instruction tuned to "solve problems" necessarily. All it can do is attempt to complete text given some starting text.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#293

Post by andy12_ »

somenameforme wrote: post #278
andy12_ wrote: post #275

Earlier quoted context omitted.

The problem is that so far, SOTA generalist models are not excellent at just one particular task. They have a very wide range of tasks they are good at, and good scores in one particular benchmarks correlates very strongly with good scores in almost all other benchmarks, even esoteric benchmarks that AI labs certainly didn't train against. I'm sure, without any uncertainty, that any generalist model able to do what E…

I see things rather differently. Here's a few points in no particular order: (1) - A major part of the challenge is in not being directed towards something. There was no external guidance for Einstein - he wasn't even a formal researcher at the time of his breakthroughs. An LLM might be able to be handheld towards relativity, though I doubt it, but given the prompt of 'hey find something revolutionary' it's obviously…

> I don't entirely understand your extreme optimism towards LLMs given this proclivity for hallucination

Simply because I don't see hallucinations as a permanent problem. I see that models keep improving more and more in this regard, and I don't see why the hallucination rate can't be abirtrarily reduced with further improvements to the architecture. When I ask Claude about obscure topics, it correctly replies "I don't know", where past models would have hallucinated an answer. When I use GPT 5.2-thinking for my ML research job, I pretty much never encounter hallucinations.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#294

Post by somenameforme »

andy12_ wrote: post #293
somenameforme wrote: post #278

Earlier quoted context omitted.

I see things rather differently. Here's a few points in no particular order: (1) - A major part of the challenge is in not being directed towards something. There was no external guidance for Einstein - he wasn't even a formal researcher at the time of his breakthroughs. An LLM might be able to be handheld towards relativity, though I doubt it, but given the prompt of 'hey find something revolutionary' it's obviously…

> I don't entirely understand your extreme optimism towards LLMs given this proclivity for hallucination Simply because I don't see hallucinations as a permanent problem. I see that models keep improving more and more in this regard, and I don't see why the hallucination rate can't be abirtrarily reduced with further improvements to the architecture. When I ask Claude about obscure topics, it correctly replies "I don…

Hahah, well you working in the field probably explains your optimism more than your words! If you pretty much never encounter hallucinations with GPT then you're probably dealing with it on topics where there's less of a right or wrong answer. I encounter them literally every single time I start trying to work out a technical problem with it.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#295

Post by shmeeed »

CGMthrowaway wrote: post #188

Is there a link where I can try it out? Edit: I figured it out "The Lord of the Rings uding the army under the command of his brother, the Duke of York, and the Duke of Richmond, who fell in the battle on the 7th of April, 1794. The Duke of Ormond had been appointed to the command of the siege of St. Mark's, and had received the victory of the Rings, and was thus commanded to move with his army to the relief of Shenh…

Meanwhile, top comments are having visions about the emergence of Einstein-level insights.

There's a disconnect somewhere that I can't quite put my finger on. Am I just lacking imagination?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#296

Post by optimalsolver »

famouswaffles wrote: post #240
PaulDavisThe1st wrote: post #236

Earlier quoted context omitted.

please pass on a link to a solid research paper that supports the idea that to "find the next probable token", LLM's manipulate concepts ... just one will do.

Revealing emergent human-like conceptual representations from language prediction - https://www.pnas.org/doi/10.1073/pnas.2512514122 Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task - https://openreview.net/forum?id=DeG07_TcZvT On the Biology of a Large Language Model - https://transformer-circuits.pub/2025/attribution-graphs/bio... Emergent Introspective Awareness in Large Langu…

@PaulDavisThe1st I'd love to hear your take on these papers.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#299

Post by mannykannot »

TeMPOraL wrote: post #221
mannykannot wrote: post #217

Earlier quoted context omitted.

If (as you seem to be suggesting) relativity was effectively lying there on the table waiting for Einstein to just pick it up, how come it blindsided most, if not quite all, of the greatest minds of his generation?

That's the case with all scientific discoveries - pieces of prior work get accumulated, until it eventually becomes obvious[0] how they connect, at which point someone[1] connects the dots, making a discovery... and putting it on the table, for the cycle to repeat anew. This is, in a nutshell, the history of all scientific and technological progress. Accumulation of tiny increments. -- [0] - To people who happen to h…

Sure - and climbing a mountain is just putting one foot down higher than it was before and repeating, once you abstract away all the hard parts.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#300

Post by andai »

geocar wrote: post #238
andai wrote: post #130

Earlier quoted context omitted.

Who art thou? (Well, not 19th century...)

The problem is the subjunctive mood of the word "art". "Art thou" should be translated into modern English as "are you to be", and so works better with things (what are you going to be), or people who are alive, and have a future (who are you going to be?). Those are probably the contexts you are thinking of.

Wherefore are you going to be Romeo?
Post reply on HN