Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimeCapsuleLLM: LLM trained only on data from 1800-1875

github.com

Post reply on HN
Print view View original

41–50 of 334 posts

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#41

Post by al_borland »

feisty0630 wrote: post #33
eqmvii wrote: post #14

Could this be an experiment to show how likely LLMs are to lead to AGI, or at least intelligence well beyond our current level? If you could only give it texts and info and concepts up to Year X, well before Discovery Y, could we then see if it could prompt its way to that discovery?

I fail to see how the two concepts equate. LLMs have neither intelligence nor problem-solving abillity (and I won't be relaxing the definition of either so that some AI bro can pretend a glorified chatbot is sentient) You would, at best, be demonstrating that the sharing of knowledge across multiple disciplines and nations (which is a relatively new concept - at least at the scale of something like the internet) lead…

I've seen many futurists claim that human innovation is dead and all future discoveries will be the results of AI. If this is true, we should be able to see AI trained on the past figure it's way to various things we have today. If it can't do this, I'd like said futurists to quiet down, as they are discouraging an entire generation of kids who may go on to discover some great things.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#42

Post by metalliqaz »

dogma1138 wrote: post #2

Would be interesting to train a cutting edge model with a cut off date of say 1900 and then prompt it about QM and relativity with some added context. If the model comes up with anything even remotely correct it would be quite a strong evidence that LLMs are a path to something bigger if not then I think it is time to go back to the drawing board.

Yann LeCun spoke explicitly on this idea recently and he asserts definitively that the LLM would not be able to add anything useful in that scenario. My understanding is that other AI researchers generally agree with him, and that it's mostly the hype beasts like Altman that think there is some "magic" in the weights that is actually intelligent. Their payday depends on it, so it is understandable. My opinion is that LeCun is probably correct.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#43

Post by lcfcjs6 »

feisty0630 wrote: post #25
abhishekjha wrote: post #17

Oh I have really been thinking long about this. The intelligence that we have in these models represent a time. Now if I train a foundation models with docs from library of Alexandria and only those texts of that period, I would have a chance to get a rudimentary insight on what the world was like at that time. And maybe time shift further more.

> I would have a chance to get a rudimentary insight on what the world was like at that time Congratulations, you've reinvented the history book (just with more energy consumption and less guarantee of accuracy)

[dead]

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#44

Post by gordonhart »

feisty0630 wrote: post #25
abhishekjha wrote: post #17

Oh I have really been thinking long about this. The intelligence that we have in these models represent a time. Now if I train a foundation models with docs from library of Alexandria and only those texts of that period, I would have a chance to get a rudimentary insight on what the world was like at that time. And maybe time shift further more.

> I would have a chance to get a rudimentary insight on what the world was like at that time Congratulations, you've reinvented the history book (just with more energy consumption and less guarantee of accuracy)

History books, especially those from classical antiquity, are notoriously not guaranteed to be accurate either.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#45

Post by feisty0630 »

zingababba wrote: post #23

LOL PROMPT:Charles Darwin Charles DarwinECCEMACY. Sir, — The following case is interesting to me : — I was in London a fortnight, and was much affected with an attack of rheumatism. The first attack of rheumatism was a week before I saw you, and the second when I saw you, and the third when I saw you, and the third in the same time. The second attack of gout, however, was not accompanied by any febrile symptoms, but…

Interesting that it reads a bit like it came from a Markov chain rather than an LLM. Perhaps limited training data?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#46

Post by myrmidon »

There was a discussion around a very similar model (Qwen3 based) some weeks ago:

https://news.ycombinator.com/item?id=46319826

I found it particularly thought-inspiring how a model with training from that time period completely lacks context/understanding of what it is itself, but then I realized that we are the same (at least for now).

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#49

Post by Rebuff5007 »

eqmvii wrote: post #14

Could this be an experiment to show how likely LLMs are to lead to AGI, or at least intelligence well beyond our current level? If you could only give it texts and info and concepts up to Year X, well before Discovery Y, could we then see if it could prompt its way to that discovery?

OF COURSE!

The fact that tech leaders espouse the brilliance of LLMs and don't use this specific test method is infuriating to me. It is deeply unfortunate that there is little transparency or standardization of the datasets available for training/fine tuning.

Having this be advertised will make more interesting and informative benchmarks. OEM models that are always "breaking" the benchmarks are doing so with improved datasets as well as improved methods. Without holding the datasets fixed, progress on benchmarks are very suspect IMO.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#50

Post by kopollo »

imjonse wrote: post #9
dogma1138 wrote: post #2

Would be interesting to train a cutting edge model with a cut off date of say 1900 and then prompt it about QM and relativity with some added context. If the model comes up with anything even remotely correct it would be quite a strong evidence that LLMs are a path to something bigger if not then I think it is time to go back to the drawing board.

I suppose the vast majority of training data used for cutting edge models was created after 1900.

I don't know if this is related to the topic, but GPT5 can convert an 1880 Ottoman archival photograph to English, and without any loss of quality.
Post reply on HN