Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TimeCapsuleLLM: LLM trained only on data from 1800-1875

github.com

Post reply on HN
Print view View original

91–100 of 334 posts

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#91

Post by Alex2037 »

catigula wrote: post #71
metalliqaz wrote: post #42

Earlier quoted context omitted.

Yann LeCun spoke explicitly on this idea recently and he asserts definitively that the LLM would not be able to add anything useful in that scenario. My understanding is that other AI researchers generally agree with him, and that it's mostly the hype beasts like Altman that think there is some "magic" in the weights that is actually intelligent. Their payday depends on it, so it is understandable. My opinion is that…

This is definitely wrong, most AI researchers DO NOT agree with LeCun. Most ML researchers think AGI is imminent.

their employment and business opportunities depend on the hype, so they will continue to 'think' that (on xitter) despite the current SOTA of transformers-based models being 3 year old GPT4, and no revolutionary new architecture in sight.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#92

Post by sleet_spotter »

actionfromafar wrote: post #72
matthewh806 wrote: post #53

Earlier quoted context omitted.

I presume that's what the parent post is trying to get at? Seeing if, given the cutting edge scientific knowledge of the day, the LLM is able to synthesis all it into a workable theory of QM by making the necessary connections and (quantum...) leaps Standing on the shoulders of giants, as it were

Yeah but... we still might not know if it could do that because we were really close by 1900 or if the LLM is very smart.

Well, if one had enough time and resources, this would make for an interesting metric. Could it figure it out with cut-off of 1900? If so, what about 1899? 1898? What context from the marginal year was key to the change in outcome?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#93

Post by throwaway18875 »

simonw wrote: post #4

Anyone seen a low-friction way to run prompts through this yet, either via a hosted API or chat UI or a convenient GGML or MLX build that runs in Ollama or llama.cpp or LM Studio?

Currently running it using LM Studio. It can download it from Hugging Face. It generates incoherent text though

===

You:

I pray you, who is this Master Newton?

timecapsulellm-v2-1800-1875-mlx:

TI offer to pay you the very same fee as you did before. It was not in the power of your master to deliver the letter to your master. He did. I will be with you as soon as I can keep my word. It is not at all clear, whether the letter has been sent or not. It is not at all clear: but it is clear also that it was written by the person who gave it. "No," I said, "I cannot give it to you." There, the letter was sent to me. "The letter is yours, I believe," I said. "But, I hope, you will not refuse to give it to me?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#94

Post by palmotea »

matthewh806 wrote: post #53
bazzargh wrote: post #31

Earlier quoted context omitted.

You would find things in there that were already close to QM and relativity. The Michelson-Morley experiment was 1887 and Lorentz transformations came along in 1889. The photoelectric effect (which Einstein explained in terms of photons in 1905) was also discovered in 1887. William Clifford (who _died_ in 1889) had notions that foreshadowed general relativity: "Riemann, and more specifically Clifford, conjectured tha…

I presume that's what the parent post is trying to get at? Seeing if, given the cutting edge scientific knowledge of the day, the LLM is able to synthesis all it into a workable theory of QM by making the necessary connections and (quantum...) leaps Standing on the shoulders of giants, as it were

But that's not the OP's challenge, he said "if the model comes up with anything even remotely correct." The point is there were things already "remotely correct" out there in 1900. If the LLM finds them, it wouldn't "be quite a strong evidence that LLMs are a path to something bigger."

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#95

Post by PaulDavisThe1st »

viccis wrote: post #78
dogma1138 wrote: post #21

Earlier quoted context omitted.

It doesn’t need to know about QM or reactivity just about the building blocks that led to them. Which were more than around in the year 1900. In fact you don’t want it to know about them explicitly just have enough background knowledge that you can manage the rest via context.

LLMs are models that predict tokens. They don't think, they don't build with blocks. They would never be able to synthesize knowledge about QM.

I am a deep LLM skeptic.

But I think there are also some questions about the role of language in human thought that leave the door just slightly ajar on the issue of whether or not manipulating the tokens of language might be more central to human cognition than we've tended to think.

If it turned out that this was true, then it is possible that "a model predicting tokens" has more power than that description would suggest.

I doubt it, and I doubt it quite a lot. But I don't think it is impossible that something at least a little bit along these lines turns out to be true.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#96

Post by thinkingemote »

al_borland wrote: post #41
feisty0630 wrote: post #33

Earlier quoted context omitted.

I fail to see how the two concepts equate. LLMs have neither intelligence nor problem-solving abillity (and I won't be relaxing the definition of either so that some AI bro can pretend a glorified chatbot is sentient) You would, at best, be demonstrating that the sharing of knowledge across multiple disciplines and nations (which is a relatively new concept - at least at the scale of something like the internet) lead…

I've seen many futurists claim that human innovation is dead and all future discoveries will be the results of AI. If this is true, we should be able to see AI trained on the past figure it's way to various things we have today. If it can't do this, I'd like said futurists to quiet down, as they are discouraging an entire generation of kids who may go on to discover some great things.

I'm struggling to get a handle on this idea. Is the idea that today's data will be the data of the past, in the future?

So if it can work with whats now past, it will be able to work with the past in the future?

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#97

Post by jaggederest »

echoangle wrote: post #90
scottlamb wrote: post #77

Earlier quoted context omitted.

What's the bar here? Does anyone say "we don't know if Einstein could do this because we were really close or because he was really smart?" I by no means believe LLMs are general intelligence, and I've seen them produce a lot of garbage, but if they could produce these revolutionary theories from only <= year 1900 information and a prompt that is not ridiculously leading, that would be a really compelling demonstrati…

> Does anyone say "we don't know if Einstein could do this because we were really close or because he was really smart?" Kind of, how long would it have realistically taken for someone else (also really smart) to come up with the same thing if Einstein wouldn't have been there?

Well, we know many watershed moments in history were more a matter of situation than the specific person - an individual genius might move things by a decade or two, but in general the difference is marginal. True bolt-out-of-the-blue developments are uncommon, though all the more impressive for that fact, I think.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#98

Post by ben_w »

opponent4 wrote: post #68
ben_w wrote: post #28

Earlier quoted context omitted.

> Could this be an experiment to show how likely LLMs are to lead to AGI, or at least intelligence well beyond our current level? You'd have to be specific what you mean by AGI: all three letters mean a different thing to different people, and sometimes use the whole means something not present in the letters. > If you could only give it texts and info and concepts up to Year X, well before Discovery Y, could we then…

> You'd have to be specific what you mean by AGI Well, they obviously can't. AGI is not science, it's religion. It has all the trappings of religion: prophets, sacred texts, origin myth, end-of-days myth and most importantly, a means to escape death. Science? Well, the only measure to "general intelligence" would be to compare to the only one which is the human one but we have absolutely no means by which to describe…

Cargo cults are a religion, the things they worship they do not understand, but the planes and the cargo themselves are real.

There's certainly plenty of cargo-culting right now on AI.

Sacred texts, I don't recognise. Yudkowsky's writings? He suggests wearing clown shoes to avoid getting a cult of personality disconnected from the quality of the arguments, if anyone finds his works sacred, they've fundamentally misunderstood him:

  I have sometimes thought that all professional lectures on rationality should be delivered while wearing a clown suit, to prevent the audience from confusing seriousness with solemnity.
- https://en.wikiquote.org/wiki/Eliezer_Yudkowsky

Prophets forecasting the end-of-days, yes, but this too from climate science, from everyone who was preparing for a pandemic before covid and is still trying to prepare for the next one because the wet markets are still around, from economists trying to forecast growth or collapse and what will change any given prediction of the latter into the former, and from the military forces of the world saying which weapon systems they want to buy. It does not make a religion.

A means to escape death, you can have. But it's on a continuum with life extension and anti-aging medicine, which itself is on a continuum with all other medical interventions. To quote myself:

  Taking a living human's heart out without killing them, and replacing it with one you got out a corpse, that isn't the magic of necromancy, neither is it a prayer or ritual to Sekhmet, it's just transplant surgery.

  …

  Immunity to smallpox isn't a prayer to the Hindu goddess Shitala (of many things but most directly linked with smallpox), and it isn't magic herbs or crystals, it's just vaccines.
- https://benwheatley.github.io/blog/2025/06/22-13.21.36.html

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#99

Post by canjobear »

auraham wrote: post #88
dash2 wrote: post #81

Mm. I'm a bit sceptical of the historical expertise of someone who thinks that "Who art Henry" is 19th century language. (It's not actually grammatically correct English from any century whatever: "art" is the second person singular, so this is like saying "who are Henry?")

Can you elaborate on this? After skimming the README, I understand that "Who art Henry" is the prompt. What should be the correct 19th century prompt?

"Who art Henry?" was never grammatical English. "Art" was the second person singular present form of "to be" and it was already archaic by the 17th century. "Who is Henry?" would be fine.

Re: TimeCapsuleLLM: LLM trained only on data from 1800-1875

#100

Post by shaky-carrousel »

p_j_w wrote: post #76
catigula wrote: post #71

Earlier quoted context omitted.

This is definitely wrong, most AI researchers DO NOT agree with LeCun. Most ML researchers think AGI is imminent.

Who is in this group of ML researchers?

People with OpenAI shares, probably.
Post reply on HN