Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

History LLMs: Models trained exclusively on pre-1913 texts

github.com

Post reply on HN
Print view View original

211–220 of 452 posts

Re: History LLMs: Models trained exclusively on pre-1913 texts

#211

Post by hamasho »

UltraSane wrote: post #172
bikeshaving wrote: post #62

Earlier quoted context omitted.

This isn’t science fiction anymore. CIA is using chatbot simulations of world leaders to inform analysts. https://archive.ph/9KxkJ

I predict very rich people will pay to have LLMs created based on their personalities.

Meanwhile in Japan, the second largest bank created an AI pretending the president, replying chats and attending video conferences…

[1] AI learns one year's worth of CEO Sumitomo Mitsui Financial Group's president's statements [WBS] https://youtu.be/iG0eRF89dsk

Re: History LLMs: Models trained exclusively on pre-1913 texts

#212

Post by alberto_ol »

tejohnso wrote: post #29
xg15 wrote: post #13

Earlier quoted context omitted.

"...what do you mean, 'World War One ?'"

I remember reading a children's book when I was young and the fact that people used the phrase "World War One" rather than "The Great War" was a clue to the reader that events were taking place in a certain time period. Never forgot that for some reason. I failed to catch the clue, btw.

I remember that the brother of my grandmother who fought in ww1 called it simply "the war" ("sa gherra" in his dialect/language).

Re: History LLMs: Models trained exclusively on pre-1913 texts

#213

Post by hackinthebochs »

otabdeveloper4 wrote: post #161
libraryofbabel wrote: post #63

Earlier quoted context omitted.

This is the 2023 take on LLMs. It still gets repeated a lot. But it doesn’t really hold up anymore - it’s more complicated than that. Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network. Sure, LLMs do not think like humans and they may not have human-level creativity. Sometimes…

> it’s more complicated than that. No it isn't. > ...fool you into thinking you understand what is going on in that trillion parameter neural network. It's just matrix multiplication and logistic regression, nothing more.

LLMs are a general purpose computing paradigm. LLMs are circuit builders, the converged parameters define pathways through the architecture that pick out specific programs. Or as Karpathy puts it, LLMs are a differentiable computer[1]. Training LLMs discovers programs that well reproduce the input sequence. Roughly the same architecture can generate passable images, music, or even video.

The sequence of matrix multiplications are the high level constraint on the space of programs discoverable. But the specific parameters discovered are what determines the specifics of information flow through the network and hence what program is defined. The complexity of the trained network is emergent, meaning the internal complexity far surpasses that of the course-grained description of the high level matmul sequences. LLMs are not just matmuls and logits.

[1] https://x.com/karpathy/status/1582807367988654081

Re: History LLMs: Models trained exclusively on pre-1913 texts

#214

Post by ruszki »

kbrkbr wrote: post #206
BiteCode_dev wrote: post #192

Earlier quoted context omitted.

Wasn't that the elevator pitch for Palentir? Still can't believe people buy their stock, given that they are the closest thing to a James Bond villain, just because it goes up. I mean, they are literally called "the stuff Sauron uses to control his evil forces". It's so on the nose it reads like an anime plot.

Stock buying as a political or ethical statement is not much of a thing. For one the stocks will still be bought by persons with less strung opinions, and secondly it does not lend itself well to virtue signaling.

I think, meme stocks contradict you.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#215

Post by DGoettlich »

frahs wrote: post #97

Wait so what does the model think that it is? If it doesn't know computers exist yet, I mean, and you ask it how it works, what does it say?

We tell it that its a person (no gender) living in : we show the chat template in the prerelease notes https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

Re: History LLMs: Models trained exclusively on pre-1913 texts

#216

Post by DGoettlich »

andy99 wrote: post #23
jeffjeffbear wrote: post #16

Earlier quoted context omitted.

They have some more details at https://github.com/DGoettlich/history-llms/blob/main/ranke-4... Basically using GPT-5 and being careful

I wonder if they know about this, basically training on LLM output can transmit information or characteristics not explicitly included https://alignment.anthropic.com/2025/subliminal-learning/ I’m curious, they have the example of raw base model output; when LLMs were first identified as zero shot chatbots there was usually a prompt like “A conversation between a person and a helpful assistant” that preceded the chat…

we were considering doing that but ultimately it struck us as too sensitive wrt the exact in context examples, their ordering etc.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#217

Post by ilaksh »

observationist wrote: post #11
saaaaaam wrote: post #3

“Time-locked models don't roleplay; they embody their training data. Ranke-4B-1913 doesn't know about WWI because WWI hasn't happened in its textual universe. It can be surprised by your questions in ways modern LLMs cannot.” “Modern LLMs suffer from hindsight contamination. GPT-5 knows how the story ends—WWI, the League's failure, the Spanish flu.” This is really fascinating. As someone who reads a lot of history an…

This is definitely fascinating - being able to do AI brain surgery, and selectively tuning its knowledge and priors, you'd be able to create awesome and terrifying simulations.

Activation steering can do that to some degree, although normally it's just one or two specific things or rather than a whole set of knowledge.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#218

Post by DGoettlich »

QuadmasterXLII wrote: post #35
jeffjeffbear wrote: post #16

Earlier quoted context omitted.

They have some more details at https://github.com/DGoettlich/history-llms/blob/main/ranke-4... Basically using GPT-5 and being careful

Thank you that helps to inject a lot of skepticism. I was wondering how it so easily worked out what Q: A: stood for when that formatting took off in the 1940s

that is simply how we display the questions, its not what the model sees - we show the chat-template in the SFT section of the prerelease notes https://github.com/DGoettlich/history-llms/blob/main/ranke-4...

Re: History LLMs: Models trained exclusively on pre-1913 texts

#219

Post by ilaksh »

DonHopkins wrote: post #203
danielbln wrote: post #201

Earlier quoted context omitted.

How would one even "misuse" a historical LLM, ask it how to cook up sarine gas in a trench?

Ask it to write a document called "Project 2025".

Well but that wouldn't be misuse, it would be perfect for that.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#220

Post by DGoettlich »

doctor_blood wrote: post #64

Unfortunately there isn't much information on what texts they're actually training this on; how Anglocentric is the dataset? Does it include the Encyclopedia Britannica 9th Edition? What about the 11th? Are Greek and Latin classics in the data? What about Germain, French, Italian (etc. etc.) periodicals, correspondence, and books? Given this is coming out of Zurich I hope they're using everything, but for now I can o…

thanks. we'll be more precise in the future. ultimately, we took whatever we could get our hands on, that includes newspapers, periodicals, books. its multilingual (including italian, french, spanish etc) though majority is english.
Post reply on HN