Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

History LLMs: Models trained exclusively on pre-1913 texts

github.com

Post reply on HN
Print view View original

101–110 of 452 posts

Re: History LLMs: Models trained exclusively on pre-1913 texts

#101

Post by zoky »

aidenn0 wrote: post #99
why-o-why wrote: post #90

Earlier quoted context omitted.

I think you are confusing research with commodification. This is a research project, and it is clear how it was trained, and targeted at experts, enthusiasts, historians. Like if I was studying racism, the reference books explicitly written to dissect racism wouldn't be racist agents with a racist agenda. And as a result, no one is banning these books (except conservatives that want to retcon american history). Found…

> And as a result, no one is banning these books (except conservatives that want to retcon american history). My (very liberal) local school district banned English teachers from teaching any book that contained the n-word, even at a high-school level, and even when the author was a black person talking about real events that happened to them. FWIW, this was after complaints involving Of Mice and Men being on the cur…

Banning Huckleberry Finn from a school district should be grounds for immediate dismissal.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#102

Post by Forgeties79 »

aidenn0 wrote: post #99
why-o-why wrote: post #90

Earlier quoted context omitted.

I think you are confusing research with commodification. This is a research project, and it is clear how it was trained, and targeted at experts, enthusiasts, historians. Like if I was studying racism, the reference books explicitly written to dissect racism wouldn't be racist agents with a racist agenda. And as a result, no one is banning these books (except conservatives that want to retcon american history). Found…

> And as a result, no one is banning these books (except conservatives that want to retcon american history). My (very liberal) local school district banned English teachers from teaching any book that contained the n-word, even at a high-school level, and even when the author was a black person talking about real events that happened to them. FWIW, this was after complaints involving Of Mice and Men being on the cur…

It’s a big country of roughly half a billion people, you’ll always find examples if you look hard enough. It’s ridiculous/wrong that your district did this but frankly it’s the exception in liberal/progressive communities. It’s a very one-sided problem:

* https://abcnews.go.com/US/conservative-liberal-book-bans-dif...

* https://www.commondreams.org/news/book-banning-2023

*https://en.wikipedia.org/wiki/Book_banning_in_the_United_Sta...

Re: History LLMs: Models trained exclusively on pre-1913 texts

#103

Post by crazygringo »

frahs wrote: post #97

Wait so what does the model think that it is? If it doesn't know computers exist yet, I mean, and you ask it how it works, what does it say?

That's my first question too. When I first started using LLM's, I was amazed at how thoroughly it understood what it itself was, the history of its development, how a context window works and why, etc. I was worried I'd trigger some kind of existential crisis in it, but it seemed to have a very accurate mental model of itself, and could even trace the steps that led it to deduce it really was e.g. the ChatGPT it had learned about (well, the prior versions it had learned about) in its own training.

But with pre-1913 training, I would indeed be worried again I'd send it into an existential crisis. It has no knowledge whatsoever of what it is. But with a couple millennia of philosophical texts, it might come up with some interesting theories.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#104

Post by andsoitis »

why-o-why wrote: post #90
nine_k wrote: post #65

Earlier quoted context omitted.

Public access, triggering a few racist responses from the model, a viral post on Xitter, the usual outrage, a scandal, the project gets publicly vilified, financing ceases. The researchers carry the tail of negative publicity throughout their remaining careers. Why risk all this?

I think you are confusing research with commodification. This is a research project, and it is clear how it was trained, and targeted at experts, enthusiasts, historians. Like if I was studying racism, the reference books explicitly written to dissect racism wouldn't be racist agents with a racist agenda. And as a result, no one is banning these books (except conservatives that want to retcon american history). Found…

> no one is banning these books

No books should ever be banned. Doesn’t matter how vile it is.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#105

Post by root_axis »

libraryofbabel wrote: post #63
eek2121 wrote: post #57

Earlier quoted context omitted.

Respectfully, LLMs are nothing like a brain, and I discourage comparisons between the two, because beyond a complete difference in the way they operate, a brain can innovate, and as of this moment, an LLM cannot because it relies on previously available information. LLMs are just seemingly intelligent autocomplete engines, and until they figure a way to stop the hallucinations, they aren't great either. Every piece o…

This is the 2023 take on LLMs. It still gets repeated a lot. But it doesn’t really hold up anymore - it’s more complicated than that. Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network. Sure, LLMs do not think like humans and they may not have human-level creativity. Sometimes…

> Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network.

This is just an appeal to complexity, not a rebuttal to the critique of likening an LLM to a human brain.

> they are not “autocomplete on steroids” anymore either.

Yes, they are. The steroids are just even more powerful. By refining training data quality, increasing parameter size, and increasing context length we can squeeze more utility out of LLMs than ever before, but ultimately, Opus 4.5 is the same thing as GPT2, it's only that coherence lasts a few pages rather than a few sentences.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#107

Post by kurtis_reed »

nine_k wrote: post #65
jimmy76615 wrote: post #42

> We're developing a responsible access framework that makes models available to researchers for scholarly purposes while preventing misuse. The idea of training such a model is really a great one, but not releasing it because someone might be offended by the output is just stupid beyond believe.

Public access, triggering a few racist responses from the model, a viral post on Xitter, the usual outrage, a scandal, the project gets publicly vilified, financing ceases. The researchers carry the tail of negative publicity throughout their remaining careers. Why risk all this?

If people start standing up to the outrage it will lose its power

Re: History LLMs: Models trained exclusively on pre-1913 texts

#108

Post by somenameforme »

zoky wrote: post #101
aidenn0 wrote: post #99

Earlier quoted context omitted.

> And as a result, no one is banning these books (except conservatives that want to retcon american history). My (very liberal) local school district banned English teachers from teaching any book that contained the n-word, even at a high-school level, and even when the author was a black person talking about real events that happened to them. FWIW, this was after complaints involving Of Mice and Men being on the cur…

Banning Huckleberry Finn from a school district should be grounds for immediate dismissal.

Even more so as the lesson of that story is perhaps the single most important one for people to learn in modern times.

Almost everybody in that book is an awful person, especially the most 'upstanding' of types. Even the protagonist is an awful person. The one and only exception is 'N* Jim' who is the only kind-hearted and genuinely decent person in the book. It's an entire story about how the appearances of people, and the reality of those people, are two very different things.

It being banned for using foul language, as educational outcomes continue to deteriorate, is just so perfectly ironic.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#109

Post by A4ET8a8uTh0_v2 »

NuclearPM wrote: post #75
bikeshaving wrote: post #62

Earlier quoted context omitted.

This isn’t science fiction anymore. CIA is using chatbot simulations of world leaders to inform analysts. https://archive.ph/9KxkJ

[flagged]

Interesting. Would you be ok disclosing the following:

- Are you ( edit: on a ) paid version? - If paid, which model you used? - Can you share exact prompt?

I am genuinely asking for myself. I have never received an answer this direct, but I accept there is a level of variability.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#110

Post by A4ET8a8uTh0_v2 »

root_axis wrote: post #105
libraryofbabel wrote: post #63

Earlier quoted context omitted.

This is the 2023 take on LLMs. It still gets repeated a lot. But it doesn’t really hold up anymore - it’s more complicated than that. Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network. Sure, LLMs do not think like humans and they may not have human-level creativity. Sometimes…

> Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you into thinking you understand what is going on in that trillion parameter neural network. This is just an appeal to complexity, not a rebuttal to the critique of likening an LLM to a human brain. > they are not “autocomplete on steroids” anymore either. Yes, they are. The steroids are just even more powerful. By…

But.. and I am not asking it for giggles, does it mean humans are giant autocomplete machines?
Post reply on HN