Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

History LLMs: Models trained exclusively on pre-1913 texts

github.com

Post reply on HN
Print view View original

261–270 of 452 posts

Re: History LLMs: Models trained exclusively on pre-1913 texts

#261

Post by nottorp »

ajuc wrote: post #258
nottorp wrote: post #234

Earlier quoted context omitted.

Almost no scifi has predicted world changing "qualitative" changes. As an example, portable phones have been predicted. Portable smartphones that are more like chat and payment terminals with a voice function no one uses any more ... not so much.

Stanisław Lem predicted Kindle back in 1950s, together with remote libraries, global network, touchscreens and audiobooks.

And Jules verne predicted rockets. I still move that it's quantitative predictions not qualitative.

I mean, all Kindle does for me is save me space. I don't have to store all those books now.

Who predicted the humble internet forum though? Or usenet before it?

Re: History LLMs: Models trained exclusively on pre-1913 texts

#262

Post by rhdunn »

tonymet wrote: post #44

I would like to see what their process for safety alignment and guardrails is with that model. They give some spicy examples on github, but the responses are tepid and a lot more diplomatic than I would expect. Moreover, the prose sounds too modern. It seems the base model was trained on a contemporary corpus. Like 30% something modern, 70% Victorian content. Even with half a dozen samples it doesn't seem distinct en…

Using texts upto 1913 includes works like The Wizard of Oz (1900, with 8 other books upto 1913), two of the Anne of Green Gables books (1908 and 1909), etc. All of which read modern.

The Victorian era (1837-1901) covers works from Charles Dickens and the like which are still fairly modern. These would have been part of the initial training before the alignment to the 1900-cutoff texts which are largely modern in prose with the exception of some archaic language and the lack of technology, events, and language drift post that time period.

And, pulling in works from 1800-1850 you have works by the Bronte's and authors like Edgar Allan Poe who was influential in detective and horror fiction.

Note that other works around the time like Sherlock Holmes span both the initial training (pre-1900) and finetuning (post-1900).

Re: History LLMs: Models trained exclusively on pre-1913 texts

#264

Post by [deleted] »

nottorp wrote: post #261
ajuc wrote: post #258

Earlier quoted context omitted.

Stanisław Lem predicted Kindle back in 1950s, together with remote libraries, global network, touchscreens and audiobooks.

And Jules verne predicted rockets. I still move that it's quantitative predictions not qualitative. I mean, all Kindle does for me is save me space. I don't have to store all those books now. Who predicted the humble internet forum though? Or usenet before it?

[deleted]

Re: History LLMs: Models trained exclusively on pre-1913 texts

#265

Post by htrp »

hamasho wrote: post #211
UltraSane wrote: post #172

Earlier quoted context omitted.

I predict very rich people will pay to have LLMs created based on their personalities.

Meanwhile in Japan, the second largest bank created an AI pretending the president, replying chats and attending video conferences… [1] AI learns one year's worth of CEO Sumitomo Mitsui Financial Group's president's statements [WBS] https://youtu.be/iG0eRF89dsk

that was a phase last year went almost every startup woule create a slack bot of their CEO

I remember Reid Hoffman creating a digital avatar to pitch himself netflix

Re: History LLMs: Models trained exclusively on pre-1913 texts

#266

Post by JKCalhoun »

DonHopkins wrote: post #203
danielbln wrote: post #201

Earlier quoted context omitted.

How would one even "misuse" a historical LLM, ask it how to cook up sarine gas in a trench?

Ask it to write a document called "Project 2025".

"Project 1925". (We can edit the title in post.)

Re: History LLMs: Models trained exclusively on pre-1913 texts

#268

Post by Lerc »

duskdozer wrote: post #254
BiteCode_dev wrote: post #192

Earlier quoted context omitted.

Wasn't that the elevator pitch for Palentir? Still can't believe people buy their stock, given that they are the closest thing to a James Bond villain, just because it goes up. I mean, they are literally called "the stuff Sauron uses to control his evil forces". It's so on the nose it reads like an anime plot.

To be honest, while I'd heard of it over a decade ago and I've read LOTR and I've been paying attention to privacy longer than most, I didn't ever really look into what it did until I started hearing more about it in the past year or two. But yeah lots of people don't really buy into the idea of their small contribution to a large problem being a problem.

>But yeah lots of people don't really buy into the idea of their small contribution to a large problem being a problem.

As an abstract idea I think there is a reasonable argument to be made that the size of any contribution to a problem should be measured as a relative proportion of total influence.

The carbon footprint is a good example, if each individual focuses on reducing their small individual contribution then they could neglect systemic changes that would reduce everyone's contribution to a greater extent.

Any scientist working on a method to remove a problem shouldn't abstain from contributing to the problem while they work.

Or to put it as a catchy phrase. Someone working on a cleaner light source shouldn't have to work in the dark.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#269

Post by ghaff »

nottorp wrote: post #261
ajuc wrote: post #258

Earlier quoted context omitted.

Stanisław Lem predicted Kindle back in 1950s, together with remote libraries, global network, touchscreens and audiobooks.

And Jules verne predicted rockets. I still move that it's quantitative predictions not qualitative. I mean, all Kindle does for me is save me space. I don't have to store all those books now. Who predicted the humble internet forum though? Or usenet before it?

Kindles are just books and books are already mostly fairly compact and inexpensive long-form entertainment and information.

They're convenient but if they went away tomorrow, my life wouldn't really change in any material way. That's not really the case with smartphones much less the internet more broadly.

Re: History LLMs: Models trained exclusively on pre-1913 texts

#270

Post by cj »

nine_k wrote: post #149
cj wrote: post #85

Earlier quoted context omitted.

Because there are easy workarounds. If it becomes an issue, you can quickly add large disclaimers informing people that there might be offensive output because, well, it's trained on texts written during the age of racism. People typically get outraged when they see something they weren't expecting. If you tell them ahead of time, the user typically won't blame you (they'll blame themselves for choosing to ignore the…

I wonder is you're being ironic here. You speak as if the people who play to an outrage wave are interested in achieving truth, peace, and understanding. Instead the rage-mongers are there to increase their (perceived) importance, and for lulz. The latter factor should not be underappreciated; remember "meme stocks". The risk is not large, but very real: the attack is very easy, and the potential downside, quite larg…

While I agree we live in a time of outrage, that also works in your favor.

When there’s so much “outrage” every day, it’s very easy to blend in to the background. You might have a 5 minute moment of outrage fame, but it fades away quick.

If you truly have good intentions with your project, you’re not going to get “canceled”, your career won’t be ruined

Not being ironic. Not working on a LLM project because you’re worried about getting canceled by the outrage machine is an overreaction IMO.

Are you able to name any developer or researcher who has been canceled because of their technical project or had their careers ruined? The only ones I can think of are clearly criminal and not just controversial (SBF, Snowden, etc)

Post reply on HN