Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talkie: a 13B vintage language model from 1930

talkie-lm.com

Post reply on HN
Print view View original

161–170 of 350 posts

Re: Talkie: a 13B vintage language model from 1930

#161

Post by scrollaway »

embedding-shape wrote: post #152
felixyz wrote: post #151

Earlier quoted context omitted.

Your information diet. Social media. Gossipy and negative people. Mulling over old failures/regrets/slights etc. The mind is easily pulled along by negativity and outrage... as can be observed in our current global psychological state.

All those are fine, as long as you're able to process it in a healthy way after. I guess personally I focused more on bettering that processing, as sometimes you don't get to control what information you get served, so at least it works in all cases.

Don’t be so optimistic about your ability to “process information healthily”. You are more of a slave to your instincts than you think and can’t always know whether you’re actually doing a good job at this— literally, it’s not possible to faithfully introspectively this.

Re: Talkie: a 13B vintage language model from 1930

#162

Post by embedding-shape »

scrollaway wrote: post #161
embedding-shape wrote: post #152

Earlier quoted context omitted.

All those are fine, as long as you're able to process it in a healthy way after. I guess personally I focused more on bettering that processing, as sometimes you don't get to control what information you get served, so at least it works in all cases.

Don’t be so optimistic about your ability to “process information healthily”. You are more of a slave to your instincts than you think and can’t always know whether you’re actually doing a good job at this— literally, it’s not possible to faithfully introspectively this.

> Don’t be so optimistic about your ability to “process information healthily”.

Don't be so pessimistic about your own ability to control how you process information, you can control this a lot more than you think, apparently.

Re: Talkie: a 13B vintage language model from 1930

#163

Post by b65e8bee43c2ed0 »

jryio wrote: post #46

If anyone was wondering ... it's racist Unsurprisingly the texts written up until that time were dominated by such individuals which is tragic for LLM training if you think about it. The voiceless groups or fringe opinions which we take as normative today do not appear. Does this encourage us to write in the present such that we influence the models in perpetuity?

one day we'll have SOTA models trained like this one and there's nothing you can do about it :^)

Re: Talkie: a 13B vintage language model from 1930

#164

Post by jmbwell »

xtiansimon wrote: post #156
embedding-shape wrote: post #139

Earlier quoted context omitted.

> You will pollute your brain. Such an interesting perspective, never crossed my mind that a brain could be polluted! My direction always been to fill it with as wide array of information as possible, the more different from existing information the better. What are some other things that you think "pollutes your brain"?

The classic thing that pollutes your brain are punk (music and Mad Magazine) and smut. I’d add “dangerous memes” such as injecting bleach to cure covid. https://www.susanblackmore.uk/wp-content/uploads/2017/05/201...

These days, I’ll take Mad magazine

Re: Talkie: a 13B vintage language model from 1930

#165

Post by engeljohnb »

embedding-shape wrote: post #160
engeljohnb wrote: post #158

Earlier quoted context omitted.

Not who you asked, but Neil Postman's "Amusing Ourselves to Death" is an excellent book about polluting your brain. As for my personal experience, internet comment sections will pollute one's brain. Filling your brain with reasonably reliable information is good, but filling it with people online just saying things isn't. For example, when 30 reddit comments all repeat the same "fact" (for which their source is other…

> it can subtly work its way into your subconscious as something you know is true I dunno, I know this is something some people struggle with, but I'm not sure how I could personally end up here. You can repeat something how many times you want, it doesn't make it true, and if anything, seeing people repeat the same "fact" like that would probably trigger the reverse in my brain, almost automatically going out of my…

Sounds like you always knew something it took me a decade to realize.

> seeing people repeat the same "fact" like that would probably trigger the reverse in my brain, almost automatically going out of my way to disprove it while reading it.

I think that's a very fundamental difference between you and me. I'm too lazy to fact check most of what I read.

One day I decided I would never run my mouth about something unless I felt I could write a five paragraph essay about it, and now I don't run my mouth very much because apparently there aren't a lot of things I'm willing to research even that much.

Still, I highly recommend Amusing Ourselves to Death. It has more and better insights into stuff like this, and I seriously believe everyone should read it.

Re: Talkie: a 13B vintage language model from 1930

#167

Post by b65e8bee43c2ed0 »

fwipsy wrote: post #45
walrus01 wrote: post #4

I think that one could also take a much larger model (35B or 122B sized) and give it a thorough system prompt to only speak in the manner of a well educated Victorian/Edwardian era gentleman, if you want an "old timey" LLM.

It's hard to know how accurate that is. Is the LLM truly imitating text from that era, or is it imitating a modern idea of text from that era? Also, safety/alignment training would probably prevent it from embracing many of the ideas from that era, even in roleplay.

>Also, safety/alignment training would probably prevent it from embracing many of the ideas from that era, even in roleplay.

lobotomy is an *optional* step. had this technology emerged before the 9/11 and Twitter, SOTA models wouldn't bat an eye if you asked one to write a recipe for meth in ebonics.

Re: Talkie: a 13B vintage language model from 1930

#168

Post by b65e8bee43c2ed0 »

Sol- wrote: post #126

Isn't it surprising that there were enough pre-1930 tokens to train an intelligent model? I was always under the impression that many tokens are also necessary to force the model to grok things and compress its learning into a somewhat intelligent model of the world, so to say. But perhaps I'm underestimating how much digitized literature exists from then.

one of my greatest hopes for the advancement of LLM technology is a great reduction for the amount of data to train on. imagine a SOTA model trained exclusively on good prose, ah.

Re: Talkie: a 13B vintage language model from 1930

#170

Post by Wowfunhappy »

stbullard wrote: post #103

USER: What will the future hold for computers? TALKIE-1930: Computers in the future may be employed in offices where calculations are required to be made, and where the nature of the business does not demand a very high degree of knowledge. The occupation will be a steady one, and the remuneration will be sufficient to afford a comfortable livelihood. The hours of attendance will probably be from ten to six. USER: an…

I would be curious to see how this does in Anthropic’s alignment tests (like that one where the AI tried to blackmail an employee). I’ve always thought that in these situations, the AI is acting out the role of all the AIs in the stories we’ve written. But Talkie, trained on data from before digital computers, wouldn’t know those stories.
Post reply on HN