Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A simulation of me: fine-tuning an LLM on 240k text messages

edwarddonner.com

Post reply on HN
Print view View original

41–50 of 145 posts

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#41

Post by infoseek12 »

krainboltgreene wrote: post #35
Sohcahtoa82 wrote: post #11

I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…

It seems really unethical to train a model on someone else's words without their permission.

It may be just about as unethical as someone discussing something they overheard in a crowd.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#43

Post by Alpha3031 »

HPMOR wrote: post #26
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

This actually seems like a really good startup idea.

cripes does anybody remember Google People

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#44

Post by skybrian »

vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

If you want to include fictional antecedents, there is Dixie Flatline from Neuromancer.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#45

Post by spiritplumber »

HPMOR wrote: post #26
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

This actually seems like a really good startup idea.

It's literally an episode of Max Headroom. https://www.maxheadroom.com/index.php?title=Episode_ABC.2.2:...

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#46

Post by [deleted] »

mewpmewp2 wrote: post #8
wyldfire wrote: post #2

> I immediately recognized this would be in extremely poor taste and removed the data. Enough on that topic! Poor taste for a blog post. But I wonder if LLMs could advance to a point where it might be therapeutic to have a conversation with a deceased loved one.

> it might be therapeutic to have a conversation with a deceased loved one. Would it? I'm not a specialist, but it would seem a bit off to me to approach it like that. I'd think best is to accept that your deceased one has gone, and it seems like having a fake one might just prolong the grievance process and possibly also develop into an obsession of trying to replace your deceased one with the simulation, upgrading…

[deleted]

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#47

Post by fractallyte »

My comment on something similar, 5 years ago (https://news.ycombinator.com/item?id=18546242):

Better be careful with that...

From "Bicycle Repairman", by Bruce Sterling (1996) (This is a spoiler, BTW!):

"The mook speaks just like the Senator did, or the way the Senator used to speak, when he was in private and off the record. The way he spoke in his diaries. As far as we can tell, the mook was his diary.... It used to be his personal laptop computer. But he just kept transferring the files, and upgrading the software, and teaching it new tricks like voice recognition and speech-writing, and giving it power of attorney and such.... And then, one day the mook made a break for it. We think that the mook sincerely believes that it’s the Senator."

Oh, and just note that this was way before Black Mirror.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#48

Post by noduerme »

vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

On the casino I used to run, I started a pilot program with a homemade poker bot (labeled as such, and only deployed on poker tables labeled as "bot friendly"). The bot had no set model of its own. It was designed to mimic specific players on the casino, regulars who had played 10,000+ hands and who agreed to have their history cloned, by ingesting their entire hand/betting history and looking for what they had done in similar scenarios as far as hole card value, probability, position, pot odds, table "heat" (a rolling average of over- or under-betting from other players in the previous hands based on what they showed if they showed down), etc.

I got a few good players to agree to let me take all their data and make poker clones of them. What was really funny was watching them get baffled and often beaten in heads-up play against their doppelgangers. The bot never had more information about the game than any other player. But its betting style varied wildly depending on who it was based on.

As a side, this also helped me build a real-time system to monitor potential collusion based on betting patterns that deviated from a player's normal behavior at times when they were sitting at the same table as someone else whose patterns deviated in their presence.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#49

Post by layman51 »

skybrian wrote: post #44
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

If you want to include fictional antecedents, there is Dixie Flatline from Neuromancer.

There’s also Ubik by Philip K. Dick where the recently deceased are able to communicate with the living AND aside from that also advertise a product.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#50

Post by altdataseller »

krainboltgreene wrote: post #35
Sohcahtoa82 wrote: post #11

I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…

It seems really unethical to train a model on someone else's words without their permission.

Isn't that basically what OpenAI did?
Post reply on HN