Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A simulation of me: fine-tuning an LLM on 240k text messages

edwarddonner.com

Post reply on HN
Print view View original

31–40 of 145 posts

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#31

Post by Minor49er »

teitoklien wrote: post #27
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

> Pay to have a ghost of yourself duplicated in a cemetery. I mean, they already do it at funerals [1]. I don’t see why not, would make cemetery a fun hangout place to talk to people from history. [1]( https://m.youtube.com/watch?v=s2FJRbRsBBY )

Apparently this works by asking questions to someone while they are still alive. Then after the individual has passed, friends and family can ask questions to a video screen which will pick out the response that it feels is the most appropriate and play it. Though I'm sure we're not far off from a service that will do this through deepfake technology

Makes me think of the 2004 movie The Final Cut where people get memory implants that record their entire lives for the purpose of creating a highlight reel of their lives after they're gone

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#32

Post by speps »

orenlindsey wrote: post #5

I wanted to do this myself but I don't have enough messages yet. Also, I wish the author shared some more technical information on how to do it.

This repo, albeit "old" in regards to how much progress there's been in LLMs, has great simple tutorials right there eg. fine-tuning GPT2 with Shakespeare: https://github.com/karpathy/nanoGPT

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#33

Post by krainboltgreene »

alchemist1e9 wrote: post #13
mewpmewp2 wrote: post #8

Earlier quoted context omitted.

> it might be therapeutic to have a conversation with a deceased loved one. Would it? I'm not a specialist, but it would seem a bit off to me to approach it like that. I'd think best is to accept that your deceased one has gone, and it seems like having a fake one might just prolong the grievance process and possibly also develop into an obsession of trying to replace your deceased one with the simulation, upgrading…

Have you ever watched the original Superman movies and Clark talks with the AI hologram of his biological father from his home planet? I believe in the future our descendants will likely find it normal to talk with AI versions of their ancestors, perhaps not us parents right now, but I suspect my children will willingly train and generate AI versions of themselves for future generations to interact with.

> Have you ever watched the original Superman movies and Clark talks with the AI hologram of his biological father from his home planet?

You get that this only played out positively in the movie because the writers/director wanted it to play out that way, right?

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#35

Post by krainboltgreene »

Sohcahtoa82 wrote: post #11

I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…

It seems really unethical to train a model on someone else's words without their permission.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#36

Post by robotbikes »

a_e_k wrote: post #15
wyldfire wrote: post #2

> I immediately recognized this would be in extremely poor taste and removed the data. Enough on that topic! Poor taste for a blog post. But I wonder if LLMs could advance to a point where it might be therapeutic to have a conversation with a deceased loved one.

Something like that is the core idea behind Black Mirror S2E1, "Be Right Back" (2013). https://en.wikipedia.org/wiki/Be_Right_Back

Also something like this happens in the Greg Egan book Zendegi. It starts off with a anecdote about someone ripping their music files and the issues that arise from the copy (or something along those lines). But definitely an interesting read and one that tackles the possible issues that could arise in a different way than other novels.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#37

Post by fnorder »

HPMOR wrote: post #26
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

This actually seems like a really good startup idea.

I'm petty sure that Ray Kurzweil was working on this very thing when Google brought him on.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#38

Post by anjel »

teitoklien wrote: post #27
vunderba wrote: post #23

I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…

> Pay to have a ghost of yourself duplicated in a cemetery. I mean, they already do it at funerals [1]. I don’t see why not, would make cemetery a fun hangout place to talk to people from history. [1]( https://m.youtube.com/watch?v=s2FJRbRsBBY )

See also: Tupak; Coachella

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#39

Post by oasisaimlessly »

krainboltgreene wrote: post #35
Sohcahtoa82 wrote: post #11

I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…

It seems really unethical to train a model on someone else's words without their permission.

Not to me.

Re: A simulation of me: fine-tuning an LLM on 240k text messages

#40

Post by mojomark »

fnorder wrote: post #37
HPMOR wrote: post #26

Earlier quoted context omitted.

This actually seems like a really good startup idea.

I'm petty sure that Ray Kurzweil was working on this very thing when Google brought him on.

The documentary with him saving his father's writings and such was one of the saddest things I've seen - that moment has stuck with me for years.

I suppose the people who will be paying for such a service will be the loved ones left behind, not the one's dying. I wonder if the doppelgangers will start to talk to each other in the graveyard in the middle of the night?

Post reply on HN