51–60 of 145 posts
> I immediately recognized this would be in extremely poor taste and removed the data. Enough on that topic! Poor taste for a blog post. But I wonder if LLMs could advance to a point where it might be therapeutic to have a conversation with a deceased loved one.
Earlier quoted context omitted.
I'm petty sure that Ray Kurzweil was working on this very thing when Google brought him on.
The documentary with him saving his father's writings and such was one of the saddest things I've seen - that moment has stuck with me for years. I suppose the people who will be paying for such a service will be the loved ones left behind, not the one's dying. I wonder if the doppelgangers will start to talk to each other in the graveyard in the middle of the night?
I wanted to do this myself but I don't have enough messages yet. Also, I wish the author shared some more technical information on how to do it.
I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…
I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…
It seems really unethical to train a model on someone else's words without their permission.
I’m far from the first to think of this. Several people — perhaps inspired by creepy Black Mirror episodes — have tried to fine-tune an LLM on their SMS or WhatsApp history in an effort to create a simulation of themselves. It's a much older concept than Black Mirror. Ever since Markov chain IRC bots got popularized in the late 90s and early 2000s, people have been trying to train their virtual doppelgängers. I'm sur…
I am not sure if Gibson found inspiration elsewear but this is pure flatline dixie.
Fundamentally I suppose it's naught but whispers from the beyond.
I tried this myself using GPT-2J and nearly 15 years of IRC logs trimmed down to only contexts that include me. Each training item would be my message and the preceding 20 messages. Still ended up being an 80 MB log file of over 1M lines. The results were...not good at all. I ran an IRC bot that would watch the channel, and any time someone sent a message, it would send the last 20 messages to the trained model to se…
It seems really unethical to train a model on someone else's words without their permission.
> I immediately recognized this would be in extremely poor taste and removed the data. Enough on that topic! Poor taste for a blog post. But I wonder if LLMs could advance to a point where it might be therapeutic to have a conversation with a deceased loved one.
> it might be therapeutic to have a conversation with a deceased loved one. Would it? I'm not a specialist, but it would seem a bit off to me to approach it like that. I'd think best is to accept that your deceased one has gone, and it seems like having a fake one might just prolong the grievance process and possibly also develop into an obsession of trying to replace your deceased one with the simulation, upgrading…
A managed service for all this surely has a market, imho
In my journal and daily notebooks I write out some pretty deep and dark thoughts too. It's about as close to a snapshot of my inner thoughts as you can imagine.
And the LLM digested all of that. It took quite a while.
It has provided some interesting insight into who I am as a person, and also how I see myself.
I’m glad someone put the time it to show it can finally work. Would love to do this too, but have a gazillion other projects that are priority.
Looking at my messages I think I don't need to train the model to generate that text, but rather identify the personality traits and emotional stance of the author, and be aware of personal facts. So probably something like RLHF would work better than SFT.
Assuming I have such a model I can further collect preference data to align it to my personality. Over time it might get good enough and I can officially bless it, if it ever gets to that level.
Getting our social account providers to export all our data without restrictions might become a new right we have to have. I know some of them are locking up since 2023 (reddit and twitter).