Live data from Hacker News

A failed AI girlfriend product, and my lessons

mazzzystar.github.io

61–70 of 400 posts

Re: A failed AI girlfriend product, and my lessons

#61

Well this went places. The dev really 'ate his own dogfood' so to speak. I thought it was going to be a despairing look at how his companion AI was continuously turned to sleazy uses by a pervy userbase but that's pretty much what he was going for it seems. I have to say, I hated this part: "...gained a lot of Chinese users. I immediately realized the political risk: users could set Dolores’ background, personality,…

Pretty much came into the comments to agree with the self-censorship part there.

But then the author did use 'Her' as the ideal example. That's a pretty good indicator they didn't get the point.

Re: A failed AI girlfriend product, and my lessons

#62

Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Depressing stuff but this is the world we live in. Thank you to the author for this interesting read.

To imagine these apps aren’t already doing such is a hard sell imo.

Full disclosure: I’ve paid for several AI services so far, most in the realm of “virtual girlfriend” experiences, and they all generally are more than subpar. Mind you, I have a couple of IRL partners. I also enjoy the act of sexual roleplaying, and finding reasonably expressive human partners is time-consuming. The “Girlfriend AI” options are pretty terrible though - I’ll take character.ai with its heavy moderation any day.

But if I’m paying to try these and explore and see what works, I’m certain that wallets are being drained already. Is this a problem? Could be, at least for some - to think otherwise feels disingenuous. The question will come down to how we address it: do we have frank conversations with people about the risks of devoting potential social time to AI engagement, or do people clamor for the AI backing tech to button up and become even more prudish?

Re: A failed AI girlfriend product, and my lessons

#63

Earlier quoted context omitted.

Could say the same about antidepressants. "Fix the causes, not the symptoms!" And yet, here we are.

antidepressants can help people fix root causes. take them for 6 months to 2 years or so, break out of a bad cycle, get some space without depression symptoms to build up healthy habits and restore a social life. there are some bad profit motives to keep people on SSRIs indefinitely. however at least they don't compete with other aspects of your life. and now that they are dirt cheap (at cheapest, $4 for a month supp…

> antidepressants can help people fix root causes

Nit-picking rather than disagreeing, but feel it worth making more obvious in the wording: antidepressants can help people deal with the symptoms while they find/fix root causes. Being on them for an extended time can bring its own problems.

Re: A failed AI girlfriend product, and my lessons

#64

Interesting read. Others have also created virtual friends using ChatGPT. What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand.

I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

Re: A failed AI girlfriend product, and my lessons

#66
post #52

Earlier quoted context omitted.

It's going to be an interesting century for sure. The human mind is simply not evolutionary adapted for what's coming up.

> The human mind is simply not evolutionary adapted for what's coming up. This can be said of pretty much everything humans ended up creating with technology so... not sure there's anything really new down the road. Humans adapt.

Humans adapt within boundaries.

A couple of examples of things humans did not adapt to:

- emphathy at distance; nowadays, a drone operator will push a button, kill a dozen people, and feel like it was a videogame

- prioritizing long-term and abstract rewards, over short term ones; this is the reason why we have phenomenons like global warming

- adjusting hunger to virtually unlimited availability of food

All the above are concrete and very serious problems for humanity.

Re: A failed AI girlfriend product, and my lessons

#67

Earlier quoted context omitted.

Could say the same about antidepressants. "Fix the causes, not the symptoms!" And yet, here we are.

> “Fix the causes” How, exactly? Yes, some people experience depression as a symptom of something than can be addressed through various therapies, but what about people who just have it wired into their brains? How do we “fix” that?

Even if it is wired into certain brains, that wiring may only cause depression when it interacts with our modern technological society.

Re: A failed AI girlfriend product, and my lessons

#68
post #52

Earlier quoted context omitted.

> The human mind is simply not evolutionary adapted for what's coming up. This can be said of pretty much everything humans ended up creating with technology so... not sure there's anything really new down the road. Humans adapt.

Humans adapt within boundaries. A couple of examples of things humans did not adapt to: - emphathy at distance; nowadays, a drone operator will push a button, kill a dozen people, and feel like it was a videogame - prioritizing long-term and abstract rewards, over short term ones; this is the reason why we have phenomenons like global warming - adjusting hunger to virtually unlimited availability of food All the abov…

The fact that you point this out, is a proof that humanity can improve, what needs to change is our culture, especially education, ofc it may take a lot of time but it will happen eventually.

Re: A failed AI girlfriend product, and my lessons

#69
Reading this, I really wonder why the dev hasn't forseen this outcome. It was already obvious in March that OpenAI will censor all sorts of content. Building a sexually charged bussiness on the OpenAI API was always a dead end for those that have a feel for american prudish approach to sexuality. If you had asked me to do this sort of product 6 motnsh ago, I would have declined because it was (to me) rather obvious content filtering will kill you at some point.

Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his girlfriend described. Guess what, no way. Consent was given, actually, she asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend looks naked... This is weird beyond description. Plain patronisation.

Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? "Censoring just declared my own boobs to be off-limits for me." If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described.

And then there is this blog post by Sam Hartman: https://hartmans.dreamwidth.org/100499.html

Re: A failed AI girlfriend product, and my lessons

#70
post #53

> GPT-based products, if not priced per usage, would fall into a dilemma: 1% of users consume 99% of tokens. A user from Sweden (seen from Cloudflare’s call volume) chatted with Dolores for 12 hours straight This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by…

The author doesn't appear to even consider any kind of obligation to their users.

I'm sorry you feel that way. The primary intention of writing this article was to discuss it from the perspective of a 'failed product,' so I didn't mention my personal feelings. I am also aware of the potential harm it could cause to users, which is why I don't resent OpenAI's review process. I hope that during the period without review, it didn't cause any psychological harm to users.
Post reply on HN