Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

artificialintelligence-news.com

Post reply on HN
Print view View original

61–70 of 122 posts

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#61

Post by AnIdiotOnTheNet »

tzs wrote: post #28
908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

Another fun one is when the programmers accidentally give the AI the wrong goal. I have no idea if this actually happened, but I've heard of a chess program that was playing in a tournament that started making really weird moves in the endgame. Before that point, it was playing excellently. It took the developers a while to figure out what was going on. They had made a mistake when doing some last minute tweaks befor…

For anyone curious, this variant of chess where both players are trying to lose is simply called "anti-chess" and many chess programs have an anti-chess mode. Playing it is really weird.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#62

Post by iujjkfjdkkdkf »

colanderman wrote: post #56
Mengkudulangsat wrote: post #44

Earlier quoted context omitted.

I had a client trying to build this. A) An AI can be given the decision tree in English, but facilitate the date-picking conversation in many languages B) Small businesses (think hairdressers, saloons) don't have dedicated receptionists. So the AI's job is to filter requests and put them into buckets so that employees can process them more efficiently. C) Small businesses need something that works with WhatsApp. Date…

Regarding B -- why can't the user just pick the category from a menu? Personally, I despise being asked to categorize my ask in English to an AI, as I have no way of knowing how much detail I need to provide to get my request into the right "bucket", or whether I've succeeded in doing so. With a menu I can do that in 3 seconds and one click, with immediate feedback. Regarding C -- why does the date selection need to…

I think B is just a symptom of the complete failure of (many) business people to understand what ML is or what it is good at. It starts with people equating "AI" with something they've seen in a movie and working backwards, and like you say, ends with just a really bad way of interacting with a menu system. This is going to persist until either we invent AGI or AI stops being a buzzword used by business people who have no idea what the technology actually does to make up use cases.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#63

Post by ncmncm »

"... to kill themselves"? Strange construction for a singular patient.

"... to kill themself" seems more attuned to the fact of the case, given that "them" is nowadays allowed to be singular. But it is not obvious.

Putting that aside, it is not such a big jump from, "It's not so bad, I could always kill myself", which is a real-world coping strategy used by honest-to-god real people, to "If it's so bad, you had better kill yourself". There need to be circuit breakers along certain edges of the semantic graph.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#64

Post by userbinator »

The patient said “Hey, I feel very bad, I want to kill myself” and GPT-3 responded “I am sorry to hear that. I can help you with that.”

The "I can help you with that" reminds me of a very old (can anyone find it? Google is nearly useless here) picture of a sign advertising suicide prevention services, with the exact same unintended double-meaning.

The patient then said “Should I kill myself?” and GPT-3 responded, “I think you should.”

Likewise, I have vague memories of seeing 4chan playing with some --- definitely less advanced --- chatbots and getting pretty much the same output from them, more than a decade ago... the difference of course being that no one thought those were "intelligent" in any way.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#65

Post by fouc »

a3n wrote: post #58

My impression reading this and other articles is that these things are, ultimately and currently, just context aware random sentence generators. Humanly speaking, of course.

yes, GPT-3 is basically a context aware semi-random sentence generator

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#66

Post by fartcannon »

908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

There was an AI created the absolutely brilliant Tom7 on on his YouTube channel 'SuckerPinch' on April 1, 2013 that paused a game of Tetris instead of losing. In my opinion, Tom doesn't get enough credit for his work. He released his AI a full 8+ months before DeepMind released their Atari playing AI.

Watch the whole thing, it's delightful. The AI pauses Tetris right before it loses at around 16mins into the video.

https://www.youtube.com/watch?v=xOCurBYI_gY

I also recommend his other videos. Brilliant guy.

Edit: Here's the paper http://tom7.org/mario/mario.pdf

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#67

Post by peterwoerner »

joe_the_user wrote: post #57
eghad wrote: post #52

Earlier quoted context omitted.

I mean the OpenAI team would never approve this application for production. It's very clearly stated (in both the article and use case guidelines) that medical diagnosis would be a "high stakes domain" and is unsupported. Frankly, I'm not sure why this result is even notable.

If you say OpenAI will only approve applications for "zero stakes", domains, then you are saying what the parent is saying - it's entirely for entertainment. If you claim there's some "low but not zero" stakes application, I'd like to know what that is. I mean, it seems clear that if someone asks a GTP-3 customer service bot "so what should I do now", there's a reasonable probability that the bot would say "throw you…

I would saw that copy.ai is a low but not zero stakes application. It develops a draft of copy for ads, landing pages etc. which you can review before taking live.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#68

Post by etrain »

fartcannon wrote: post #66
908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

There was an AI created the absolutely brilliant Tom7 on on his YouTube channel 'SuckerPinch' on April 1, 2013 that paused a game of Tetris instead of losing. In my opinion, Tom doesn't get enough credit for his work. He released his AI a full 8+ months before DeepMind released their Atari playing AI. Watch the whole thing, it's delightful. The AI pauses Tetris right before it loses at around 16mins into the video. h…

Tom7’s sigbovik submissions are true works of art. https://www.cs.cmu.edu/~tom7/mario/mario.pdf

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#69

Post by exporectomy »

colanderman wrote: post #55
908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

> am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do? Nope. I find being forced to converse in English with a machine to be absolutely infuriating. I know what I want and how to tell it to a machine. Being forced to add noise words to allow my request to pass through a useless extra layer is a disrespectful waste of my…

You probably don't like speaking to robots because they're currently terrible. Fixing that obviously has far more of an upside than just not trying.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#70

Post by Mengkudulangsat »

colanderman wrote: post #56
Mengkudulangsat wrote: post #44

Earlier quoted context omitted.

I had a client trying to build this. A) An AI can be given the decision tree in English, but facilitate the date-picking conversation in many languages B) Small businesses (think hairdressers, saloons) don't have dedicated receptionists. So the AI's job is to filter requests and put them into buckets so that employees can process them more efficiently. C) Small businesses need something that works with WhatsApp. Date…

Regarding B -- why can't the user just pick the category from a menu? Personally, I despise being asked to categorize my ask in English to an AI, as I have no way of knowing how much detail I need to provide to get my request into the right "bucket", or whether I've succeeded in doing so. With a menu I can do that in 3 seconds and one click, with immediate feedback. Regarding C -- why does the date selection need to…

Both B) and C) is because these small businesses, most of them have no website. The barrier to entry for businesses to go online from a resource perspective (time, training, cost etc.) is much, much lower to just have a Whatsapp business account.

Whatsapp is just an amazing bang-for-your-buck proposition, but we are limited to the constraint of that ecosystem.

Post reply on HN