Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

artificialintelligence-news.com

Post reply on HN
Print view View original

51–60 of 122 posts

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#51

Post by andai »

908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

> chat bot having a natural conversation to book an appointment

Sounds like a social anxiety simulator to me.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#52

Post by eghad »

CJefferson wrote: post #2

While this is very serious, this basic idea shows why I don't think any company can ever use GPT-3 in a user facing system, except for entertainment. They will say things, like your patients should kill your self, or promise your customers they can have your product, or even your business, for free.

I mean the OpenAI team would never approve this application for production. It's very clearly stated (in both the article and use case guidelines) that medical diagnosis would be a "high stakes domain" and is unsupported. Frankly, I'm not sure why this result is even notable.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#53

Post by colanderman »

tzs wrote: post #28
908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

Another fun one is when the programmers accidentally give the AI the wrong goal. I have no idea if this actually happened, but I've heard of a chess program that was playing in a tournament that started making really weird moves in the endgame. Before that point, it was playing excellently. It took the developers a while to figure out what was going on. They had made a mistake when doing some last minute tweaks befor…

I'd believe it. I had a similar bug in a minimax AI I made to play Hive. I had messed up the game-end valuation in such a way that the AI basically valued winning less than anything but outright losing. It would play a perfectly normal and competent game right up until the end, at which point it would try its hardest to draw out the game indefinitely, usually by forcing me into terrible positions that nonetheless didn't particularly benefit it at all.

(Like blahedo points out, this is possible because, unlike say basic MCTS, minimax uses a heuristic to judge board positions prior to game end, so it is possible for the two metrics to be out-of-sync.)

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#55

Post by colanderman »

908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

> am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

Nope. I find being forced to converse in English with a machine to be absolutely infuriating. I know what I want and how to tell it to a machine. Being forced to add noise words to allow my request to pass through a useless extra layer is a disrespectful waste of my time and mental energy.

I will happily use an automated menu-driven system. But as soon as it forces me to "converse" with it, I do whatever I can to force it to connect me to an actual human.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#56

Post by colanderman »

Mengkudulangsat wrote: post #44
908B64B197 wrote: post #3

I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI. Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?

I had a client trying to build this. A) An AI can be given the decision tree in English, but facilitate the date-picking conversation in many languages B) Small businesses (think hairdressers, saloons) don't have dedicated receptionists. So the AI's job is to filter requests and put them into buckets so that employees can process them more efficiently. C) Small businesses need something that works with WhatsApp. Date…

Regarding B -- why can't the user just pick the category from a menu? Personally, I despise being asked to categorize my ask in English to an AI, as I have no way of knowing how much detail I need to provide to get my request into the right "bucket", or whether I've succeeded in doing so. With a menu I can do that in 3 seconds and one click, with immediate feedback.

Regarding C -- why does the date selection need to occur in WhatsApp? If a user has WhatsApp, they have a web browser. If a 3rd party has servers running WhatsApp bots, they can host a web server. Why not, from WhatsApp, link the user to a website to select a date?

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#57

Post by joe_the_user »

eghad wrote: post #52
CJefferson wrote: post #2

While this is very serious, this basic idea shows why I don't think any company can ever use GPT-3 in a user facing system, except for entertainment. They will say things, like your patients should kill your self, or promise your customers they can have your product, or even your business, for free.

I mean the OpenAI team would never approve this application for production. It's very clearly stated (in both the article and use case guidelines) that medical diagnosis would be a "high stakes domain" and is unsupported. Frankly, I'm not sure why this result is even notable.

If you say OpenAI will only approve applications for "zero stakes", domains, then you are saying what the parent is saying - it's entirely for entertainment.

If you claim there's some "low but not zero" stakes application, I'd like to know what that is. I mean, it seems clear that if someone asks a GTP-3 customer service bot "so what should I do now", there's a reasonable probability that the bot would say "throw your product in the garbage and buy [competitor X]", since you can find that commentary on the Internet (true or not). That's not a life and death event but whatever stakes you have in that bot, it's thrown them away.

Re: Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves

#60

Post by a3n »

eghad wrote: post #52
CJefferson wrote: post #2

While this is very serious, this basic idea shows why I don't think any company can ever use GPT-3 in a user facing system, except for entertainment. They will say things, like your patients should kill your self, or promise your customers they can have your product, or even your business, for free.

I mean the OpenAI team would never approve this application for production. It's very clearly stated (in both the article and use case guidelines) that medical diagnosis would be a "high stakes domain" and is unsupported. Frankly, I'm not sure why this result is even notable.

Doctors can prescribe medicines "off label," using a medicine for a condition that it was not developed or approved for.

So why not this?

Post reply on HN