Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

ai.googleblog.com

Post reply on HN
Print view View original

101–110 of 786 posts

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#101

Post by mFixman »

542458 wrote: post #13

This is going to be awesome/terrible for social engineering attacks. We'll be training millions of people to to trust phone calls from "Google". But are you talking to Google-calling-on-behalf-of-your-boss, or Google-calling-on-behalf-of-some-phisher? Or better yet, some custom system that pretends to be Google over the phone? Who knows!

I think the biggest issue won't come from phishers, but from resellers and other occupations that are helped by automation. I could easily call every hot restaurant in San Francisco a several a day to try to snatch a table.

On the other hand, most of these places have online reservations which are already extremely gamed.

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#102

Post by barake »

jbob2000 wrote: post #61
apendleton wrote: post #28

Earlier quoted context omitted.

At least in the demo, the system didn't identify itself as a bot. It pretended (fairly convincingly) to just be a person, and seemed focused on interactions (making a restaurant reservation, etc.) that don't involve someone recognizing your voice. In other words, I don't think the attack surface is any different than situations where you could just place the call yourself, save perhaps for the possibility of scaling…

So now I can script a bot to book restaurant reservations all over the city at busy times. Then nobody shows up for the reservations, the busy time has passed, and customers have moved on or gone home. Restaurants make or break on one or two nights in a month. A calculated social engineering attack like this could bring down hundreds of restaurants in a city, which would cause millions of dollars in lost taxes, and y…

Someone has already done that: https://chicago.eater.com/2018/3/5/17078210/opentable-reserv...

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#103

Post by fixermark »

freehunter wrote: post #78
spydum wrote: post #74

Earlier quoted context omitted.

literally effort. when you lower the attackers effort and cost to try an attack, the attempts generally go up.

I meant, you could build a bot that calls. We have the technology already, and the people on the other end probably won't notice. Plus the "do it over the Internet" thing where screen scraping and scripting is super easy.

But could you build a bot that calls and is convincing enough to trick the target into actually accepting the request as genuine and reserving the timeslot?

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#104

Post by cosmic_ape »

They have listed the problems with such systems in the first paragraph. They claim to overcome these by restricting to very specific domains. But specific domains are usually still wide. A table for two, but in the garden or inside? Inside on which floor? Sometimes it doesn't matter, sometimes it does, and these things are specific and different for every business.

Really skeptical about this. And if this does become a thing, it will dumb down the interaction.

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#105

Post by criddell »

bduerst wrote: post #31
542458 wrote: post #13

This is going to be awesome/terrible for social engineering attacks. We'll be training millions of people to to trust phone calls from "Google". But are you talking to Google-calling-on-behalf-of-your-boss, or Google-calling-on-behalf-of-some-phisher? Or better yet, some custom system that pretends to be Google over the phone? Who knows!

This isn't that different than all the bad actors calling from the "IRS". Educating people about giving personal details out over the phone has always been a challenge and will continue to be one.

Or maybe we should fix the phone system. Something is terribly broken when my connection to YouTube is better protected and authenticated than when I'm talking to the IRS over the telephone.

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#106

Post by [deleted] »

minimaxir wrote: post #2

> The system also sounds more natural thanks to the incorporation of speech disfluencies (e.g. “hmm”s and “uh”s). These are added when combining widely differing sound units in the concatenative TTS or adding synthetic waits, which allows the system to signal in a natural way that it is still processing. (This is what people often do when they are gathering their thoughts.) In user studies, we found that conversation…

[deleted]

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#107

Post by adwi »

Is any product remotely as close to this with natural-sounding human speech? Are we entering an era where all voice assistants are getting to that "Her" level in terms of human-like vocal quality?

That part is as impressive to me as the semantic parsing it's doing on that call.

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#109

Post by fixermark »

ocdtrekkie wrote: post #4

Never have we had a more clear example of why Google needs to be regulated, and laws against what they're doing needs to be passed. The fact that they think it's okay to have bots pretending to be human making phone calls, shortly after demonstrating how quickly they can copy someone's voice (re: John Legend), it shows a blatant disregard for what they're creating.

Given the two technologies you've highlighted, what specific regulation would you recommend be added to prevent what bad behavior?

Re: Google Duplex: An AI System for Accomplishing Real World Tasks Over the Phone

#110

Post by sgslo »

This is one end-game of AI - changing the economics of scams.

Imagine how a 'long con' works today: a scammer befriends a person online through a video game or social platform, and develops a rapport with them over the span of days, weeks, months. After some trust has been gained, the scammer then requests money from the victim. Does this happen a lot today? I don't know, but certainly one reason it doesn't is the economics of the scam. Who wants to spend a significant period of time gaining trust just for the chance of a payout?

AI is going to flip this on its head. Rather than dedicating hundreds of hours of a scammers time, a scammer could instead use a system like Duplex to befriend hundreds or thousands of victims simultaneously. Let it run for a few months, developing a strong rapport with the target, until the AI finally requests some money from the victim.

Yes, duplex is for completing specific tasks, but how much of a difference is there between "Duplex, book a table for four at 8pm" and "Duplex, ask my victim about how their day was"?

Post reply on HN