Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Preparing for Malicious Uses of AI

blog.openai.com

Post reply on HN
Print view View original

71–80 of 233 posts

Re: Preparing for Malicious Uses of AI

#71

Post by tree_of_item »

edanm wrote: post #68
tree_of_item wrote: post #37

Earlier quoted context omitted.

At the moment "strong AI" is science fiction so what could you possibly say about it? How do you even know what the "only defense" is?

A super-virus that has a 100% mortality rate, is incredibly infectious, and stays dormant for 1 year before killing the host. There's a fiction (I literally just made it up). Is it impossible to say anything about it? A bomb that is 1,000 times for powerful than a nuclear weapon. Again, currently science fiction. Can't say anything about that either? The fact that it doesn't exist right now doesn't mean we can't say…

A "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built?

Yes. It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they're not similar.

Most of the literature I've seen about AGI is about theories on how to build it, or theories on how to build it safely (e.g. so it has stable preferences that align with human values). Not stuff on "how to defend yourself against a rogue AGI", because there really is nothing you can say about that.

Re: Preparing for Malicious Uses of AI

#72

Post by adrianN »

nradov wrote: post #51
taneq wrote: post #12

It may be superfluous to point this out but this seems to be talking about malicious uses of narrow AI, rather than malicious strong AI. The only defense against a malicious strong AI is, of course, a friendly strong AI.

A JDAM through it's power supply would be an effective defense against a malicious strong AI.

Only if you recognize its malicious intent before it convinced you to implant the fancy new chips that it invented right into your brain.

Re: Preparing for Malicious Uses of AI

#73

Post by tjr225 »

tyrex2017 wrote: post #63
Mizza wrote: post #30

The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones…

i dont share your concerns at all. my daily feelings and activities wouldnt change, whether facebook and google know about them or not. Contrast that to being killed by a drone. privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they dont. you can argue that loss of privacy allows corporations to manipulate us,…

> privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they don't.

It is a real problem and plenty of people do change their use of their tools.

Re: Preparing for Malicious Uses of AI

#74

Post by zitterbewegung »

I am wondering does anyone have a survey or list of AI exploits or malicious actions done on production services or systems ? For example like if a misclassified image that would target a image recognition system (such as Clarfai)? I have only seen papers that have theoretical attacks so far.

Re: Preparing for Malicious Uses of AI

#75

Post by seren »

tyrex2017 wrote: post #63
Mizza wrote: post #30

The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones…

i dont share your concerns at all. my daily feelings and activities wouldnt change, whether facebook and google know about them or not. Contrast that to being killed by a drone. privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they dont. you can argue that loss of privacy allows corporations to manipulate us,…

Some every day scenario, with an impact,

* You apply for a loan, it is refused

* Your admission, for you or your children, to a school or university is refused because more worthy people have applied

* Your job application is refused,

All because of a fuzzy social score, that is mined from your contacts, activities on social network, meta data gathered from your smartphone in an opaque way, CCTV cameras, etc

I mean if we have a way to rate people, why not use it everywhere !

This is already partially implemented in China, I remember reading an example where with a low social score you have to pay for a deposit, and you don't have to with a high social score.

In a way, that sound efficient and meritocratic, but also ruthless if you end up with a low score.

edit : add a reference with some every day life examples

https://www.wired.com/story/age-of-social-credit/

Re: Preparing for Malicious Uses of AI

#76

Post by tjr225 »

simias wrote: post #11
akerro wrote: post #2

You can have 1000 of companies that act fair and don't use AI for malicious purposes, but there it that one company or community that doesn't... and then someone sends you gay porn with your face in it.

I'm not too worried about that. The moment that type of technology becomes widely available is the moment this type of blackmail loses all edge. You might even actually start doing gay porn IRL and people will assume that it's been "deepfaked". The corollary is a little more worrying: any kind of incriminating document about a politician or public figure will be dismissed as a fake immediately. I mean, they already d…

> We have a brave new world ahead of us where you won't be able to trust anything you see or hear through any media, no matter how convincing it seems. That's pretty terrifying IMO.

Anyone can correct me if I'm being a bit dramatic but personally, it feels like we are very much already there.

Re: Preparing for Malicious Uses of AI

#77

Post by jonkiddy »

edanm wrote: post #68
tree_of_item wrote: post #37

Earlier quoted context omitted.

At the moment "strong AI" is science fiction so what could you possibly say about it? How do you even know what the "only defense" is?

A super-virus that has a 100% mortality rate, is incredibly infectious, and stays dormant for 1 year before killing the host. There's a fiction (I literally just made it up). Is it impossible to say anything about it? A bomb that is 1,000 times for powerful than a nuclear weapon. Again, currently science fiction. Can't say anything about that either? The fact that it doesn't exist right now doesn't mean we can't say…

This made me think of a riddle I once heard recently from a high school student. "What is sexually transmitted, 100% fatal, and literally everyone you know is infected? ...Life."

Re: Preparing for Malicious Uses of AI

#78

Post by ynniv »

kypro wrote: post #35
Mizza wrote: post #30

The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones…

> I'm more concerned about Google using AI to mine every conversation I've ever had or my browsing history to classify me as a dissiden before I apply for a visa to travel to China or the United States, or as a deadbeat before I apply for a bank loan, or sick before I apply for insurance, or as unrehabilitatable before I apply for parole. I'm quite paranoid about this, yet whenever I speak to people about it either p…

The "best" part is how good an AI would be at privilege escalation. Go ahead, build out that security camera network, automate your cars, store your money using cryptography. I'm a geek and I love the way technology increases our collective prosperity, but most of our security relies on the attackers being approximately as smart as any other human. To a sufficiently capable intelligence, encrypted machine code is just as understandable as the most elegant source code. Once you fully understand something, anticipated edge cases are easy to find and exploit... every day we expand the capabilities of a future strong AI.

Re: Preparing for Malicious Uses of AI

#79

Post by ynniv »

mwaitjmp wrote: post #25
strawcomb wrote: post #18

I would suggest that we shouldn't ever rely on digital archiving of important information. There should always be a copy of the information in analog, that can be dated & verified with analog methods.

This is one feature a blockchain excels at. People have stored the Bitcoin white paper in the blockchain. Anyone can then download it, and verify it is the untouched original.

Only until the hash is broken.

Re: Preparing for Malicious Uses of AI

#80

Post by tyrex2017 »

tjr225 wrote: post #73
tyrex2017 wrote: post #63

Earlier quoted context omitted.

i dont share your concerns at all. my daily feelings and activities wouldnt change, whether facebook and google know about them or not. Contrast that to being killed by a drone. privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they dont. you can argue that loss of privacy allows corporations to manipulate us,…

> privacy concerns is rather an interesting intellectual problem, and not a real one. if it was a real problem, people would change their use of those tools but they don't. It is a real problem and plenty of people do change their use of their tools.

i dont believe so. can you give an example?

duck duck go, for example, has 0.3% market share.

you need to look at the typical john and mary on the street.

Post reply on HN