Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Preparing for Malicious Uses of AI

blog.openai.com

Post reply on HN
Print view View original

201–210 of 233 posts

Re: Preparing for Malicious Uses of AI

#201

Post by eslaught »

kakarot wrote: post #158
3pt14159 wrote: post #103

Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponization of everything is actually happening. Since I wrote about it a month ago (Self-Crashing Cars) a number of people have reached out including people with actual insight into the military aspect of it. Militaries around the world are getting ready for true AI…

Here's the crux of the issue with respect to nuclear WMDs: In order to prevent citizens and other countries from creating nuclear bombs, our government severely limits access to relevant tools and materials, and actively seeks to censor the knowledge of how to build modern nuclear devices from other parties. In order to prevent citizens and other countries from creating dangerous rogue AIs, our government ___________…

There are really two kinds of attacks: ones that involve subverting existing networked devices to perform malicious acts (hacking), and making one's own malicious devices (building).

Of the two, attacking existing devices sounds scarier because there is a large fleet of vulnerable existing devices that can potentially be exploited. However, of the two problems, it's also the one we know more about: it's basically straight up cyber security. The AI aspect is mostly a tangent, a way to control the system once you have access.

It seems the most straightforward way to defeat these attacks is to increase their cost by promoting better cyber security (sandboxing, use of safer languages, cryptography as appropriate, etc.). That's not to say it'll be easy, but the problems are largely known problems.

On the other hand, anything that involves the attacker actually building something themselves is either going to be much smaller scale, or is going to leave a physical trail that can be traced like we do for any other sort of weapons mitigation.

Re: Preparing for Malicious Uses of AI

#202

Post by sudouser »

katzgrau wrote: post #57
Mizza wrote: post #30

The disturbing thing about this paper to me - flashy though it may be - is what they left out rather than what they kept in. OpenAI appears to only be thinking of the crimes-against-individuals segment of malicious AI, rather then the crimes-against-humanity type of malicious AI that the surveillance advertising corporations who are supporting OpenAI are building. I am far, far less worried about an assassin's drones…

I'm more concerned that AI will become a modern deity in that it's considered science and ought not to be questioned. Things like predictions of academmic success, criminal behavior, etc that impact the rights of individuals.

sounds like the minority report story, they detained people before doing crimes

Re: Preparing for Malicious Uses of AI

#204

Post by goatlover »

shkkmo wrote: post #164
tree_of_item wrote: post #71

Earlier quoted context omitted.

A "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built ? Yes . It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they…

The problem may not have been your point (that we can't reason about strong AI), but the way you justified it. Strong AI is not hard to reason about because it is science fiction. As edanm pointed out, it is entirely possible to reason about science fiction. The difference is that the examples edanm points out are extensions or modifications of things that do exist and that provides a basis from which we can reasonab…

> We can't reason about strong AI precisely because we don't have as good of an existing example from which we can extrapolate. The best we have is the human mind.

It's actually human minds. Organizations of humans like major corporations and government possess the sort of resources that we imagine a super AI would have.

Re: Preparing for Malicious Uses of AI

#205

Post by ewjordan »

shkkmo wrote: post #164
tree_of_item wrote: post #71

Earlier quoted context omitted.

A "superintelligent" AI that is to a human as a human is to a fly. Is it impossible to say anything about the world after it's been built ? Yes . It's not the fact that it doesn't exist, it's the fact that not only is it not clear that it's even physically possible, anything you come up with can just be handwaved away by this mystical property known as "intelligence". Your two examples don't have this problem so they…

The problem may not have been your point (that we can't reason about strong AI), but the way you justified it. Strong AI is not hard to reason about because it is science fiction. As edanm pointed out, it is entirely possible to reason about science fiction. The difference is that the examples edanm points out are extensions or modifications of things that do exist and that provides a basis from which we can reasonab…

> We don't even know if or to what degree 'super-intelligence' is possible. For all we know, there may be fundamental trade offs between the different skills necessary for general intelligence that may provide functional limits to the degree to which intelligence can be super human.

While this is true, it would be extremely surprising if evolution's first stab at higher intelligence (humans) came anywhere near the theoretical limits of what intelligence could be if directly selected for.

Just remove the size and energy consumption constraints imposed on the human brain by what the human body can support, and you should be able to get a significant improvement without changing the base architecture much at all.

Digitize and allow for copy/paste of "brains", and now you've got 30,000,000 copies of Einstein working together on problems but each specializing in their own pursuits, each of which doesn't get tired and lives forever, and can open very high-throughput direct neural communication channels with the other Einsteins whenever they need to share knowledge.

Start tweaking the virtual brain's "chemical" environment, and you can probably supercharge them by doing all sorts of stuff that would kill a real human but is fine when all you care about is making a simulated brain work better (virtual megadoses of Ritalin, massive tweaks to neurotransmitter behaviors, etc.).

These are just a few obvious ideas, but it's hard to imagine that together they couldn't scale intelligence up at least past the point where we could ever hope to keep up, as long as we got it into digital form in the first place. You're correct that there will be an architectural "wall" somewhere, but chances are that human intelligence is more in the "vacuum tubes and punch cards" regime than the "Moore's law is done" one. Whatever is there when that wall is hit would be so far beyond us that it might as well be called super-intelligent, even if it can't improve further.

Re: Preparing for Malicious Uses of AI

#206

Post by breuleux »

ynniv wrote: post #78
kypro wrote: post #35

Earlier quoted context omitted.

> I'm more concerned about Google using AI to mine every conversation I've ever had or my browsing history to classify me as a dissiden before I apply for a visa to travel to China or the United States, or as a deadbeat before I apply for a bank loan, or sick before I apply for insurance, or as unrehabilitatable before I apply for parole. I'm quite paranoid about this, yet whenever I speak to people about it either p…

The "best" part is how good an AI would be at privilege escalation. Go ahead, build out that security camera network, automate your cars, store your money using cryptography. I'm a geek and I love the way technology increases our collective prosperity, but most of our security relies on the attackers being approximately as smart as any other human. To a sufficiently capable intelligence, encrypted machine code is jus…

It depends on the rate of improvement of AI and what the intrinsic strength of defence is against attack. Arguably, security will start incorporating AI techniques, and by the time AI attackers surpass human capability, AI defenders would likely be too strong for humans, and quite possibly strong enough to hold against a higher intelligence.

Re: Preparing for Malicious Uses of AI

#207

Post by rland »

reggieband wrote: post #197
Diederich wrote: post #123

Earlier quoted context omitted.

> ... feel we've lost the battle for privacy. I agree with pretty much all of the concerns you and many others have cited, and I worry about that stuff too. On the other hand, I think about the past. How much privacy did most humans have in history? To be clear: I'm nothing like an expert on these topics. I'm specifically not considering the wandering nomad types. I think that the vast majority of all humans who have…

I think asymmetry plays a role here. In communities as you describe the amount of information I have is roughly equivalent to the amount of information you have. The amount of data Google has and the amount of data a government will have gives an order of magnitude advantage to whoever gets to harness it.

Beyond that, it's a power asymmetry as well.

If you have to collect info on someone "the old fashioned way" (like being siblings with them or marrying them) the power differential in the relationship is bounded by definition. To me, privacy is almost like a vector with two components—the level of privacy, plus the degree of intimacy of the relationship.

Re: Preparing for Malicious Uses of AI

#208

Post by taneq »

shkkmo wrote: post #166
taneq wrote: post #108

Earlier quoted context omitted.

> Is it impossible to say anything about the world after it's been built? Well, actually no. It's possible to say one thing (and as far as I'm aware, only this thing): You'd better have an even smarter friendly AI on your side, or you're almost certainly boned.

> You'd better have an even smarter friendly AI on your side, or you're almost certainly boned. Are you sure? Was the world better off with both the US and Russia having nukes? Perhaps, or perhaps we got incredibly lucky that we didn't destroy ourselves. It is quite possible that a world where we have a single malicious AI is better than a world where a malicious AI and a friendly AI are locked in a battle to the dea…

You're missing the point that an active, malicious strong AI is a nuke that's already been detonated.

Re: Preparing for Malicious Uses of AI

#209

Post by BatFastard »

3pt14159 wrote: post #103

Reading the comments on this article make me realize how unserious most engineers and software developers are taking this. These threats are real. The weaponization of everything is actually happening. Since I wrote about it a month ago (Self-Crashing Cars) a number of people have reached out including people with actual insight into the military aspect of it. Militaries around the world are getting ready for true AI…

Read the book Daemon is you want to see creative use of this type of tool.

Re: Preparing for Malicious Uses of AI

#210

Post by vicpara »

I don't think these issues have anything to do with AI but 100% with humans. AI is used everywhere where money is to be made. Let's take ads.

Why do we allow ads to happen all over the place (TV, radio, internet, magazines)? Because they feel stupid, harmless, how else can we sell products, etc? Well, then when a ML system is serving you ads, it's doing it so at the right time and will target you with the bang on content so that you buy exactly what it's needed. For eg. modern ML algos can pick up in which mood and mental state you are (out of 27 different states) by looking at how you read content online, what you read, how many tabs you open or how related the pages you visit are . I know this because I build these things. It works incredibly well. It finds your weak spot, your passion, your habits, your indulgences and makes sure you're always tempted. It can tell more about yourself in a split second than you can think in an hour about who you really are. That bullshit you tell yourself about you doesn't matter because you are the rat the ML algo is baiting you until you bite.

The problem is not that the AI is selling to the diabetic more cupcakes, the alcoholic booze and hipsters new gadgets like no other salesman. The idea of advertising is perverted itself from skin till bone marrow. How can you allow that to be done to the people? And we think this is how things are? OMFG.. AI is just a tool.

WMD are another dead stupid idea only humans could come up with. Me country builds insane big weapons that can whack entire cities in split second just because it can & has more money & and we were first & we won the war & no reason. It also tries to make sure nobody can build it (like Iran) so they all kneel and kiss the ring.

AI kind of levels the field. It's just a bigger nuclear bomb. My is bigger than yours taken to the next level. And that's just the beginning. AI makes it game on for more players than we like it to be. And it's not just AI (which is just math run by computers), it's mathematics, human genome research, physics, nuclear physics...all STEM subjects. You can come up with a weapon from any mix of these.

There are only two ways out of it.

The humanity dies or we learn how to all work together, chill our entitlement and find our common values regardless of religion, money, skin colour, gender or part of the world. A fart made in any part of the world can kill people everywhere.

If humanity is to turn against itself because countries or corporations play a retarded game in AI/ pharma/ nuclear/fill_in_the_blank .. this only only accelerates a bit the process but the result will be the same.

Let's ditch advertising, pharmaceuticals, OTC, bogus pills created without any fundamentals, weapon industries and all other nonsense industries or acts focused on massive profit at the cost of humanity. Let's ditch all kinds of waste and put people first for once. Let's ditch olympics where we try to prove that my country has it bigger than yours.

Let's invest in education and health, inclusive societies where everyone works hard to solve humanity's tough problems.

We keep building bigger guns but we as a society didn't grow much in empathy, inclusiveness or respect for the planet or other countries.

People are killing people not AI. A machine gun is the safest thing in the world. No single machine gun on this universe loaded itself and started firing precisely to people or animals.

Probably the best thing that can happen to this planet is to have humans vanish while there are still trees and animals around.

Post reply on HN