Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Philanthropy Project awards a grant of $30M to OpenAI

openphilanthropy.org

Post reply on HN
Print view View original

171–180 of 186 posts

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#171

Post by Smaug123 »

joshuamorton wrote: post #163
Smaug123 wrote: post #160

Earlier quoted context omitted.

Have you ever seen a CPU running a program which is trying to communicate with the outside world through its electron side-channels? As far as I can see, your argument is "no computer has ever done this, and also we understand computers well enough to know that no computer ever will". The first clause is obvious, since no computer has ever been made to do this. The second clause is assuming without proof that we will…

Pray explain how you could use the electrons coming out of a CPU as a side channel. I don't need anything specific, but I'd prefer something that doesn't sound like its taken out of a Heinlein novel. You're again using terms incorrectly. A "side channel" implies that someone is listening to information that is unintentionally leaked. Unless your expectation is that this CPU is going to start side channeling our minds…

A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence of the physical implementation", by analogy with the standard "information"-related "side channel".

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#172

Post by idlewords »

mistercow wrote: post #169
idlewords wrote: post #95

Earlier quoted context omitted.

Holden is engaged to the sister of the guy he is funding. Hence, "future brother-in-law".

That's not a response to my comment.

You said "sister's fiancé", which is backwards.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#173

Post by MichaelGG »

joshuamorton wrote: post #165
MichaelGG wrote: post #109

Earlier quoted context omitted.

Right, because there's no way even small computers can communicate through air. Just one tiny crack is all that's needed. And that's not even going into things the AI might say that'll convince the gatekeepers to just voluntarily let it out.

This makes me think you don't know what the word "airgapped" means in this context.

Sure, you're counting on the AI to not be able to exploit its hardware to communicate with anything. That seems like a huge assumption against a super intelligence.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#174

Post by nbouscal »

Smaug123 wrote: post #171
joshuamorton wrote: post #163

Earlier quoted context omitted.

Pray explain how you could use the electrons coming out of a CPU as a side channel. I don't need anything specific, but I'd prefer something that doesn't sound like its taken out of a Heinlein novel. You're again using terms incorrectly. A "side channel" implies that someone is listening to information that is unintentionally leaked. Unless your expectation is that this CPU is going to start side channeling our minds…

A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence…

I distinctly remember a paper about AI figuring out how to either get wifi or send radio waves without access to the relevant hardware. Can't find the link at the moment though :/

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#175

Post by nbouscal »

rspeer wrote: post #158
nbouscal wrote: post #69

Earlier quoted context omitted.

What's your suggested alternative? It's really easy to criticize, but a lot harder to do any better. If you aren't suggesting any other methods, you're implicitly saying we should just ignore the problem. I really don't think that will end well.

My suggested alternative is prioritizing problems appropriately. This "Open Philantropy Project" is a guy taking money that used to be for curing malaria (and I respect what he used to do on that front!) and giving it to his roommate. You can do better solving a terrible problem that does exist than solving a sci-fi problem that doesn't exist. I know it's usually uncouth to compare different charities like this, but…

Nobody thinks we won't notice when it's more imminent; the concern is that capability research has always been faster than safety research. If we wait until it's imminent, we risk having no chance of catching up in time. Why not devote a tiny fraction of our funding to starting the research now, so we can be ready on time? Worst case we're early... which in this context is a lot better than being late.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#176

Post by Avshalom »

nbouscal wrote: post #175
rspeer wrote: post #158

Earlier quoted context omitted.

My suggested alternative is prioritizing problems appropriately. This "Open Philantropy Project" is a guy taking money that used to be for curing malaria (and I respect what he used to do on that front!) and giving it to his roommate. You can do better solving a terrible problem that does exist than solving a sci-fi problem that doesn't exist. I know it's usually uncouth to compare different charities like this, but…

Nobody thinks we won't notice when it's more imminent; the concern is that capability research has always been faster than safety research. If we wait until it's imminent, we risk having no chance of catching up in time. Why not devote a tiny fraction of our funding to starting the research now, so we can be ready on time? Worst case we're early... which in this context is a lot better than being late.

Worst case the safety research ends up causing the AGI to be actively antagonistic. Slightly less worst case it never leads anywhere and AGI ends up turning us into mouthless slugs anyway. Third-ish worst it makes AGI suborn-able and whoever gets to it first becomes our god-king.

many many worse situations later we get to:

It never leads anywhere but AGI never happens anyway and it just ends up being welfare-for-future-phobes instead of curing debilitating disease or buying every homeless person in a city a new pair of shoes or whatever.

"we're early" is in fact the best possible scenario, actually it's basically the only positive scenario.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#177

Post by Jach »

icebraining wrote: post #143
nbouscal wrote: post #141

Earlier quoted context omitted.

Nobody is saying human-level AI is ten years away, people are saying there's some chance of it in ten years. Median 50% estimate is quite a bit longer. But a 10% chance just means divide the impact by ten, and with an event like this that still leaves you with a really damn big number.

I get that, I still find the numbers absolutely unrealistic. They're just setting it up for another AI winter.

If an AI winter is going to happen again it's not going to be from the AGI side, but from the over-hyped deep learning stuff failing to provide much business value at the enterprise level (where it's only starting to be exploited by the big players). At least compared to simple linear regression.

The AlphaGo situation is an interesting one. Did you have any predictions for when an AI would beat a top pro at Go? I didn't really learn to play until sometime in 2015 but I was amazed AIs still hadn't dominated, they weren't even close. Still I saw that with the single trick of MCTS AIs had improved a lot, and seemed to have some steady improvement year after year for a while. I don't think it would be unreasonable to have predicted that at some point with X amount of computing power an AI could be made to win. Then later that year I saw a paper that reported they were beating all the MCTS bots with a deep learning based one they were making. Immediately it seemed clear that the first person to create a fusion of deep learning + MCTS would create a very strong Go AI bot, but would it beat pros? Maybe with a year of effort by a big company using custom hardware like IBM's Deep Blue or more likely these days GPU clusters, but would it happen soon? Not for a year at least. Turns out it was already happening (AlphaGo guys started in 2014, and Facebook had a project going too) and the Fan Hui matches being announced took a lot of people by surprise. Some because their predictions were ignorant of advances in either or both of MCTS or deep learning and so still predicted many years before computers would win. I was more surprised it was done without anyone hearing about it sooner. Even so it wasn't clear it could beat Lee Sedol a few months later, since there's another big gap between lower pros and higher pros, but it did.

A couple lessons to take from AlphaGo: we don't necessarily know what's actively being worked on around the world nor how far along it is, and problems that seem insurmountable with current computer hardware can suddenly become solved with the right fusion of existing ideas that haven't yet been fused.

Black swans are hard to predict, disagreements over the predictions are totally normal and fine, I think the harder disagreement is just getting people to accept that should nothing specifically hinder it (like an extinction event from an asteroid/disease, or successfully enforced bans on AI research), AGI is inevitable at some point in humanity's future. There probably should be some research done into its safety, and since the prediction problem is so uncertain, there's no pressing reason not to start now or indeed 10 years ago. "You could have used that money for feeding Africans!" is a non-argument.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#178

Post by joshuamorton »

nbouscal wrote: post #174
Smaug123 wrote: post #171

Earlier quoted context omitted.

A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence…

I distinctly remember a paper about AI figuring out how to either get wifi or send radio waves without access to the relevant hardware. Can't find the link at the moment though :/

I expect you're referring to this article:

https://www.damninteresting.com/on-the-origin-of-circuits/

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#179

Post by joshuamorton »

Smaug123 wrote: post #171
joshuamorton wrote: post #163

Earlier quoted context omitted.

Pray explain how you could use the electrons coming out of a CPU as a side channel. I don't need anything specific, but I'd prefer something that doesn't sound like its taken out of a Heinlein novel. You're again using terms incorrectly. A "side channel" implies that someone is listening to information that is unintentionally leaked. Unless your expectation is that this CPU is going to start side channeling our minds…

A side channel is a channel through which information may leak because of the physical instantiation of an algorithm. It's not much of a stretch to include "things which let us manipulate the world" in that; do you have a better term? I thought the meaning was obvious, but apparently it's not: by "side channel" I here mean "unintended means of affecting the world by a mechanism derived from an accidental consequence…

I think the closest conventional thing would be a sandbox escape/backdoor. Although (not that I'm an expert) I've never heard of anything close to a sandbox escape using side-channel like things. That said, like I said, most side channel attacks are either time based, or involve things like heat and power usage of the system.

The thing about all of these is that they generally allow you to get a small amount of data out that can sometimes help you with things. But again, without ascribing magic powers to the system, all the stuff that it can directly affect: power draw, temperature, disk spin speeds, monitoring LED blink speeds, noises, even the relatively insane things like EM frequency emissions can all be controlled relatively easily, and no matter how smart it is, I don't see an AGI violating physics.

Re: Open Philanthropy Project awards a grant of $30M to OpenAI

#180

Post by schoen »

Smaug123 wrote: post #159
schoen wrote: post #144

Earlier quoted context omitted.

This is an abuse of Rice's theorem (which seems to be getting more common). Rice's theorem says that no program can correctly decide what an arbitrary program will do , not that no properties of programs can be proven. There are useful programs about which non-trivial properties can be, and have been, proven, and Rice's theorem is no limit on the complexity of an individual program about which a property may be prove…

My argument is as follows. You're right, and I'm appealing to a stronger Wolfram-esque version of Rice, together with the fact that humans readily throw non-essential goals under the bus in pursuit of speed. * Building AI is a race against time, and in such races, victory is most easily achieved by those who can cut the most corners while still successfully producing the product. * As a route to general AI, a neural…

Those arguments are plausible, and thanks for the clarification.

I just hate to see Rice's theorem interpreted as "nobody can ever know if a program is correct or not". People have been making a ton of progress on knowing if (some) programs are correct, and Rice's theorem never said they can't.

Post reply on HN