Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why AI is harder than we think

arxiv.org

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Why AI is harder than we think

#71

Post by Isinlor »

someguyorother wrote: post #53
audunw wrote: post #43

I think another common fallacy is to assume that humans have more general intelligence than we actually do. This is manifest in three ways: 1. Assuming that we can build an AI than can do what a human does, without being embedded in the physical world and in constant communication with other humans in the same way we are. I think this is covered by "Intelligence is all in the brain" in the article 2. Not considering…

I like this refutation of the paperclip maximizer scenario, which ties in to your observation: Let's say there's a runaway superintelligence equipped with an optimization function that says it should produce a maximal amount of paperclips. Then either its objective, "produce a maximal amount of paperclips" is defined literally or not. If it's defined literally, e.g. "make your sensors return this data consistent with…

Thinking about it in terms of intelligence is actually misleading. It's not about intelligence, it's about power to change the world.

Take SARS-CoV-2. It is a small RNA program ~32kb that duplicates and optimizes itself by random search.

It managed to hijack more computing power than our best supercomputers have [0] and kill a lot of us in the process.

By my estimates just the process of copying all SARS-CoV-2 vrions in the world with reproduction rate of 1 takes at least 1 petaFLOPS and up to 100 exaFLOPS.

That's in the range of TOP500 supercomputers (1.3 petaFLOPS - 442 petaFLOPS) [1].

SARS-CoV-2 has certainly a lot of computing power and is still able to outsmart the whole human civilization. Is that a super-intelligence?

Imagine if we manage to create equally stupid program that will figure out how to hijack heat or electrical power of our civilization and feed it into its own growth.

Quite likely it would result in nuclear meltdowns all around the world.

Exponentially growing computational processes are dangerous no matter if they are intelligent or whether they even have any objectives besides just being.

[0] https://news.ycombinator.com/item?id=26646029

[1] https://www.top500.org/statistics/perfdevel/

Re: Why AI is harder than we think

#72

Post by nl »

YeGoblynQueenne wrote: post #26
nl wrote: post #14

This paper is written by a non-technical author, criticising AI based on predictions made by non-technical pundits. By concentrating on binary outcome predictions ( Do we have "full self driving cars" available to the general public? ) it misses the real progress made in a huge number of areas. For example, one of the claims this paper claims to be false is Zuckerberg's 2015 declaration that: One of [Facebook’s] goal…

The paper criticises the benchmarks that are typically used to show that "AI systems are already better than humans in vision and [speech processing]" etc. For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not lik…

> For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems.

I'd note that I said "clear progress on language understanding".

> we could all be AIs debating the paper

if those benchmarks really did measure language understanding in terms of the level of proficiency between the researchers, then it's no surprise that the results are very different. The next chart shows the results of the most common tests for language comprehension. If we look at the results of the different tests, then we see that the average proficiency is very high (12.9 out of 20). If we look at the results of the tests for other things, then the average proficiency is very low. Again, the charts are very similar to the results of the two separate results of the two separate tests for language comprehension. So, the question is, what does this tell us about the impact of language learning on literacy? I'll give you what I think is the relevant point. I think the key point here is that language learning is the process of learning new things. So, there are some things that are learning in the language.

The text here was generated by a GPT-2 based model (aitextgen). I think there is a fair argument that it isn't far off the average level of discourse on HN.

Re: Why AI is harder than we think

#74

Post by someguyorother »

Isinlor wrote: post #71
someguyorother wrote: post #53

Earlier quoted context omitted.

I like this refutation of the paperclip maximizer scenario, which ties in to your observation: Let's say there's a runaway superintelligence equipped with an optimization function that says it should produce a maximal amount of paperclips. Then either its objective, "produce a maximal amount of paperclips" is defined literally or not. If it's defined literally, e.g. "make your sensors return this data consistent with…

Thinking about it in terms of intelligence is actually misleading. It's not about intelligence, it's about power to change the world. Take SARS-CoV-2. It is a small RNA program ~32kb that duplicates and optimizes itself by random search. It managed to hijack more computing power than our best supercomputers have [0] and kill a lot of us in the process. By my estimates just the process of copying all SARS-CoV-2 vrions…

Which the refutation makes clear: it's not about intelligence, because a sufficiently powerful intelligence will just outwit its objective function in the most efficient way possible (which is closer to "porn" than "destroy the Earth").

Grey goo is powerful not because it can outwit, but because it has so much brute force. On the other hand, intelligence is closer to efficiency of action: not requiring exponential time to solve something that's NP-complete, for instance.

I don't think you can translate computing power like you do (a crypto miner ASIC handles a lot of bits per second, but zero FLOPS as it's all integer math; a nuclear bomb is a self-propagating reaction that alters the mass equivalent of lots of bits, but zero FLOPS).

But apart from that, I agree. And I just think the "beware AI, it will become so smart that it turns into the sorcerer's apprentice" fear is misguided, because it focuses on the wrong thing. It sure makes a good narrative though! That's why it's so popular.

Re: Why AI is harder than we think

#75

Post by YeGoblynQueenne »

nl wrote: post #72
YeGoblynQueenne wrote: post #26

Earlier quoted context omitted.

The paper criticises the benchmarks that are typically used to show that "AI systems are already better than humans in vision and [speech processing]" etc. For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not lik…

> For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. I'd note that I said "clear progress on language understanding". > we could all be AIs d…

I think you're being unnecessarily mean, rather than "fair". Anyway I'll take the opportunity to point out that one reason why language understanding benchmarks are not adequate for the task is that it's very difficult to know what a system "understands" (if anything) just by looking at its output. It is also very difficult to evaluate language _generation_ tasks accurately. Finally, I have no idea how you could convince me that the above was not written by a human, rather than generated by a language model, if I didn't believe you. Which is to say, there are no good ways to know this kind of thing for sure.

At the end of the day, we have a bunch of metrics that don't measure what we want them to measure and a bunch of systems that don't learn what we want them to learn, and that are very good at gaming any metric we throw at them. The end result is a lot of uncertainty regarding true capabilities of those systems. And when careful scholarship is turned to the analysis of those systems' results, it tends to find that they're not as good as the metrics suggest. I'm repeating the point made by the article, but I agree with it very much.

Edit: the paper I link to above proposes a new benchmark called HANS (Heuristic Analysis for NLI Systems) that tries to correct for learning shortcuts in language models. It finds that (state of the art language model) BERT performs dismally on that benchmark. That's one datum. The other is that I haven't so far seen results on HANS reported in papers or benchmark aggregators etc. Language modelling work likes to cite results on e.g. GLUE, SUPERGLUE, etc, but these are exactly the kinds of benchmarks that are full of loopholes for the current approaches to exploit (even though they're supposed to not be).

Re: Why AI is harder than we think

#76

Post by nl »

YeGoblynQueenne wrote: post #75
nl wrote: post #72

Earlier quoted context omitted.

> For example, given state of the art in current benchmarks on language understanding, if those benchmarks really did measure language understanding, we could all be AIs debating the paper that itself could have been written by AI. Suffice it to say, this is not likely given the current level of "understanding" in modern systems. I'd note that I said "clear progress on language understanding". > we could all be AIs d…

I think you're being unnecessarily mean, rather than "fair". Anyway I'll take the opportunity to point out that one reason why language understanding benchmarks are not adequate for the task is that it's very difficult to know what a system "understands" (if anything) just by looking at its output. It is also very difficult to evaluate language _generation_ tasks accurately. Finally, I have no idea how you could conv…

In the HANS paper I'd point out this bit:

"we retrained each model on the MNLI training set augmented with a dataset structured exactly like HANS (i.e. using the same thirty subcases) but containing no specific examples that appeared in HANS.... In general, the models trained on the augmented MNLI performed very well on HANS"

I think it's great to find examples where language models break. But it doesn't look like they have found a fundamental problem here - just a weakness that can be fixed with "just" engineering.

I do agree there are things we don't know how to do yet. I think Chollet's "On the Measure of Intelligence" paper[1] is the best writing I've seen on this, which shows specific things that are currently hard for computers to do and give a route towards general intelligence.

[1] https://arxiv.org/abs/1911.01547

Re: Why AI is harder than we think

#77

Post by Isinlor »

someguyorother wrote: post #74
Isinlor wrote: post #71

Earlier quoted context omitted.

Thinking about it in terms of intelligence is actually misleading. It's not about intelligence, it's about power to change the world. Take SARS-CoV-2. It is a small RNA program ~32kb that duplicates and optimizes itself by random search. It managed to hijack more computing power than our best supercomputers have [0] and kill a lot of us in the process. By my estimates just the process of copying all SARS-CoV-2 vrions…

Which the refutation makes clear: it's not about intelligence, because a sufficiently powerful intelligence will just outwit its objective function in the most efficient way possible (which is closer to "porn" than "destroy the Earth"). Grey goo is powerful not because it can outwit, but because it has so much brute force. On the other hand, intelligence is closer to efficiency of action: not requiring exponential ti…

Objective functions are just weak anthropocentric abstractions. If you take a random 110 rule or any other Turing machine how can you know its objective function? What was the objective function of abiogenesis? Can we outsmart the objective function of life?

> I don't think you can translate computing power like you do.

Yes, certainly human cell nucleus will not be a good general purpose computer and in this sense it's more similar to an ASIC. So, take it as a lower bound on computing power necessary to brute-force simulate the virus replication. I think it should be accurate this way. Classical computers will need at least n operations to physically copy n bits.

There are also arguments that biology is very close to the thermodynamic limits of computations [0] [1].

> Here we show that the computational efficiency of translation, defined as free energy expended per amino acid operation, outperforms the best supercomputers by several orders of magnitude, and is only about an order of magnitude worse than the Landauer bound.

[0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/

[1] https://en.wikipedia.org/wiki/Landauer%27s_principle

And yes, nuclear bombs are extremely useless as computational devices, but also extremely dangerous. They share runaway aspect with viruses, but the good thing is that nuclear bombs self-destroy. Unfortunately programs like viruses reach equilibrium with the environment and are extremely successful at keeping on existing under natural selection. There are also good reasons to believe that viruses are in a local maximum. The global maximum would be spreading to the whole universe.

Overall, it does not matter if a computation is efficient as long as it has access to enough power and stars provide ridiculous amounts of power.

Post reply on HN