Gary Marcus literally gave a talk about this last week in my department's ML seminar. I asked him how sure he was that humanity would eventually achieve AGI and he said 100%. When I asked him when that would be he replied 30-100 years. Interesting perspective.
We can’t trust AI systems built on deep learning alone
101–110 of 133 posts
Re: We can’t trust AI systems built on deep learning alone
#102Earlier quoted context omitted.
> Once a human learns chess Humans have orders of magnitude more neurons, more complicated neurons, more intricate neural structures, and their training data is larger and more varied.
Right, which is why it makes sense to apply the word “learning” to what we do. In contrast to “machine learning” which is merely a fancy way to say “data processing with massive compute”.
Well if arguments by reduction are on the table then "learning" is just a fancy way to say "complex chemical process in the hippocampus that we don't understand that well".
Re: We can’t trust AI systems built on deep learning alone
#103I've been waiting for the Symbolic/NN pendulum to starting swinging back the other way and start settling in the center. NN/DL is great for the interface between the outer world and the inner world of the mind (pattern recognition and re-construction), and symbolic AI more straightforwardly represents more "language of the mind" tasks, and easily handles issues like explanation and other meta-behaviors that with DL i…
I'm not an expert but it seems like whatever symbolic reasoning humans have is pretty rudimentary anyways compared to what we're doing on computers already, so I could see the union being very powerful.
Re: We can’t trust AI systems built on deep learning alone
#104Earlier quoted context omitted.
Right, which is why it makes sense to apply the word “learning” to what we do. In contrast to “machine learning” which is merely a fancy way to say “data processing with massive compute”.
Well if arguments by reduction are on the table then "learning" is just a fancy way to say "complex chemical process in the hippocampus that we don't understand that well".
I mean, if that combination of words causes people to stop making or believing inflated claims about this kind of tech, go right ahead.
Re: We can’t trust AI systems built on deep learning alone
#105Earlier quoted context omitted.
Of course if you mess with a function's inputs in ways it's never seen it's going to "not understand" what's going on. This is an agent which only knows 8x8 space. Train it on variable spaces, and you'll get an agent that can play on variable spaces. In fact, you can probably speed things up drastically by using transfer learning from a model which already learned 8x8 space and modifying the inputs and outputs to mat…
> What, precisely, are you attacking here? Please refer, precisely, to my earlier comment in this thread. https://news.ycombinator.com/item?id=21109193
The poster you responded to is exactly right. The algorithm is general with respect to variants of games and completely different games.
Re: We can’t trust AI systems built on deep learning alone
#106> AlphaGo can play very well on a 19x19 board but actually has to be retrained to play on a rectangular board. This right here is the soft underbelly of the entire “machine learning as step towards AGI” hype machine, fueled in no small part by DeepMind and its flashy but misleading demos. Once a human learns chess, you can give it a 10x10 board and she will perform at nearly the same skill level with zero retraining.…
> Once a human learns chess Humans have orders of magnitude more neurons, more complicated neurons, more intricate neural structures, and their training data is larger and more varied.
Agreed except for training data is larger. Training data is often far smaller for humans. You probably saw a few cats before generalizing and understanding what a cat looks like. A neural net might require hundreds of thousands if not more samples to be a robust classifier for cats. AlphaGo et al look at tens of millions of games, humans look at a small fraction.
Re: We can’t trust AI systems built on deep learning alone
#107Earlier quoted context omitted.
Former world chess champion Capablanca suggested a 10x8 board with two additional pieces in the 1920s, but there have been many variants proposed earlier and later [0]. Grand Chess [1] is the most known 10x10 variant, also with 2 additional pieces and a different start position, castling not allowed. See wikipedia for links to programs implementing these rules. [0] https://en.wikipedia.org/wiki/Capablanca_Chess [1] h…
Interesting. Note that both of those types involve introducing new pieces. I would argue that this changes the fundamental nature of the game in a way that increasing board size alone doesn’t. The reason is that I (as a human chess master) would need to retrain myself to learn the new piece movements. What I really want is a 10x10 or even 8x10 board using the original set of pieces. This would be sufficient to prove…
Wouldn't adding more of the existing pieces in the back row change the game a bit too? Which piece(s) would you suggest having more of to accomodate the extra fields?
Re: We can’t trust AI systems built on deep learning alone
#108Earlier quoted context omitted.
Interesting. Note that both of those types involve introducing new pieces. I would argue that this changes the fundamental nature of the game in a way that increasing board size alone doesn’t. The reason is that I (as a human chess master) would need to retrain myself to learn the new piece movements. What I really want is a 10x10 or even 8x10 board using the original set of pieces. This would be sufficient to prove…
Wouldn't adding more of the existing pieces in the back row change the game a bit too? Which piece(s) would you suggest having more of to accomodate the extra fields?
It would change the game but not enough to reduce the performance gap between an expert and a novice. See my grandparent comment for a specific suggestion on which pieces to use.
Re: We can’t trust AI systems built on deep learning alone
#109Earlier quoted context omitted.
Almost all of the human strategies and concepts would still apply: center control, square control, development, initiative, king safety, the opposition, etc. The only exceptions would be fringe concepts like opening theory (already moot in Chess960) and endgame edge cases. If you’re still not convinced, I’ll prove that skills transfer by playing bullet against anyone who can make a 10x10 variant playable online. [Edi…
I'm pretty sure this isn't how chess masters actually play chess. That'd be too slow and error prone. They pattern match very heavily in the beginning and towards the end. And all those patterns would be wrong on a larger board with more chess pieces. At least for chess.
No, huge majority of those patterns are going to transfer over to the 10x10 board.
Re: We can’t trust AI systems built on deep learning alone
#110The opposing argument, by Rich Sutton, Distinguished Research Scientist at DeepMind: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
there are some surprisingly weak arguments in the text. It's correct to not treat computational resources as constant, but ot treat them as unimportant or negligible is awful as well. Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. If the goal is AGI this might get exponentially worse. Intelligence needs to take resource c…
Is the weak argument that computation is not guaranteed to increase in the future, or that it won't increase fast enough for AI researchers?
Most of the article is describing past scenarios, only the last 3 paragraphs make the argument that the past is a good representation of the present