Live data from Hacker News

What AlphaGo Zero teaches us about what’s going wrong with innovation

timharford.com

31–40 of 116 posts

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#31
post #26
post #5

Mmmh, I am not a specialist and I don't know the numbers, but it seems to me that fundamental research is not less active than it used to be. Physics made a lot of progress in materials (nano tech, weird polymers, and so on), in building batteries, in finding the higgs boson and gravitational waves, and I'm sure plenty of other fields. Medical research has advanced a lot with the invention of CRISPR. CS has grown a l…

Cryptography has also been progressing well. Same for various encoding formats; Opus is both better quality per bit & uses less CPU than Vorbis

> this matters because most basic research ends up being commercially useful eventually

That is quite a positive view. As a university professor I see a lot of speculative work. It is presented as fundamental, but makes assumption which do not hold in the real world. Hence a lot is mostly ignored.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#32

I don't think there is a very sharp distinction between results oriented R&D and "basic research". In the article, IBM's deep blue is dismissed as a dead-end victory but apparently alphago is not? Why? They both seem identical to me in goals and research methodology. On a side note, I cannot wait for general super intelligence. It cannot come soon enough. I'm tired of being poor and stuck in a fucking rut, and contem…

If their goals were similar, Google would have stopped when AlphaGo beat top go players, and AlphaGo Zero wouldn't exist.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#33
post #17

This article is off in so many dimensions. Fundamental research is not less active, but it's happening in different places (e.g. the Google Brain team). Find the most profitable companies and you'll find the research. And to suggest a computer Go player, taught in a few days, is a "marginal improvement" over decades of "AI research": as my kids say, "wut?" If anything, today's deep learning driven AI is a prime examp…

The articles wasn't saying that AlphaGo Zero was a marginal improvement. On the contrary, I think it was saying that the type of improvement AlphaGo offered is an outlier because it approached the task in a fundamentally different way (especially with AlphaGo Zero where it didn't use training data).

I think the primary argument is that much of the industry has generally not been doing that sort of research for ground-breaking moon-shots. Instead most AI researchers focus on optimizing for metrics which lead to small and safer short-term improvements in a particular niche application as opposed to pursuing large and riskier long-term ones.

I've seen this same line of criticism before of the way in which AI systems have been generally designed for decades and to me it rings true. Things like the Turing award arguably lead researchers astray. Essentially, the crux of the argument is that most people focus on climbing trees to achieve success when the real goal is reaching the moon or they build better springs when the real goal is achieving flight. [1] Both show some short-term improvements on certain heuristics, but they are obviously the wrong approach if you want serious breakthroughs.

I think it's a valid criticism that most of us are still focusing too much on the wrong metrics to make many serious advances. If you look at AI, there haven't really been that many major breakthroughs. We just haven't been all that creative and most advances are ultimately tweaks on existing technology or involve throwing more data through deeper neural nets. Back-propagation (popularized by Geoffrey Hinton) for deep-learning with neural nets was a big deal and an important idea. Since then, there hasn't been much that's earth-shattering. Generative adversarial networks are arguably a big idea. LSTM as well. The work by Naftali Tishby's group on understanding what's going on with information in neural nets is a significant development. Hinton's capsule networks also seem like they may be a big idea. A few people have recently started publishing some work on building AI that builds better AI. However for the most part, it seems like the vast, vast majority people in AI aren't aiming to do any fundamentally ground-breaking things. They mostly look around at the existing body of research and slightly tweak the tools that seem most suitable for the problem domain they are working on. (This isn't entirely surprising for a number of reasons involving the incentives in the industry, but it is something worth discussing.)

Personally, I don't think that it's necessarily a bad thing that we've been sluggish with AI advances. You don't necessarily want to hand power tools to children. I think our society is unfortunately full of unwise and unkind people with already more power than they should have. Our social institutions for distributing power wisely and responding to abuse of power are far less mature than our technology and I suspect the risks of abusing advanced technology dwarf the enormous benefits they can bring.

[1] https://www.eecs.harvard.edu/shieber/Biblio/Papers/loebner-r...

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#34
post #5

Mmmh, I am not a specialist and I don't know the numbers, but it seems to me that fundamental research is not less active than it used to be. Physics made a lot of progress in materials (nano tech, weird polymers, and so on), in building batteries, in finding the higgs boson and gravitational waves, and I'm sure plenty of other fields. Medical research has advanced a lot with the invention of CRISPR. CS has grown a l…

If you read the science fiction series "The Three-Body Problem" (highly recommended), it makes a very compelling argument that fundamental research is the most important investment in the future.

For example, fusion drives, not traditional stored rocket propellant engines, will be necessary to navigate between planets and the outer solar system. Also, existing known behaviors/laws of physics aside, the book posits that colonization of other planets/stars in the universe requires achieving light speed travel (along with hibernation technology).

However, the other argument the book series makes is that there needs to be a strong motivator to get all the countries and economies of the world to focus on fundamental research and applying it.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#35
Projects like DeepBlue and AlphaGo are not fundamental innovation nor research, they are just PR stunts that show the expertise of the company making them.

TBH, winning a game of chess or go has little value in itself, except for the limited market of selling chess or go software. The reason they are doing that is mostly for publicity. IBM makes computers, and they show how good they are at it by having one beat top players at chess, and Google makes machine learning based products and they use AlphaGo to show how good they are at it.

Chess and go don't drive innovation, they are just a side effect of real innovation.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#36

It is strange to read this, remembering how Microsoft has been lambasted time and again for doing so many interesting things with Microsoft Research and hardly ever taking any of them to the product stage. Is Microsoft unique in it's too much R and not enough D approach?

MSR has been heavily prioritized onto doing products and commercialization, stimulated by a re-org. In my research area, there have been a number (more than ten) of high profile departures of researchers from MSR, frequently going back to academia, which is madness.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#38
post #35

Projects like DeepBlue and AlphaGo are not fundamental innovation nor research, they are just PR stunts that show the expertise of the company making them. TBH, winning a game of chess or go has little value in itself, except for the limited market of selling chess or go software. The reason they are doing that is mostly for publicity. IBM makes computers, and they show how good they are at it by having one beat top…

While I understand your cynicism in the practical applicability of a chess or go-playing AI, I think you are significantly underestimating the theoretical innovations contributed to the field every time these models are substantially improved. Much of the work that goes into improving something like AlphaGo is cross-applicable and cross-pollinated to other research projects, and gradually trickles out into other domains with much more real-world impact.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#39
post #13

Earlier quoted context omitted.

I think charity improves their public image more than fundamental research, that will be known only by specialists. Moreover, they have large tax incentives for charitable activities (not sure if they have for research too) For the military computers, they had a clear interest in doing so, and the military keeps most of their research hidden, so I think it's not really comparable to public research

> large tax incentives for charitable activities People misunderstand how these work. You don't get money by giving away money. What happens is that the charity gets the money as if it were pre-tax, that's all. (Trying to get the money back into the company from the charity after you've got the tax break is fraud)

> People misunderstand how these work

Perhaps, but that doesn't make their conclusion incorrect.

Charitable contributions, properly structured and carefully targeted, are basically tax-free ad spend.

Re: What AlphaGo Zero teaches us about what’s going wrong with innovation

#40
"It was a time when companies weren’t afraid to invest in basic science." No they were probably afraid, but they were forced to invest in science by states. AT&T did not decide to invest massively in science and risky projects like Unix, they were forced to. Please stop thinking companies are behind innovation. A great piece of article that demestify this myth: https://www.theguardian.com/technology/2017/may/11/tech-inno...
Post reply on HN