AI winter is well on its way

This is a deep, significant post (pardon pun etc).

The author is clearly informed and takes a strong, historical view of the situation. Looking at what the really smart people who brought us this innovation have said and done lately is a good start imo (just one datum of course, but there are others in this interesting survey).

Deepmind hasn't shown anything breathtaking since their Alpha Go zero.

Another thing to consider about Alpha Go and Alpha Go Zero is the vast, vast amount of computing firepower that this application mobilized. While it was often repeated that ordinary Go program weren't making progress, this wasn't true - the best, amateur programs had gotten to about 2 Dan amateur using Makov Tree Search. Alpha Go added CNNs for it's weighting function and petabytes of power for it's process and got effectiveness up to best in the world, 9 Dan professional, (maybe 11 Dan amateur for pure comparison). [1]

Alpha Go Zero was supposedly even more powerful, learned without human intervention. BUT it cost petabytes and petabytes of flops, expensive enough that they released a total of ten or twenty Alpha Go Zero game to the world, labeled "A great gift".

The author convenniently reproduces the chart of power versus results. Look at it, consider it. Consider the chart in the context of Moore's Law retreating. The problems of Alpha Zero generalizes as described in the article.

The author could also have dived into the troubling question as of "AI as ordinary computer application" (what does testing, debugging, interface design, etc mean when the app is automatically generated in an ad-hoc fashion) or "explainability". But when you can paint a troubling picture without these gnawing problems appearing, you've done well.

[1] https://en.wikipedia.org/wiki/Go_ranks_and_ratings

Seriously. Frankly, based on Uber's culture I would have been surprised if they didn't kill at least one person with their self-driving efforts. It's a total non-data point. The fact that Uber got as far as they did without killing anyone is strong evidence that the problem is tractable.

As for Tesla - Tesla isn't even trying to make proper self-driving cars. Tesla's goal has always been assisted driving. However you feel about that, it's really not relevant to the success or failure of self-driving cars.

OP can't possibly have been ignorant of the fact that Waymo is the clear leader here with a substantial head start, and a proven record (and an actual fleet of self driving cars now on the road), and yet he chose not to mention it. That really undermines his credibility for me - he seems clearly more interested in making his point than in accurately engaging with reality.

I've always understood the claim that deep learning scales to be a claim about deployment and use of trained models, not about training. The whole point is that you can invest (substantial) resources upfront to train a sufficiently good model, but then the results of that initial investment can be used with very small marginal costs.

OP's argument on this front seems disingenous to me.

His focus on Uber and Tesla (while not even mentioning Waymo) is also a truly strange omission. Uber's practices and culture have historically been so toxic that their failures here are truly irrelevant, and Tesla isn't even in the business of making actual self driving cars.

I'm the first to argue that right now AI is overhyped, but this is just sensationalist garbage from the other end of the spectrum.

I'm sure the same could be said for early computer graphics before the GPU race. You don't need Moore's Law to make machine learning fast, you can also do it with hardware tailored to the task. Look at Google's TPUs for an example of this.

If you want an idea of where machine learning is in the scheme of things, the best thing to do is listen to the experts. _None_ of them have promised wild general intelligence any time soon. All of them have said "this is just the beginning, it's a long process." Science is incremental and machine learning is no different in that regard.

You'll continue to see incremental progress in the field, with occasional demonstrations and applications that make you go "wow". But most of the advances will be of interest to academics, not the general public. That in no way makes them less valuable.

The field of ML/AI produces useful technologies with many real applications. Funding for this basic science isn't going away. The media will eventually tire of the AI hype once the "wow" factor of these new technologies wears off. Maybe the goal posts will move again and suddenly all the current technology won't be called "AI" anymore, but it will still be funded and the science will still advance.

It's not the exciting prediction you were looking for I'm sure, but a boring realistic one.

You’re arguing around the point here.

Almost all reasearch domains live on a log curve; a little bit gets you a lot to start with, but eventually you exhaust the easy solutions and a lot of work gets you very little improvement.

You’re arguing we haven’t reached the plateau at the top yet, but you’ve offered no meaningful evidence that is the case.

There are real world indicators that we are reaching diminishing returns for investment in compute and research now.

The ‘winter’ becomes a thing when it becomes apparent to investors that their financial bets are based off nothing more concrete than opinions like yours, when they don’t work out.

Are we there yet? Not sure, myself, I think we can get some more wins from machine generated architectures... but I can’t see any indication that the ‘winter’ isn’t coming sooner or later.

Investment is massively outstripping returns right now... we’ll just have to see if that calms down gradually, or pops suddenly.

History does not have a good story to tell about responsible investors behaving in a reasonable manner and avoiding crashes.