Live data from Hacker News

AI Update, Late 2019

blog.piekniewski.info

31–40 of 50 posts

Re: AI Update, Late 2019

#31
As someone who has worked as a researcher in one of the big AI research labs, I completely agree with this post. There has been true progress in a few ML subfields over the past few years, most noticeably representation learning for image recognition and text/ translation, but 99% of what you read in both scientific papers (which are more PR than ever) and the general media is nothing but hype. Especially over the last 2-3 years or so I haven't seen anything novel. IMO that's mostly a result of the confluence of perverse incentives at various levels:

- Academics need to create PR and hype to increase their chances for grants

- PhD students need to publish papers, and thus convince reviewers, with unnecessarily complex and hype-filled language, that their papers are good. They are also more incentivized than ever to create their own personal brand (via hype-filled blog post or videos) to increase future employment opportunities. More PR also means more citations, which is metric academics are often evaluated on. After all, if you work on something related, you're pretty much obligated to cite research that everyone has heard about, right?

- Startups, as it has always been, jump on the latest trend to increase their chance of raising money from investors. They slap AI/ML onto their pitch decks to differentiate themselves from others, or to become eligible for AI-focused funds. In reality, none of them will ever use any of the new ML techniques because they are too brittle to work in real-world products or require many orders of magnitude more data then the startup will ever have.

- Big companies want to brand themselves as "thought-leaders" in AI to drive up their share prices, hire better talent, improve their public image, convince investors, etc.

- The general media has no idea what they are talking about and wants to generate clicks. Same as always.

Put all this together and you get the current AI hype cycle. We've seen this happen with lots of other technologies in the past, what's kind of new this time is the entrance of academia into the cycle. When I first started in (ML) academia I was under the naive impression that I would be doing hard and cold science - I was so wrong. Everyone is optimizing for their own objective (grants, salary, publications, etc, see above), which makes most of the published research completely useless or simply wrong. One of the, sometimes unspoken, criteria of choosing ML projects in many of these labs is "how much PR will this create". This useless "research" is then treated as if it was a proven method and picked up by startups to convince clueless investors or customers with "look at this latest paper, it's amazing, we will monetize this, we're at the forefront of AI!", or by the general media to create more hype and drive clicks.

One important point that the blog post makes that is always overlooked is this:

> Now what this diagram does not show, is the amount of money which went into AI in corresponding time periods.

With all the hype over the last few years, just think about how many billions of dollars and tens of thousands of some of the smartest people on this planet got into the field, often to make a quick buck. With this many resources invested, would you expect there to be no progress? Obviously there will be, but most of it is smoke and mirrors. People think that the progress comes due to new AI techniques (Neural Nets), but in reality, if you were to take the same people and money and forced them to make progress on the same problems using some other technique, let's say probabilistic models or even rule-based or symbolic systems, they would've done just as well, if not better.

Re: AI Update, Late 2019

#32
Seems like people will keep being needlessly negative and dismissive of AI right up until the singularity.

But really, what did AI do to this guy? ML really does have real world applications. Though many self driving start ups are overblown, my Tesla really does drive me to work everyday.

As always, things are easy to critize, and hard to create.

Re: AI Update, Late 2019

#33

This blog gets passed around a lot recently. While I do draw value from the thorough observations of developments, the amount of text the author spends on shallow negativity can feel like the same waste of time as the overhyping PR machine he is reacting to. There is without doubt something novel in the successes of convnets for sensory perception, deep Q-learning for decades-old and new game problems, artificial cur…

[deleted]

Re: AI Update, Late 2019

#34

Sure there is a lot of hype in AI/ML right now, but this post reads like there is an axe to grind with all ML. it ignores true progress made in a lot of areas and denigrates the whole field. to me it did not read like an objective post, but more like just a "all AI is bullshit" style blog post

It's a "Reverse AI Effect". If the AI Effect is that anything we actually understand cannot possibly constitute artificial intelligence, the Reverse AI Effect is that nothing can possibly constitute useful AI until it rises up and kills all humans.

Re: AI Update, Late 2019

#35
post #32

Seems like people will keep being needlessly negative and dismissive of AI right up until the singularity. But really, what did AI do to this guy? ML really does have real world applications. Though many self driving start ups are overblown, my Tesla really does drive me to work everyday. As always, things are easy to critize, and hard to create.

>But really, what did AI do to this guy?

He's a founder of an ML startup with published papers.

Re: AI Update, Late 2019

#36
This post is full of non sequitur like links to the PG&E wildfire prevention shutoffs after talking about how model training (which happens offline in some datacenter) will always cost lots of energy (why would you build a data center north of the bay where it can be affected by wildfires and sky high utility/real estate prices). Maybe it is meant to be humorous and I just didn't get it.

Yeah everything is harder than the first wave of hype made it seem, no this list of ridiculous hype proved to be ridiculous doesn't mean it's all useless or doomed. I get the impression the author knows this from reading the about page though, which makes me think I just missed the joke.

Re: AI Update, Late 2019

#37
post #32

Seems like people will keep being needlessly negative and dismissive of AI right up until the singularity. But really, what did AI do to this guy? ML really does have real world applications. Though many self driving start ups are overblown, my Tesla really does drive me to work everyday. As always, things are easy to critize, and hard to create.

>But really, what did AI do to this guy? He's a founder of an ML startup with published papers.

So hes a masochist or?

Seems to hate ML.

Re: AI Update, Late 2019

#38
post #30

This blog gets passed around a lot recently. While I do draw value from the thorough observations of developments, the amount of text the author spends on shallow negativity can feel like the same waste of time as the overhyping PR machine he is reacting to. There is without doubt something novel in the successes of convnets for sensory perception, deep Q-learning for decades-old and new game problems, artificial cur…

We need contrarian voices for both spotting any issues we might have overlooked, and assuring ourselves we know better. It's still better than what I can read from "LinkedIn influencers" in my feed like "Logistic regression is still the best" or "Self-driving cars will never work because of long tail"...

It’s a good read but the negativity makes it appear irrational. It would be better if he left the ranting away and focused on the realistic recap without the PR hype.

Note that AI has a history of being stalled by overly pessimist evaluations (Minsky / Papert on the perceptron, Lighthill report).

Re: AI Update, Late 2019

#39
The author uses these ([1][2]) diagrams to argue that more compute has diminishing returns. But the 'diminishing returns' are on the accuracy of correctly picking the single right category for a photo out of one thousand. Photos may simply not carry enough information to be able to meaningfully distinguish between them at that level of accuracy; existing models already exceeded humans' ability at top-5 accuracy in 2015 [3]. It wouldn't be surprising if SOTA models exceeded humans at top-1 already.

It's possible that the human baselines were bored and so performed sub-optimally when picking between the 1K classes. But the argument has now become a subtler one, much less clear cut.

As an example of categories that may be difficult to distinguish between, do you feel confident that you can reliably distinguish between the Norwich terrier [4] and the Norfolk terrier [5]? These are two separate categories in ImageNet1k.

[1] https://i0.wp.com/blog.piekniewski.info/wp-content/uploads/2...

The first diagram shows exponential growth in the compute usage of state of the art deep learning architectures.

[2] https://i1.wp.com/blog.piekniewski.info/wp-content/uploads/2...

The second diagram shows diminishing returns on Imagenet1k top-1 accuracy from doubling the size of Resnext.

[3] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.725...

[4] https://www.google.com/search?tbm=isch&as_q=norwich+terrier&...

[5] https://www.google.com/search?tbm=isch&as_q=norfolk+terrier&...

Re: AI Update, Late 2019

#40
post #9

A refreshing view of AI, this excerpt I particulary enjoyed: > I mentioned in my previous half-year update, Open AI came up with a transformer based language model called GPT-2 and refused to release the full version fearing horrible consequence that may have to the future of humanity. Well, it did not take long before some dude - Aaron Gokaslan - managed to replicate the full model and released it in the name of sci…

Just wait until your GPT2-generated MBA homework gets you full points first time, then you either won't compute, start weeping, shake rapidly or laugh like a madman. Automated essays scoring is already reality, now you get GPT2-automated ones as well.

HN crowd is often intellectual elite; imagine regular persons reading what GPT-2 produces when they can't understand what a regular grad student writes. I can use e.g. talktotransformer.com to complete some quotes like "Intel CEO said that the new 10nm CPUs will...", then post that to some Reddit thread, it would get picked up by search engines, and at some point somebody would use it in some serious work or it would spread like wildfire on sites that don't check their references.

Post reply on HN