Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI winter is well on its way

blog.piekniewski.info

Post reply on HN
Print view View original

451–460 of 518 posts

Re: AI winter is well on its way

#451

Post by sqrt17 »

ujal wrote: post #389
bitL wrote: post #348

I was recently "playing" with some radiology data. I had no chance to identify diagnoses myself with untrained eyes, something that probably takes years for a decent radiologist to master. Just by using DenseNet-BC-100-12 I ended up with 83% ROC AUC after a few hours of training. In 4 out of 12 categories this classifier beat best human performing radiologists. Now the very same model with no other change than adjust…

I don't get the sentiment of the article either. I can't speak for researchers but software engineers are living through very exciting times. State of the art in numbers: Image Classification - ~$55, 9hrs (ImageNet) Object Detection - ~$40, 6hrs (COCO) Machine Translation - ~$40, 6hrs (WMT '14 EN-DE) Question Answering - ~$5, 0.8hrs (SQuAD) Speech recognition - ~$90, 13hrs (LibriSpeech) Language Modeling - ~$490, 74h…

If the dollar amounts refer to the training cost for the cheapest DL model, do you have references for them? A group of people at fast.ai trained an ImageNet model for 26$, presumably after spending a couple hundered on getting everything just right: http://www.fast.ai/2018/04/30/dawnbench-fastai/

Re: AI winter is well on its way

#452

Post by stillsut »

bitL wrote: post #437
whatshisface wrote: post #397

Earlier quoted context omitted.

The problem could be fixed by asking doctors to put their diagnosis into the machine before the machine reveals what it thinks. Then, a simple Bayesian calculation could be performed based on the historical performance of that algorithm, all doctors, and that specific doctor, leading to a final number that would be far more accurate. All of the thinking would happen before the device polluted the doctor's cognitve bi…

There is a problem with that approach that at some point hospital management starts rating doctors by how well their diagnoses match those automated ones, and punish those who deviate too much, removing any incentives to be better/different. I wouldn't underestimate this, dysfunctional management exhibits these traits in almost any mature business.

No, it's a "second opinion", and the human doctors are graded with how well their own take differs with the computer's advice, when the computer's advice is different from the ground truth.

And there's probably not even a boolean "ground truth" in complicated bio-medicine problems. Sometimes the right call is neither yes or no, but: this is not like anything I've seen before, I can't give a decision either way, I need further tests.

Re: AI winter is well on its way

#453

Post by bitL »

timr wrote: post #449
bitL wrote: post #429

Earlier quoted context omitted.

> Unless my doctor is exactly average Just to get back to this point: what if the vision system of your doctor is below average and you augment her by giving her a statistically better vision system, while allowing her to use the additional sources as she sees fit. Wouldn't be that an improvement? We are talking about vision subsystem here, not the whole "reasoning package" human doctors posses.

Again, check that table. It says a lot: https://stanfordmlgroup.github.io/competitions/mura/ On just about every test set, the model is beaten by radiologists. Even the mean performance is underwhelming.

I was referring mainly to this one (from the same group and it actually surpassed humans on average):

https://stanfordmlgroup.github.io/projects/chexnet/

In their paper they even used "weaker" DenseNet-121 instead of DenseNet-169 for Mura/bones. DenseNet-BC I tried is another refinement of the same approach.

Re: AI winter is well on its way

#454

Post by eli_gottlieb »

digitalzombie wrote: post #146
imh wrote: post #40

FYI This post is about deep learning. It could be the case that neural networks stop getting so much hype soon, but the biggest driver of the current "AI" (ugh I hate the term) boom is the fact that everything happens on computers now, and that isn't changing any time soon. We log everything and are even starting to automate decisions. Statistics, machine learning, and econometrics are booming fields. To talk about t…

Bayesian can be seen as a subset of deep learning or hell a superset. AI is a superset and Machine learning is a subset of AI and most funding is in deep learning. Once Deep Learning hit the limit I believe there will be an AI winter. Maybe there will be hype around statistic (cross fingers) which will lead to Bayesian and such.

>Bayesian can be seen as a subset of deep learning or hell a superset.

eh-hem

DIE, HERETIC!

eh-hem

Ok, with that out of my system, no, Bayesian methods are definitely not a subset of deep learning, in any way. Hierarchical Bayes could be labeled "deep Bayesian methods" if we're marketing jerks, but Bayesian methods mostly do not involve neural networks with >3 hidden layers. It's just a different paradigm of statistics.

Re: AI winter is well on its way

#455

Post by sh33mp »

timr wrote: post #447
sh33mp wrote: post #441

Earlier quoted context omitted.

This thread is a microcosm of this whole issue of overhyping. On one hand, we have one commenter saying he can train a model to do a specific thing with a specific quantitative metric, to demonstrate how deep learning can incredibly powerful/useful. On the other hand, we have another commenter saying "But this won't replace my doctor!" and therefore deep learning is overhyped. The two sides aren't even talking about…

Agree that the thread is a microcosm of the debate, but ironically, I'm not trying to say anything like "this won't replace my doctor" . That kind of hyperventilating stuff is easy to brush off. The problem with deep-learning hype is that comments like "my classifier gets a ROC/AUC score of 0.8 with barely any work!" are presented as meaningful. The difference between a 0.8 AUC and a usable medical technology means t…

Agreed. I think it comes down to the presentation/interpretation of results. The response to "My classifier gets score of X" can be either "wow, that's a good score for a classifier, this method has merit" or "but X is not a good measure of [actual objective]".

So I think it's come down to conflict between

1. Which the author is trying to present 2. What an astute reader might interpret it as 3. What an astute reader might worry an uninformed reader might interpret it as

And my feeling is that, given all the talk about hype in pop-sci, we're actually on point 3 now, even when the author and reader are actually talking about something reasonable. Whereas personally I'm more interested in the research and interpretations from experts, which I find tend to be not so problematic.

Re: AI winter is well on its way

#456

Post by badminton1 »

mastrsushi wrote: post #136
badminton1 wrote: post #133

Earlier quoted context omitted.

The history of humanity is full of examples of people using technology to prevail over other groups of people. Applications of AI and ML will be no exception: computer vision, game theory, autonomous systems, material design, espionage, cryptography... You name it. Supraintelligent AI is not required to cause severe problems.

I'm sure they have, but none of those fields have been exploited for marketability, at least nowhere near this degree.

Not sure what you mean, but for the most part AI and ML is being integrated into systems where there's too much data to be processed manually... like satellite/aerial imagery, tapped internet backbones, security cameras, etc.

Re: AI winter is well on its way

#457

Post by ujal »

sqrt17 wrote: post #451
ujal wrote: post #389

Earlier quoted context omitted.

I don't get the sentiment of the article either. I can't speak for researchers but software engineers are living through very exciting times. State of the art in numbers: Image Classification - ~$55, 9hrs (ImageNet) Object Detection - ~$40, 6hrs (COCO) Machine Translation - ~$40, 6hrs (WMT '14 EN-DE) Question Answering - ~$5, 0.8hrs (SQuAD) Speech recognition - ~$90, 13hrs (LibriSpeech) Language Modeling - ~$490, 74h…

If the dollar amounts refer to the training cost for the cheapest DL model, do you have references for them? A group of people at fast.ai trained an ImageNet model for 26$, presumably after spending a couple hundered on getting everything just right: http://www.fast.ai/2018/04/30/dawnbench-fastai/

Thats what you get with Google TPUs on reference models. The ImageNet numbers are from RiseML, the rest is from here - https://youtu.be/zEOtG-ChmZE?t=1079

Re: AI winter is well on its way

#458

Post by jimothywales »

scardine wrote: post #407
jimothywales wrote: post #205

Earlier quoted context omitted.

That's not entirely correct. People repeatedly like to take hard yes/no stances on blockchain because they seem to confuse the issue with cryptocurrencies. Cryptocurrencies are quite obviously overhyped and self-destructing. But blockchain is actually being implemented in many effective ways around the world - just in areas that you wouldn't generally hear about. Blockchain has its place, just not at the current mome…

I don't know any problem where blockchains are the best solution - it is possible there are some, I don't know. I'm just talking about the very inflated social/political expectations, the "blockchain will change everything" mantra: I don't see this happening.

How can we know if a solution is effective if we have not studied it thoroughly enough? There might be use cases for blockchain that have yet to be probed. It's certainly being taken seriously in the finance industry, especially in terms of financial records-keeping and accounting for governance by regulatory bodies.

Re: AI winter is well on its way

#459

Post by meh2frdf »

dx034 wrote: post #296
Jach wrote: post #139

Earlier quoted context omitted.

You can finish a CS undergrad without taking any AI course? Or just haven't taken one yet? It's very helpful to go through even a tiny bit of AI: A Modern Approach to cut through a lot of the hype. What annoys me is that when people say "Machine Learning" these days they almost invariably mean deep learning, ignoring all the rest of AI. > I can't even skim through the news without hearing Elon Musk and Steven Hawking…

> What annoys me is that when people say "Machine Learning" these days they almost invariably mean deep learning, ignoring all the rest of AI. But that's not people's fault. Companies only say AI if they mean deep learning. I've yet to hear a company advertising AI if they accomplished it with a linear regression. Maybe experts should stop talking about AI and use specific terms instead (Deep Learning in the case of…

I hate it when people say deep learning instead of neural networks.

Re: AI winter is well on its way

#460

Post by pascalxus »

But, they're getting more and more data every year, right? All those almost millions of teslas running around could provide enough video input for the training data

Besides "Good software takes 10 years", according to Joel Spolsky. As I see it, we're, what 5 year into ML.

Post reply on HN