Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI winter is well on its way

blog.piekniewski.info

Post reply on HN
Print view View original

351–360 of 518 posts

Re: AI winter is well on its way

#351

Post by zerostar07 »

jdonaldson wrote: post #345
zerostar07 wrote: post #335

Earlier quoted context omitted.

Neuroscience has been in constant winter, mainly because the methods are too crude and small scale, and that by observing 100 neurons out of billions makes it impossible to tell the whole story. The deep mind papers are interesting, but they are just a start imho. It may be that they are focusing on a mere coincidence. Nevertheless it is exciting to see progress in that direction, thats why i don't think DL research…

I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain ac…

I 'm actually of the opposite idea - let the two fields evolve by their own darwinian process as this will yield more interesting results. DL itself has created its own scientific questions and puzzles that may lead to important discoveries (which may transfer to neuroscience). E.g. "why does batch normalization / dropout work."

Re: AI winter is well on its way

#353

Post by xamuel »

dx034 wrote: post #289
imh wrote: post #40

FYI This post is about deep learning. It could be the case that neural networks stop getting so much hype soon, but the biggest driver of the current "AI" (ugh I hate the term) boom is the fact that everything happens on computers now, and that isn't changing any time soon. We log everything and are even starting to automate decisions. Statistics, machine learning, and econometrics are booming fields. To talk about t…

I think the problem is the definition of AI. It appears most in the field define it as a superset of ML, encompassing all kinds of statistical methods and data analysis. For the general public, AI is a synonym for deep learning. When large companies speak about AI they always mean deep learning, never just a regression (probably also because many don't see a regression as intelligent). So AI in the public's perceptio…

>For the general public, AI is a synonym for deep learning

I'd contend for the general public, AI is a synonym for machines like: HAL; The Terminator; Star Trek's "Data"; the robots in the film "AI"; and so on.

We're nowhere remotely in the vicinity of that, and no-one even has any plausible ideas about how to start.

A random person outside of tech probably doesn't even know what deep learning is. They might have heard of it somewhere in passing.

Re: AI winter is well on its way

#354

Post by lispm »

fvdessen wrote: post #339
lispm wrote: post #276

Earlier quoted context omitted.

Google Translate of Goethe: Have now, ah! Philosophy, Law and medicine, And unfortunately also theology Thoroughly studied, with great effort. Here I am, I poor gate! And I'm as smart as before word for word not completely bad, but then it breaks when we have to translate 'Tor'. Google Translate is clueless because it is unable to derive that here the 'fool' is meant. It's unable to 'understand' that 'I poor gate' ma…

On the other hand Deepl gives translations for news articles that are of such quality that it allows me to read international news as if it were local. Definitely useful.

DeepL is better, but it basically has the same problems. I understand both German and English - and I can easily detect where DeepL also shifts the meaning of sentences, sometimes even to mean the opposite. DeepL like Google Translate has no concept for 'is the meaning preserved?'.

You may think that you can now read German news, but in fact you would not know if the sentence meaning has been preserved in the English translation. The words itself might look as if the sentence makes sense - but the meaning is actually shifted - slight differences, but also possibly the complete opposite.

The translation also does not give you any indication where this might be and where the translation is based on weak training material or where there is some inference needed for a successful translation.

Re: AI winter is well on its way

#355

Post by evgen »

kokey wrote: post #272
PeterisP wrote: post #159

Earlier quoted context omitted.

The previous AI winters were funded by speculative investments (both public research and industry) with the expectation that this might result in profitable technologies. And this didn't happen - yes, "the other previous AI which experience AI Winter (expert system, and whatever else) didn't produce useful enough technologies to have funding", the technologies developed didn't work sufficiently well to have widesprea…

I've recently had a number of calls from recruiters about new startups in the UK in the AI space, some of them local and some of them extensions of US companies. Some of them were clearly less speculative (tracking shipping and footfall for hedge funds) while others were certainly more speculative sounding. The increase of the latter gives me the impression that there is a bit of speculation going on at the moment.

A lot of this is because there is a somewhat mis-informed (which we will be polite and not call 'gullible') class of investors out there, primarily in the VC world, that thinks that most AI is magic pixie dust and so 'we will use AI/DL' and 'we will do it on the blockchain' has become the most recent version of 'we will do it on the web' in terms of helping get funding. Most of these ventures will flame out in 6-12 months and the consequences of this are going to be the source of the upcoming AI winter OP was talking about.

Re: AI winter is well on its way

#356

Post by eitland »

AstralStorm wrote: post #224
jcelerier wrote: post #213

Earlier quoted context omitted.

> What decision-making problems are there, in the same class, that I'd pay for? an awful lot of graph and optimization problems. See for instance some examples in https://en.wikipedia.org/wiki/A*_search_algorithm

Perfect information problem solving is not interesting anymore. Did they manage to extend it to games with hidden and imperfect information? (Say, chess with fog of war also known as Dark Chess. Phantom Go. Pathfinding equivalent would be an incremental search.) Edit: I see they are working on it, predictive state memory paper (MERLIN) is promising but not there yet.

> Perfect information problem solving is not interesting anymore.

I guess in the same way as lab chemistry isn't interesting anymore ? (Since it often happens in unrealistically clean equipment :-)

I think there is nothing preventing lab research from going on at the same time as industrialization of yesterday's results. Quite on the contrary: in the long run they often depend on each other.

Re: AI winter is well on its way

#357

Post by imant »

bitL wrote: post #348

I was recently "playing" with some radiology data. I had no chance to identify diagnoses myself with untrained eyes, something that probably takes years for a decent radiologist to master. Just by using DenseNet-BC-100-12 I ended up with 83% ROC AUC after a few hours of training. In 4 out of 12 categories this classifier beat best human performing radiologists. Now the very same model with no other change than adjust…

Would you mind sharing which other, unrelated dataset you have used the model on?

Re: AI winter is well on its way

#358

Post by mtgx »

dekhn wrote: post #324

I'm a scientist from a field outside ML who knows that ML can contribute to science. But I'm also really sad to see false claims in papers. For example, a good scientist can read an ML paper, see claims of 99% accuracy, and then probe further to figure out what the claims really mean. I do that a lot, and I find that accuracy inflation and careless mismanagement of data mars most "sexy" ML papers. To me, that's what'…

You hear Facebook all the time saying how it "automatically blocks 99% of the terrorist content" with AI to the public and governments.

Nobody thought to ask: "How do you know all of that content is terrorist content? Does anyone check every video afterwards to ensure that all the blocked content was indeed terrorist content?" (assuming they even have an exact definition for it).

Re: AI winter is well on its way

#359

Post by [deleted] »

bitL wrote: post #348

I was recently "playing" with some radiology data. I had no chance to identify diagnoses myself with untrained eyes, something that probably takes years for a decent radiologist to master. Just by using DenseNet-BC-100-12 I ended up with 83% ROC AUC after a few hours of training. In 4 out of 12 categories this classifier beat best human performing radiologists. Now the very same model with no other change than adjust…

[deleted]

Re: AI winter is well on its way

#360

Post by DonaldFisk »

icc97 wrote: post #337
DonaldFisk wrote: post #277

Earlier quoted context omitted.

It's Deep Blue, not Big Blue. The parameters used by its evaluation function were tuned by the system on games played by human masters. But it's a mistake to think that a system learning by playing against itself is something new. Arthur Samuel's draughts (chequers) program did that in 1959.

Sorry, mix up, thanks for the correction. It's not that it's new, it's that they've achieved it. Chess was orders of magnitude harder than draughts. The solution for draughts didn't scale to chess but Alpha Go zero showed that chess was ridiculously easy for it once it had learned Go.

Both Samuel's chequer's program and Deep Blue used alpha-beta pruning for search, and a heuristic function. Deep Blue's heuristic function was necessarily more complex because chess is more complex than draughts. I think the reason master chess games were used in Deep Blue instead of self-play was the existence of a large database of such games, and because so much of its performance was the result of being able to look ahead so far.
Post reply on HN