Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Path-breaking Papers About Image Classification

blog.paralleldots.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Path-breaking Papers About Image Classification

#11

Post by falcolas »

Some really cool information, but this concluding bit annoyed me:

> By Moore’s law, we will reach computing power of human brain by 2025 and all of the humanity by 2050.

Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade. ML improvements now depend upon better algorithms to make them more parallel, and the economies of scale which make more parallel computation units available. I don't think we're anywhere near that exponential graph, however, and we'll keep getting further from it.

Perhaps quantum computing will become a widespread reality and blow the field open, but I'm not holding my breath that it will happen in the next few decades.

Re: Path-breaking Papers About Image Classification

#12

Post by londons_explore »

falcolas wrote: post #11

Some really cool information, but this concluding bit annoyed me: > By Moore’s law, we will reach computing power of human brain by 2025 and all of the humanity by 2050. Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade. ML improvements now depend upon better algorithms to make them more parallel, and t…

Moore’s law is still alive and well, you just have to move over to the parallel architectures like GPU's.

Considering machine learning is all on GPU's and TPU's now, I think this is still a fair assessment.

Re: Path-breaking Papers About Image Classification

#13

Post by AndrewOMartin »

The caption for the top graph appears a bit out of whack.

It states "exponential decline in top 5 error rate", the decline looks more like diminishing returns to me, especially if you push the 2017 data point out to where it should be (they've omitted 2016).

It's nice that the error rate is low, but the caption appears to oversell it.

This graph reminds me of a very closely related one I saw in a talk a few years ago [1]. It was showing decline in voice recognition error rates over time, with a highlighted band for "human performance".

The speaker, Roger Moore (the academic, not the actor, and not the Moore with the law), pointed out that this line, while encouraging, hid two important points.

1) For linear improvement, exponentially more training data was needed. 2) No insight into how living beings solve the same task.

These aren't necessarily fatal flaws, but they're worth remembering.

[1] https://www.youtube.com/watch?v=iYbVsvxd3bE

Re: Path-breaking Papers About Image Classification

#16

Post by sanxiyn »

lalp2119 wrote: post #7

It would be great if you can share the links to pretrained weights if the networks mentione here in python framework.

Here are some. They all have pretrained weight download.

ResNet: https://github.com/KaimingHe/deep-residual-networks Wide ResNets: https://github.com/szagoruyko/wide-residual-networks ResNeXt: https://github.com/facebookresearch/ResNeXt DenseNet: https://github.com/liuzhuang13/DenseNet

Re: Path-breaking Papers About Image Classification

#17

Post by kayoone »

falcolas wrote: post #11

Some really cool information, but this concluding bit annoyed me: > By Moore’s law, we will reach computing power of human brain by 2025 and all of the humanity by 2050. Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade. ML improvements now depend upon better algorithms to make them more parallel, and t…

GPUs still improve a lot year over year, so i think Moore's law still holds true for at least a few more years.

Re: Path-breaking Papers About Image Classification

#18

Post by zitterbewegung »

A more accurate idea of what a computer sees is actually that ML models figure out what parts of the signal to throw away and pay attention to. This is why you can slightly perturb the image so that humans see a picture of two hot dogs while an ML model can be confused into two different things (hot dog and an egg plant).

Re: Path-breaking Papers About Image Classification

#19

Post by falcolas »

londons_explore wrote: post #12
falcolas wrote: post #11

Some really cool information, but this concluding bit annoyed me: > By Moore’s law, we will reach computing power of human brain by 2025 and all of the humanity by 2050. Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade. ML improvements now depend upon better algorithms to make them more parallel, and t…

Moore’s law is still alive and well, you just have to move over to the parallel architectures like GPU's. Considering machine learning is all on GPU's and TPU's now, I think this is still a fair assessment.

Not really - since GPUs have started to hit the same "Process" size as CPUs - and haven't been showing a lot of growth in that area. The best improvement per Wikipedia's chart is a small foray into 14 and 12nm, and those haven't doubled the transistor counts (per square mm).

What we are seeing is an increase in die sizes; more parallel cores. Parallel cores still require parallel algorithms, so I stand by my earlier statement.

Re: Path-breaking Papers About Image Classification

#20

Post by scj »

falcolas wrote: post #11

Some really cool information, but this concluding bit annoyed me: > By Moore’s law, we will reach computing power of human brain by 2025 and all of the humanity by 2050. Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade. ML improvements now depend upon better algorithms to make them more parallel, and t…

> Their graph does show exponential growth, but the data points cut off at the year 2000. Not surprising, given that Moore's law has reached its end in the last decade.

I think the graph was originally produced for Ray Kurzweil's 1999 book "The Age of Spiritual Machines".

Post reply on HN