Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

One model to learn them all

blog.acolyer.org

Post reply on HN
Print view View original

41–50 of 51 posts

Re: One model to learn them all

#41

Post by npatrick04 »

YeGoblynQueenne wrote: post #32

For me, as a non-practitioner (I'm happy with my GOFAI, thank you), the big problem with neural nets is that there are too many architectures, each tailored to a specific problem. On the one hand it's great that there's a broad toolset, on the other hand there are so many competing claims about best-of-class performance that it's hard to know what is even the state of the art. There is too much noise, you know? So it…

I get your frustration with architecture of architecture. However this kind of AOA has been going on for a long time in a similar domain of control theory.

Model Predictive Control is a method of applying a model of a process to estimate it's state. Per Wikipedia, it's been used since the 80s. When you apply it to tracking something you aren't in control of, say an enemy plane, you need to use multiple models, and then pick the most probable solution.

It's not elegant, but it's not actually a bad way to get good results.

Re: One model to learn them all

#42

Post by zo1 »

mod wrote: post #40
larister wrote: post #37

Earlier quoted context omitted.

True in normal conditions, but our neocortex is highly adaptable. I can’t remember the studies, but there have been cases where a grid of actuators was taped to the body of a blind man and hooked up to a camera mounted on his head. In time, he was able to “see” edges based on the signals coming from his skin.

That's amazing. I guess probably the orientation didn't even matter. So likely, you could seamlessly wear something like that on a patch of skin that stayed out of your way (on your back, perhaps).

It's actually a little tab that the blind person puts on their tongue. Link below.

https://www.popsci.com/fda-allows-marketing-device-lets-blin...

Re: One model to learn them all

#45

Post by leoc »

zo1 wrote: post #42
mod wrote: post #40

Earlier quoted context omitted.

That's amazing. I guess probably the orientation didn't even matter. So likely, you could seamlessly wear something like that on a patch of skin that stayed out of your way (on your back, perhaps).

It's actually a little tab that the blind person puts on their tongue. Link below. https://www.popsci.com/fda-allows-marketing-device-lets-blin...

That's the current model, the Brainport. The original was indeed a chair, back in the late '60s, and there were several users. https://en.wikipedia.org/wiki/Paul_Bach-y-Rita

Re: One model to learn them all

#46

Post by erikpukinskis »

joshmarlow wrote: post #24
erikpukinskis wrote: post #21

Legit question: if human intelligence is fundamentally limited by scale, why aren’t there some humans running around with heads twice as big and twice the neurons? If it’s such an advantage why hasn’t nature selected for it in at least one place on Earth? (My answer: above the scale of the human brain there’s diminishing returns on more neurons. Two individuals with their own volition are smarter than one individual…

Here's my understanding/interpretation - if I get any details wrong, I hope someone will correct me! Brains are very expensive and, for our body size, we have very large brains; evolution had to build a brain that could operate within the calorie constraints imposed by our ecological niche - presumably a hunter-gatherer niche. If you graph adult body volume relative to gestation time across the placental mammals, the…

The question isn't whether something could exist that's smarter than every human

The question is whether something could exist that's smarter than every group of humans is

Humans are social, so the social group, and in fact groups of social groups are the highest form of intelligence on the planet. That's the bar to beat, not an individual Homo Sapiens in a observation room.

Re: One model to learn them all

#47

Post by erikpukinskis »

omalleyt wrote: post #26
erikpukinskis wrote: post #21

Legit question: if human intelligence is fundamentally limited by scale, why aren’t there some humans running around with heads twice as big and twice the neurons? If it’s such an advantage why hasn’t nature selected for it in at least one place on Earth? (My answer: above the scale of the human brain there’s diminishing returns on more neurons. Two individuals with their own volition are smarter than one individual…

I think there's a simple explanation for this. Increased head size == greater risk of death in childbirth Female hips have already evolved to be as large as possible while still being load-bearing. It would be interesting to see what the wide-spread use of Caesarean sections would do to the human brain, if we weren't all going to be transhuman within a few generations anyway As far as whether there are diminishing re…

I'm sure there have been individuals throughout time who had normal heads at birth, but whose heads grew more in childhood.

Re: One model to learn them all

#48

Post by tripzilch »

andreyk wrote: post #12
maxpupmax wrote: post #2

Wow. Can someone pull a Hacker News and explain to me why I'm allowed to be super pessimistic about this result? I want to believe.

No need to be super pessimistic, but there is reason to temper your excitement - this is quite preliminary research (little results to indicate much benefit to this - "But the results show that even on the ImageNet task, the presence of such blocks does not detract from performance, and may even slightly improve it."), and is still entirely supervised. Something like this able to learn in a semi-supervised fashion fr…

To be fair, their ImageNet results were 86% versus 95% in state of the art. In ML, those last few percentages, the closer you get to 100% become exceedingly harder to beat.

Re: One model to learn them all

#49

Post by hndamien »

mod wrote: post #40
larister wrote: post #37

Earlier quoted context omitted.

True in normal conditions, but our neocortex is highly adaptable. I can’t remember the studies, but there have been cases where a grid of actuators was taped to the body of a blind man and hooked up to a camera mounted on his head. In time, he was able to “see” edges based on the signals coming from his skin.

That's amazing. I guess probably the orientation didn't even matter. So likely, you could seamlessly wear something like that on a patch of skin that stayed out of your way (on your back, perhaps).

This is essentially how cochlear implants work. Just get a usable signal to the brain (early in life) and let the brain sort out what to do with the information.

Re: One model to learn them all

#50

Post by joshmarlow »

tormeh wrote: post #36
joshmarlow wrote: post #24

Earlier quoted context omitted.

Here's my understanding/interpretation - if I get any details wrong, I hope someone will correct me! Brains are very expensive and, for our body size, we have very large brains; evolution had to build a brain that could operate within the calorie constraints imposed by our ecological niche - presumably a hunter-gatherer niche. If you graph adult body volume relative to gestation time across the placental mammals, the…

Very interesting! I happen to know of some other problems: Heat and cross-brain communication time. Bird brains are much smarter than we are per weight and volume. Not a little bit, but a lot. I think part of the reason they can do that is that they have much better cooling because their heads are small. Having a bigger brain than we do means that it would need to have a lower activity level because of cooling constr…

I read once that bird brains (or some part of them) are more modular and consist of densely connected components - any idea/references about that?

Also, this blog-post (and associated links) may interest you - http://www.rifters.com/crawl/?p=6116

Post reply on HN