Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer scientists prove why bigger neural networks do better

quantamagazine.org

Post reply on HN
Print view View original

111–120 of 151 posts

Re: Computer scientists prove why bigger neural networks do better

#111

Post by 0ldskool »

user90349032 wrote: post #79
gwern wrote: post #75

Earlier quoted context omitted.

Density is also important. If we look at other things - some recent studies have been done on number-counting ( https://royalsocietypublishing.org/doi/10.1098/rstb.2020.052... ) or bird brains ( https://www.gwern.net/docs/psychology/neuroscience/2020-herc... ) - density jumps out as a major predictor. African elephants may have some more neurons, but the density isn't as great as a human where it counts, so they are…

And yet, no animal except humans is self aware. Really makes you wonder why that is.

Is that something that can/has been proven? From my understanding there are many other smart animals that are self aware.

Re: Computer scientists prove why bigger neural networks do better

#112

Post by tpoacher »

user90349032 wrote: post #79
gwern wrote: post #75

Earlier quoted context omitted.

Density is also important. If we look at other things - some recent studies have been done on number-counting ( https://royalsocietypublishing.org/doi/10.1098/rstb.2020.052... ) or bird brains ( https://www.gwern.net/docs/psychology/neuroscience/2020-herc... ) - density jumps out as a major predictor. African elephants may have some more neurons, but the density isn't as great as a human where it counts, so they are…

And yet, no animal except humans is self aware. Really makes you wonder why that is.

depends on your definition of self aware.

Most cats and dogs I've seen correctly recognise themselves in the mirror after the novelty of seeing one for the first time wears off.

Re: Computer scientists prove why bigger neural networks do better

#113

Post by 988747 »

moomin wrote: post #82
user90349032 wrote: post #79

Earlier quoted context omitted.

And yet, no animal except humans is self aware. Really makes you wonder why that is.

Probably that you don’t know how to measure what you’re describing. Plenty of animals recognise themselves in the mirror, for instance.

I wouldn't say "plenty" - few primates, dolphins, orcas, elephants, and, strangely, magpies. But the grounds for that claim are shaky for some of them, the only 3 species we are 100% sure about are chimpanzees, orangutans, and humans. Magpies, for example, require "a training" (whatever that means).

Re: Computer scientists prove why bigger neural networks do better

#114

Post by csomar »

pishpash wrote: post #68
cloogshicer wrote: post #37

Earlier quoted context omitted.

You probably already know this (since you wrote "silly thought"), but real-life neurons are ridiculously more complex than simulated "neurons" in an NN. So the analogy doesn't really hold.

They're more complex in biological construction and in signaling mechanism, but no proof that they are more complex in function.

Do you think they have accumulated complexity for no reason. Because usually evolution kicks out things that aren't needed.

Re: Computer scientists prove why bigger neural networks do better

#118

Post by heavenlyblue »

mattkrause wrote: post #86
gwern wrote: post #65

Earlier quoted context omitted.

Volume != neurons. In any case, 75 is awful and is usually considered borderline retarded. (If you're tempted to respond with other cases of higher IQ, note that they are often retracted or unconfirmed and likely fraudulent in some way; see https://www.gwern.net/Hydrocephalus .)

As a counterpoint, rats without a cortex can do...basically everything normal rats can do--except trim their toenails. The classic reference for this is Whitslaw's 1990 chapter "The decorticate rat". This thread has links to a copy, plus a bunch of related studies in humans and animals. https://twitter.com/markdhumphries/status/107105276276554137...

Have you read the PDF given?

The whole nail part is basically a single sentence in the paper.

For example, decorticate rats are unable to escape narrow alleyways because they can not turn around due to their tonsils touching the walls and them being unable to ignore that feeling.

Another example is that they take a few seconds vs (!) 5 minutes to groom themselves on average.

Re: Computer scientists prove why bigger neural networks do better

#119

Post by rjzzleep »

988747 wrote: post #113
moomin wrote: post #82

Earlier quoted context omitted.

Probably that you don’t know how to measure what you’re describing. Plenty of animals recognise themselves in the mirror, for instance.

I wouldn't say "plenty" - few primates, dolphins, orcas, elephants, and, strangely, magpies. But the grounds for that claim are shaky for some of them, the only 3 species we are 100% sure about are chimpanzees, orangutans, and humans. Magpies, for example, require "a training" (whatever that means).

Probably cause we've only tested a few, not that it matters though. Humans take a pretty long time to recognize themselves in the mirror. I wonder if the mirror test would change if we would expose the animals for a almost a year before doing the test, just like humans.

That said even ants pass the test, i.e. they were recently(2015) tested.

But the whole thing can be characterized as: "Let me make up a random test, according to my personal opinion of what defines cognition and then see if a random animal I choose passes it".

Every couple of years we have requests of slews of psychology papers requested to be invalidated because they're unreproducible.

Re: Computer scientists prove why bigger neural networks do better

#120

Post by joshvm »

yalogin wrote: post #115

Can someone ELI5 how one increases the size of an NN? If I take the handwritten digit classifier that people use as ML 101, is it just a matter of increasing the size of the hidden layers?

EfficientNet is a nice simple example of this. It's family of networks with different sizes, controlled by a single scaling parameter. That parameter controls depth, width and resolution - see here for more details, the figure is easy to understand: https://paperswithcode.com/method/efficientnet
Post reply on HN