Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Past Performance is Not Indicative of Future Results (2020)

locusmag.com

Post reply on HN
Print view View original

191–200 of 285 posts

Re: Past Performance is Not Indicative of Future Results (2020)

#191

Post by roenxi »

gonehome wrote: post #135
skohan wrote: post #109

Earlier quoted context omitted.

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

A good example of those constraints is there are hard upper limits to heat and energy use by a brain that are simply outdone by, say, a massive supercomputer.

A rough calculation, humans can feasibly consume 4-5 TJ/annum of energy, of which a lot is going to go into motion or whatever. And if devoted to mental activity, it has a shelf life of ~70 years before they die.

A distributed computer might theoretically burn TJ/hr and once the weights are known they may as well be in a permanent record. The upper limits of what computers can learn and get good at are much higher than what humans can. They won't need as much implementation trickery as biology to get results.

Re: Past Performance is Not Indicative of Future Results (2020)

#192

Post by cratermoon »

pbhjpbhj wrote: post #144
cratermoon wrote: post #116

Earlier quoted context omitted.

There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Chicago’s “Heat…

I'm curious how the physics of light is termed racial bias, it's skin-colour bias if anything -- you can be "black" and be lighter skinned than a "white" person, for example -- but surely it's a consequence of how cameras/light works rather than a bias. Of course if you don't take account of the difficulties that come with using the tool then you might be acting with racial bias, but that's different. Or, all cameras…

Well, if you really want to know, I have done the research and can recommend several other papers in addition to the one linked. The short answer is that it's not "just physics", and choices made by the chemists and technicians at Kodak, Fuji, Ilford, Agfa, etc to decide how films depicted skin tones were made with racial bias. Digital imaging built on the color rendering tools and tests that originated in the film industry, and thus inherited their flaws.

Re: Past Performance is Not Indicative of Future Results (2020)

#193

Post by giantg2 »

skohan wrote: post #109
jstx1 wrote: post #16

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI - I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. - That doesn't make me a skeptic towards the current state of machine learning thoug…

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

And there's so much we still don't know about the nervous system and cognition.

Re: Past Performance is Not Indicative of Future Results (2020)

#194

Post by cratermoon »

mgraczyk wrote: post #127
cratermoon wrote: post #116

Earlier quoted context omitted.

There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Chicago’s “Heat…

It's very easy to fix these problems though. There's nothing inherently broken about the models or direction that prevents error rates from being made more uniform. In fact newer facial recognition models with better datasets do perform approximately equally well across skin tones and sex

Easy to fix technically, but first the issue must be recognized and demonstrated, then the delicate process of negotiating the social and economic realities in which the technology operates.

And that's the problem with ML in general: its failure to recognize the implicit biases in choice of dataset and training and the resulting problems, of which Microsoft racist chatbot Tay[1] is merely the most blatantly ludicrous.

1 https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...

Re: Past Performance is Not Indicative of Future Results (2020)

#195

Post by kortilla »

marcinzm wrote: post #72
kortilla wrote: post #47

Earlier quoted context omitted.

There is general knowledge about the world for humans to know that there isn’t a giant human in the sky no matter how good the face looks. Train it with as many images as you want and as long as a good enough face shows up, the model is going to have a positive match. The entire problem is it’s missing that upper level of intelligence that evaluates “that looks like a face, could it actually be a human?”

>There is general knowledge about the world for humans to know that there isn’t a giant human in the sky no matter how good the face looks. Is there? Humans used to think the gods were literally watching them from the sky and the constellations were actual creatures sent into the night. So this seems learned behavior from data rather than some inherent part of human thinking. >Train it with as many images as you want…

Children will think there is a monster under their bed based on no evidence at all. That speaks to something beyond object recognition that happens at a different processing layer.

Humans do not need to be trained on billions of images from around the globe to semantically understand where human faces are not expected to appear. Modern AI can certainly recognize faces really well with that level of training now, but it still doesn’t even understand what a face is (i.e. no model of reality to verify its identification against).

Re: Past Performance is Not Indicative of Future Results (2020)

#196

Post by username90 »

epgui wrote: post #173

What baffles me is the number of humans who think they are in the personal possession of some super special sacred form of magical and unexplainable intelligence. "AI is just stats" yes, indeed, but so is human intelligence. In many ways, AI from 2010 was already better than human intelligence. Three remarks: - The task many people seem to be benchmarking against is not just a measure of general intelligence, but a m…

General intelligence requires it to solve real problems in the real world. It isn't about emulating humans, but emulating anything resembling an intelligent being we are aware of. It would be totally exceptional if we could properly emulate the intelligence of a fly or an ant, but we can't even do that. "Emulate a human brain" you say, but we can't even emulate brains a million times smaller than that.

Re: Past Performance is Not Indicative of Future Results (2020)

#197

Post by BurningFrog »

gonehome wrote: post #135
skohan wrote: post #109

Earlier quoted context omitted.

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> The way a plane flies is quite different than the way a bird flies in complexity

And a plane is a vastly simpler machine than a bird!

Re: Past Performance is Not Indicative of Future Results (2020)

#198

Post by username90 »

dane-pgp wrote: post #152
okareaman wrote: post #94

Earlier quoted context omitted.

I didn't say I knew how. I said I believe materialism is a dead end, by which I mean I doubt the consciousness arises out of atoms configured as neurons. How those neurons receive a conscious field seems a more productive line of inquiry, but for some reason people resist this idea. Not sure why.

By studying the atoms configured as neurons, we've managed to develop machines that can learn to play board games and Atari games better than humans, and can write prose and poetry at a convincingly human level. Those skills may not require consciousness, but it's not clear that these machines would be more useful if they could "receive a conscious field". Do you think that animals receive a conscious field? Could we…

Start that argument once we can model insect brains. Mouse brains aren't even on the horizon of what we can do.

Re: Past Performance is Not Indicative of Future Results (2020)

#199

Post by mgraczyk »

cratermoon wrote: post #194
mgraczyk wrote: post #127

Earlier quoted context omitted.

It's very easy to fix these problems though. There's nothing inherently broken about the models or direction that prevents error rates from being made more uniform. In fact newer facial recognition models with better datasets do perform approximately equally well across skin tones and sex

Easy to fix technically , but first the issue must be recognized and demonstrated, then the delicate process of negotiating the social and economic realities in which the technology operates. And that's the problem with ML in general: its failure to recognize the implicit biases in choice of dataset and training and the resulting problems, of which Microsoft racist chatbot Tay[1] is merely the most blatantly ludicrou…

And the first cars didn't have seatbelts.

It's fine, these are not complicated problems, and they are much easier to spot and fix than most problems in software engineering at scale. Don't be fooled by the negative PR campaigns and clickbait, there's no reason to be skeptical about ML in general because of this.

Also, Tay attempted to solve a much harder problem than image classification. It's hard to build a safe hyperloop. It's no longer hard to build a safe microwave oven.

Re: Past Performance is Not Indicative of Future Results (2020)

#200

Post by Nevermark »

kortilla wrote: post #180
version_five wrote: post #155

Earlier quoted context omitted.

Thanks for your reply. I suppose a quick way to summarize my criticism is that it reads to me like you've dismissed the strengths of ML on technical grounds, while you imply you don't have any real technical experience in the field. You make a superficial comparison between the compexity of biology and ML, without providing any real insight, just saying one has lots going on and the other is matrix multiplication. If…

You don’t have to be an expert in a field to recognize that the current popular approaches to something aren’t even close to getting there.

Actually you do have to be an expert to make sweeping statements with any credibility in a young field making advances every day. Huge ones and surprising ones every year.

If you can’t characterize the technical problem that creates a limitation then you are just expressing an uninformed opinion.

Even if you were an expert!

Post reply on HN