Live data from Hacker News

Past Performance is Not Indicative of Future Results (2020)

locusmag.com

221–230 of 285 posts

Re: Past Performance is Not Indicative of Future Results (2020)

#221
post #153

Earlier quoted context omitted.

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

self-driving cars are available to consumers now. Search for FSD on youtube and see all the consumers using their self driving cars. Or, watch the latest Veritasium https://www.youtube.com/watch?v=yjztvddhZmI

And yet all of them force you to keep your eyes on the road at all times or you can die. Can you honestly call that FSD?

Re: Past Performance is Not Indicative of Future Results (2020)

#222
post #173

What baffles me is the number of humans who think they are in the personal possession of some super special sacred form of magical and unexplainable intelligence. "AI is just stats" yes, indeed, but so is human intelligence. In many ways, AI from 2010 was already better than human intelligence. Three remarks: - The task many people seem to be benchmarking against is not just a measure of general intelligence, but a m…

"We have a very selective memory indeed. We have absolutely terrible judgment, are super irrational, and pretty reliably make decisions that are against our own interests, "

This is a really bad argument - human intelligence is not highly rational, but it is deeply nuanced, using social cues, emotions, instincts and a miriad of other things.

Computers can never be anti-knowledge because they lack the free will and social behavior of humans - they didn't chose to be pro knowledge either.

Re: Past Performance is Not Indicative of Future Results (2020)

#223
post #109
post #16

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI - I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. - That doesn't make me a skeptic towards the current state of machine learning thoug…

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I think most of the complexity of biology is accidental, not essential. Eg. why don’t we have a normal abstraction for sending signals? Instead, we have like 10s of slightly different ones with different failings each, but each having many repetitive machinery leading to inefficient “spaghetti code”.

And while our brain is objectively very impressive, I don’t see how our complex abilities are anything but emerging features.

Re: Past Performance is Not Indicative of Future Results (2020)

#224
post #153

Earlier quoted context omitted.

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

We used to think that machines would be bad at arithmetic and pure logical reasoning, and good at the more primitive animalistic ones, but it turns out the latter is a much harder problem.

Also, self-driving cars were mostly hyped up by companies, FSD is quite obviously a hard problem, much closer to general intelligence than the average NN application.

Re: Past Performance is Not Indicative of Future Results (2020)

#225
post #153

Earlier quoted context omitted.

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

self-driving cars are available to consumers now. Search for FSD on youtube and see all the consumers using their self driving cars. Or, watch the latest Veritasium https://www.youtube.com/watch?v=yjztvddhZmI

It’s as much FSD as my robot vacuum not hitting the wall…

These are just overhyped drive assist tools that market themselves immorally as something they aren’t.

Re: Past Performance is Not Indicative of Future Results (2020)

#226
post #109

Earlier quoted context omitted.

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> Few would have predicted the results we’re seeing today in 2010.

That's hardly accurate - didn't Musk and Co. promise self-driving cars by 2012? We're in 2020, and the SDC's are great for making youtube videos, but not any good at piloting a vehicle without human intervention.

Since the 90s it has been clear that the only thing holding back what we have today is limited processing power. While there may be some new insights and directions in AI, they are not "general" and they require 3 orders of magnitude more processing power for a lot smaller improvement in performance.

What has been clear since 2010 is that this field has passed the point of diminishing returns already. We throw vastly more computational power at problems that we ever did before, and then call the result an improvement.

Deep blue beat the best human at chess using 11.8 GFLOPS of computational power. Alphago beat the best human at go using 720000 GFLOPS of power. The complexity difference between Chess and Go are within a single order of magnitude - 10x to 99x difference in complexity (https://en.wikipedia.org/wiki/Game_complexity). The difference in AI processing power to beat the best human between Chess and Go is between 4 and 5 orders of magnitude (1000x and 100000x).

This does not look like a success to me - it looks like a brute-force approach. If you spend 10000x more resources for a 10x more benefit, you're at the point of diminishing returns.

Here's a great paper that should be written (but won't be) - plot the improvements in AI and the usage of computational power for AI on the same chart.

From the 90s (https://en.wikipedia.org/wiki/History_of_self-driving_cars#1...): "The robot achieved speeds exceeding 109 miles per hour (175 km/h) on the German Autobahn, with a mean time between human interventions of 5.6 miles (9.0 km), or 95% autonomous driving."

Yup, 95% autonomous. Today we have 95.x% autonomous with roughly 10000x the resource power thrown at the problem.

So, yeah, your assertion that "Few would have predicted the results we’re seeing today in 2010." is wildly off mark, we predicted more than what we see today because we did not expect to hit a point of diminishing returns quite so quickly.

The people who did the 95% SDC in 1997 would have been disbelieving if anyone told them, in 1997, that even with 10000x more processing power thrown at the problem and new sensor hardware that was not available to them, it won't get much better than what they had.

Re: Past Performance is Not Indicative of Future Results (2020)

#227

Earlier quoted context omitted.

You don’t have to be an expert in a field to recognize that the current popular approaches to something aren’t even close to getting there.

Actually you do have to be an expert to make sweeping statements with any credibility in a young field making advances every day. Huge ones and surprising ones every year. If you can’t characterize the technical problem that creates a limitation then you are just expressing an uninformed opinion. Even if you were an expert!

Not to get into the rest of the discussion, but I disagree with the classification of ML as a young field. AI is an established field and I would argue that nothing in modern ML is _fundamentally_ so different that it would justify classifying it as a new field.

Re: Past Performance is Not Indicative of Future Results (2020)

#228
"He quit high school,[8][verification needed] received his Ontario Academic Credit (high school diploma) from the SEED School in Toronto,[citation needed] and attended four universities without obtaining a degree"

Shouldn't we collectively listen to experts,PhDs etc instead of famous bloggers for very technical stuff? Much like in medicine I would say.

Re: Past Performance is Not Indicative of Future Results (2020)

#229

Earlier quoted context omitted.

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

> they share an underlying mechanism, but planes don't need to flap wings. A lot of birds don't need to flap their wings either.

Not if we mount a jet engine to their backs.

Re: Past Performance is Not Indicative of Future Results (2020)

#230
post #153

Earlier quoted context omitted.

> It’s hard to predict timelines for this kind of thing, and people are notoriously bad at it. Few would have predicted the results we’re seeing today in 2010. What would you expect to see in the years leading up to AGI? Does what we’re seeing look like failure? Few have predicted a reasonably-capable text-writing engine or automatic video face replacement, but many have predicted self-driving cars would have been re…

self-driving cars are available to consumers now. Search for FSD on youtube and see all the consumers using their self driving cars. Or, watch the latest Veritasium https://www.youtube.com/watch?v=yjztvddhZmI

If you can't go to sleep, the car is not fully self-driving
Post reply on HN