Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An understanding of AI’s limitations is starting to sink in

economist.com

Post reply on HN
Print view View original

301–310 of 403 posts

Re: An understanding of AI’s limitations is starting to sink in

#301

Post by snarf21 »

dreamcompiler wrote: post #216

Here we are in 1989 again. The cycle keeps repeating. A new advancement in computing power, networking, or algorithms means there's a new batch of low-hanging fruit for AI to pick, so we pick it. Investors say "What about the high-hanging fruit?" and we say "No problem. We just need a slightly longer ladder." Two years later everybody finally realizes the high-hanging fruit is on the moon.

It seems clear that were limited by access to data and computer power. That has largely changed. However, the hard part has always been around deciding what question to ask and how to score the result. This requires domain knowledge and a scientific process (conjecture, measure, analyze, repeat). There is no magic bullet just better tools. Someone still has to define the problem and solution. Just look at IBM Watson and what a failure that is. They did okay on Jeopardy because they spent years optimizing for just that. Jeopardy could have changed the way they wrote clues by including irony and other language constructs and the humans would have done fine and Watson would have bombed.

Re: An understanding of AI’s limitations is starting to sink in

#302

Post by saberience »

thu2111 wrote: post #254
mellosouls wrote: post #17

Earlier quoted context omitted.

There has been next to ZERO progress towards genuine AGI despite a never-ending deluge of AI articles; that's normally the cause of scepticism. After several decades and a much-hyped last few years we have fake cleverness - impressively so in both cases - but nothing more.

Disagree. GPT-2/GPT-3 are able to pass for humans when the reader isn't paying close attention. This article seems insightful to me: http://www.overcomingbias.com/2017/03/better-babblers.html A scarily large amount of human speech is essentially word prediction, especially in cases where someone wants to seem impressive without having actually done the work. We're all familiar with the problem of people "bullshitting…

We've been able to produce that kind of speech before though. I wrote programs using Markov Chains 10 years ago which could produce almost real sounding speech but I didn't think it was intelligent.

The GPT models are essentially massive databases along with the use of probability models for how words are connected. Do you think that means intelligence? Can it generate new knowledge of things outside its database? Can it generate novel theories for things outside its understanding?

The GPT models often generate streams of text which can found word for word in its database, i.e. it's not really generating anything new as just mimicking. When a parrot repeats a human, it doesn't understand what the words mean, it's just making sounds.

Re: An understanding of AI’s limitations is starting to sink in

#303

Post by marcosdumay »

papeda wrote: post #275
AlexandrB wrote: post #220

Earlier quoted context omitted.

> There's far more to do with ML and AI than self driving cars and shitty ad recommendations. Yeah, there's also shitty sentencing recommendations[1], new-age phrenology[2], and high-tech redlining[3]. I think your entire field needs to take a year off and take some ethics and philosophy courses before going any further. Otherwise we're all going to end up much worse off. [1] https://www.nytimes.com/2017/10/26/opinio…

I agree that plenty of people have been too cavalier about slapping together some models, predicting something, and calling it a day. On the other hand, it's not like fair sentencing or fair loan recommendation is a solved problem for humans either. There is evidence that, when carefully designed, algorithms can produce more equitable outcomes than humans, for example when deciding who and how to release on bail [1].…

Besides, the options are not restricted to "a machine decides everything" and "a human decides everything".

We can use machines to guide the humans, finding flaws or biases, recommending further analysis, and in a lot of other ways while still keeping the decisions made by humans.

Re: An understanding of AI’s limitations is starting to sink in

#304

Post by antipaul »

coverup wrote: post #288
TrackerFF wrote: post #278

Earlier quoted context omitted.

That's actually something ML is incredibly useful at, when it comes to machines with sensors - failure prediction / anomaly detection, etc. In the industry, (preventive) maintenance takes up a pretty huge chunk of resources. It's something techs need to do often, and it's often a laborious task, but it's obviously done to reduce downtime. So the business insight, as they like to call it, is to reduce costs tied up to…

Yep, you and the user you're replying to are both right in different ways. One thing's for sure - machines don't generate "insights" on their own. Let's define an "insight" as "new meaningful knowledge", just for fun. We could talk about what comprises "new" and "meaningful" but it would be beside the point I'm making. In a supervised learning problem, the range of possible outputs is already known, meaning the model…

It’s not only outputs/labels that provide “insights”.

Knowing how the outputs relate to the inputs is where most new insights could come from.

For example, what feature (input) is driving the “failure” of the machine (output/prediction)?

This is where ML explainability comes in.

Re: An understanding of AI’s limitations is starting to sink in

#305

Post by OldHand2018 »

megaframe wrote: post #77
ManuelKiessling wrote: post #23

Earlier quoted context omitted.

Ok, consider me intrigued. What is that we are going to see/experience once you folks had some time? Genuine question! Can you give us a basic idea of the things that you are already sure by now will see the light of day?

One big one is farming, think harvesting machinery. There's a number of startups trying to get that going. I think most are not at the scale to be successful the way the market themselves, but their collective learning will eventually lead to some consolidation. Why AI for that, it's vision AI to know when fruit is ripe or vegetables are ready for harvest. Then hand eye coordination to not bruise the fruit/vegetables…

> What I think is most plausible is one large manufacturer will consolidate designs and rent the equipment to do the harvesting

No way. Everybody wants the machines at the same time. You're going to incur the capital cost of buying/manufacturing the machines, the cost to store and maintain them for 50 weeks out of the year, and then rent them out at a profit for 2 weeks? No, the farmers will just keep hiring human labor.

If you can solve this problem, you can sell a lot of machines for a lot of money for a short few years. After that, they'll be a commodity and be dominated by all the other capital-intensive manufacturers that can run a factory better than you can.

Re: An understanding of AI’s limitations is starting to sink in

#306

Post by bildung »

MiroF wrote: post #295
foldr wrote: post #279

Earlier quoted context omitted.

Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.

In the past three years we have dropped error rates by a factor of 3, I don’t really think that claim holds water. We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.

Both voice recognition had first working research machines in the 1950s. Of course present models are way better than these, but fundamentally these are the same as those in the '50s, "just" with tremendously better hardware and algorithms. But there is zero intelligence in these, the models have no internal concept of language or the world around them.

This will certainly produce many great specific solutions for specific problems, but generic problems like driving a car are most probably not solvable without an internal world model.

I'm one of the naysayers and have a background in educational science, I'm constantly baffled why AI research seemingly never looks at theories of human learning (though it may very well be that I haven't looked closely enough!). What essentially all AI approaches do is modelling associative learning (https://en.wikipedia.org/wiki/Learning#Associative_learning), but employing more and more processing power for training. This is akin to having a fruit fly brain and copy-pasting that over and over again, in the hope somehow a higher order will organize from that by itself. But most things humans reason about are not learned this way, but rather by infering meaning from things and situations, i.e. one learns how a steering wheel works not through hundreds and hundreds of trial and error cases, but through single Aha! moments in which ad-hoc generated mental models (concepts) are validated in the environment. And GPT-3 has no knowledge organized in a hierarchical system of concepts, just as ELIZA didn't have one.

Re: An understanding of AI’s limitations is starting to sink in

#307

Post by elwes5 »

laingc wrote: post #96
mtgp1000 wrote: post #11

I'm not sure how anyone who's watched the exponential growth of a brand new domain can pick a point today to and say that things aren't as good as we expected. What may have happened was that some eager CEOs have overpromised on timelines and resources. But the revolution is coming, ML is already starting to change society. We're building the tech. Right now. The author does not even realise the immeasurable potentia…

I completely agree with this. Let me add a non-US perspective that may surprise some people. I think a big part of what is holding back many companies from making effective, genuine, real-world use of AI is that a significant majority of the individuals involved are bad at their jobs . On the business side, there is an widespread unwillingness to acknowledge that technical people may be better placed to make decision…

Your perspective is not just 'non-us'. The thing is since the early 80s computers can generate huge swaths of data. ML gives you a way to filter that data in a particular way. The same was true of data warehouses, smart systems, etc. The issue is not the business people vs technical people. It is an understanding of what do you want it to do. A few years ago I had a system that could generate 2k in data samples every few seconds (switches, voltages, temps, etc). What are you even looking for in that pile of stuff? You can not just feed that into a ML network and hope for the best. You have to describe what you are looking for. I had this same conversation over and over when working with data warehouse projects. A good BA on a project like that is amazing. Someone who is just kinda meh on it will kill the project dead. I do not understand your business, you do. I can apply what I have learned from other companies but only to a point. After that point I basically have to become a BA in that company just to understand what to write.

Re: An understanding of AI’s limitations is starting to sink in

#308

Post by kec »

smeeth wrote: post #297
AlexandrB wrote: post #220

Earlier quoted context omitted.

> There's far more to do with ML and AI than self driving cars and shitty ad recommendations. Yeah, there's also shitty sentencing recommendations[1], new-age phrenology[2], and high-tech redlining[3]. I think your entire field needs to take a year off and take some ethics and philosophy courses before going any further. Otherwise we're all going to end up much worse off. [1] https://www.nytimes.com/2017/10/26/opinio…

This is a bit like telling the whole CS community in the 90s that we need a year off the internet to study ethics because people can use it for child porn, money laundering, and cybercrime. Should we have?

Yes. The lack of ethics requirements in CS curriculums is a terrible oversight.

Would an ethics course and professional society eliminate child porn online? Of course not. Would it have shaped and possibly tempered social media giants into something less toxic? Maybe.

Re: An understanding of AI’s limitations is starting to sink in

#309

Post by foldr »

MiroF wrote: post #295
foldr wrote: post #279

Earlier quoted context omitted.

Progress in speech recognition has been slow and incremental, not fast and impressive. Hidden Markov Models did a decent job in the 90s. Now we have more data and more computing power.

In the past three years we have dropped error rates by a factor of 3, I don’t really think that claim holds water. We have seen huge progress in a number of other fields as well. Anecdotally, voice recognition has definitely gotten way better as well.

>In the past three years we have dropped error rates by a factor of 3

What are you referring to here?

Re: An understanding of AI’s limitations is starting to sink in

#310

Post by wokwokwok »

p1esk wrote: post #259
joe_the_user wrote: post #159

Earlier quoted context omitted.

The thing is, if we could write a specific, closed-end, prescriptive definition of "meaningful" or "understanding" or whatever, then we'd be able to program it. And we can't, so we have to settle for something else, usually how a thing fails to be what we (indeed subjectively) consider meaningful. Still, it's not arbitrary. The way that something like GPT-3 tends to fail basically is that you 2-3 paragraphs where par…

the thing has no fixed world-model What is "world-model"? What makes you think GPT-3 does not have some kind of a world model? It's clearly not a very good one, but at the same time it does not mean it can't get better. A 3 year old also does not have a very good world model, what's the difference between his world model and one of GPT-3? Again, clearly there's a big difference, I'm just not sure we know enough about…

> What if I finetune GPT-3 ...

> if you train a much bigger model on much bigger data...

Yep, no one is denying that the potential exists for something to be created that can do something useful.

However, the point being made is that GPT-3 doesn't do anything particularly useful.

The generated text from is not consistent, and apparently fine tuning can result in poor performance (ie. generates random crap). It scored better at a bunch of metrics, which is great, but of questionable practical value.

"world model" or not, it's currently interesting, amazing... probably not useful unless you're writing a spam bot.

What is unclear, is if bigger models will actually solve that or not.

Post reply on HN