Talk about “AI” in the press these days tends to conflate two things • Hacker News Board

Re: Talk about “AI” in the press these days tends to conflate two things

#61

Post by Veedrac » Sat, Dec 12, 2020, 4:41 PM UTC

qsort wrote: post #51

Veedrac wrote: post #48

Earlier quoted context omitted.

This is what I've started thinking of as the ‘pretty dumb for a human’ take on AI. If your requirement for believing something is on the path to AGI is for it to practically already be AGI, it's no longer predictively meaningful. This sort of argument suffices to dismiss, for example, the possibility that today's humans evolved from apes, or even that today's humans could have come from pre-language, pre-civilization…

Not OP, but to be honest there isn't much at all that suggests significant steps have been made towards AGI. GPT-3 isn't AGI, it's the opposite of AGI. To be really frank, I doubt AGI is more than a teleological narrative.

I disagree, but that wasn't the argument I was objecting to.

Re: Talk about “AI” in the press these days tends to conflate two things

#62

Post by username90 » Sat, Dec 12, 2020, 4:41 PM UTC

Veedrac wrote: post #59

username90 wrote: post #57

Earlier quoted context omitted.

> Ant minds are not general intelligences The fact that you don't consider ants as general intelligences means you agree that we aren't making progress towards general intelligences, since ants intelligence is way closer to general intelligence than any computer program we have. > Their pathfinding and identification and anthill-making and communication are all fixed programs in their genetics, not learnt during life…

> The fact that you don't consider ants as general intelligences means you agree that we aren't making progress towards general intelligences, since ants intelligence is way closer to general intelligence than any computer program we have. Please don't argue like this. It's your job to explain what you think is true, and my job to explain what I think is true. Up to this point in this conversation I explicitly haven'…

I still don't see your point. The way we do AI today is that we have a static dataset, train a fuzzy function and then let it do work in production. The production model doesn't identify when it makes mistakes and update itself like every animal does. Ants does this. They don't do it perfectly, but they do it.

So the way I see it until AI starts meaning only models that updates themselves as they evaluate input it is a pointless term. Currently basically every single instance of AI you see people talk about are just dumb immutable functions.

Edit: And to clarify, I believe it is impossible to reach human level intelligence without having a model that mutates itself as it tries to solve the problem. Humans are smart since we learn while doing, not that we first take a long course before we start doing.

Re: Talk about “AI” in the press these days tends to conflate two things

#63

Post by tpoacher » Sat, Dec 12, 2020, 4:42 PM UTC

My own distinction between ML and AI (which I agree with the article that marketing buzzheads do not care about) is that one is a set of optimization techniques not necessarily involving an agent, and the other involves an agent, who needs to make (intelligent) decisions. The decisions themselves may rely on said ML-based optimization techniques, or they might not. Similarly, optimization techniques may employ an agent-based approach, but equally they may not.

Re: Talk about “AI” in the press these days tends to conflate two things

#64

Post by Veedrac » Sat, Dec 12, 2020, 5:03 PM UTC

username90 wrote: post #62

Veedrac wrote: post #59

Earlier quoted context omitted.

> The fact that you don't consider ants as general intelligences means you agree that we aren't making progress towards general intelligences, since ants intelligence is way closer to general intelligence than any computer program we have. Please don't argue like this. It's your job to explain what you think is true, and my job to explain what I think is true. Up to this point in this conversation I explicitly haven'…

I still don't see your point. The way we do AI today is that we have a static dataset, train a fuzzy function and then let it do work in production. The production model doesn't identify when it makes mistakes and update itself like every animal does. Ants does this. They don't do it perfectly, but they do it. So the way I see it until AI starts meaning only models that updates themselves as they evaluate input it is…

There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’.

It's easiest to train on IID data, and also common to train in specific RL environments, because those paper over some issues with catastrophic forgetting and overfitting, and it's also preferable to do all the compute-intensive updates during training for efficiency. However, lots of people agree that online learning is fundamental to cracking AGI, and it's neither neglected nor unsuccessful.

Re: Talk about “AI” in the press these days tends to conflate two things

#65

Post by username90 » Sat, Dec 12, 2020, 5:17 PM UTC

Veedrac wrote: post #64

username90 wrote: post #62

Earlier quoted context omitted.

I still don't see your point. The way we do AI today is that we have a static dataset, train a fuzzy function and then let it do work in production. The production model doesn't identify when it makes mistakes and update itself like every animal does. Ants does this. They don't do it perfectly, but they do it. So the way I see it until AI starts meaning only models that updates themselves as they evaluate input it is…

There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. It's easiest to train on IID data, and also common to train in specific RL environments, because those paper over some issues with catastrophic forgetting and overfitting, and it's also preferable to do all the compute-intensive updates during training for efficiency. However, lots of people agree that online learning…

> There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’.

For some definition of plenty. I'd argue there isn't since almost all resources poured into AI are spent getting better at optimizing immutable functions. I don't see progress in getting better at optimizing immutable functions as progress towards general intelligence, hence I don't see almost anything people do in AI today as progress towards general intelligence.

Do you understand now? I don't say nobody works on the things I think is important, just that it is mostly neglected compared to the enormous amount of resources spent on dumb models.

Re: Talk about “AI” in the press these days tends to conflate two things

#66

Post by Veedrac » Sat, Dec 12, 2020, 5:24 PM UTC

username90 wrote: post #65

Veedrac wrote: post #64

Earlier quoted context omitted.

There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. It's easiest to train on IID data, and also common to train in specific RL environments, because those paper over some issues with catastrophic forgetting and overfitting, and it's also preferable to do all the compute-intensive updates during training for efficiency. However, lots of people agree that online learning…

> There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. For some definition of plenty. I'd argue there isn't since almost all resources poured into AI are spent getting better at optimizing immutable functions. I don't see progress in getting better at optimizing immutable functions as progress towards general intelligence, hence I don't see almost anything people do in A…

Progress on online learning builds on progress on offline learning, because it's the same optimization procedure.

I don't really agree with your stance on it, but I was not objecting to the claim that people should focus more on online learning. I was objecting to the claims “super computers can't even compete with the intelligence of an ant”, which is false, since the impressive behaviours an ant displays are not learned, and the learned aspects aren't competitive with what computers can do, and that “[the] way we do AI can never achieve that, since it always relies on human intelligence to decide all of those things”, which is also false, because online learning research exists and works, if imperfectly.

Re: Talk about “AI” in the press these days tends to conflate two things

#67

Post by username90 » Sat, Dec 12, 2020, 5:27 PM UTC

Veedrac wrote: post #66

username90 wrote: post #65

Earlier quoted context omitted.

> There is plenty of research on exactly this sort of learning-while-doing, termed ‘online learning’. For some definition of plenty. I'd argue there isn't since almost all resources poured into AI are spent getting better at optimizing immutable functions. I don't see progress in getting better at optimizing immutable functions as progress towards general intelligence, hence I don't see almost anything people do in A…

Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I don't really agree with your stance on it, but I was not objecting to the claim that people should focus more on online learning. I was objecting to the claims “super computers can't even compete with the intelligence of an ant”, which is false, since the impressive behaviours an ant displays are not le…

> Progress on online learning builds on progress on offline learning, because it's the same optimization procedure.

I disagree with this assertion, treating online learning like offline learning means you aren't making any progress. The main difference is that online learning lets you interact with the problem and seek more information, offline learning doesn't. Offline learning doesn't help you at all solving that problem.

Re: Talk about “AI” in the press these days tends to conflate two things

#68

Post by Veedrac » Sat, Dec 12, 2020, 5:44 PM UTC

username90 wrote: post #67

Veedrac wrote: post #66

Earlier quoted context omitted.

Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I don't really agree with your stance on it, but I was not objecting to the claim that people should focus more on online learning. I was objecting to the claims “super computers can't even compete with the intelligence of an ant”, which is false, since the impressive behaviours an ant displays are not le…

> Progress on online learning builds on progress on offline learning, because it's the same optimization procedure. I disagree with this assertion, treating online learning like offline learning means you aren't making any progress. The main difference is that online learning lets you interact with the problem and seek more information, offline learning doesn't. Offline learning doesn't help you at all solving that p…

Exploration is a major part of RL already, see AlphaGo for example.

Re: Talk about “AI” in the press these days tends to conflate two things

#69

Post by kerng » Sat, Dec 12, 2020, 7:26 PM UTC

NalNezumi wrote: post #33

kerng wrote: post #26

Earlier quoted context omitted.

You might underestimate the speed and velocity of advancement in the field. A few years ago creating artificial data or images that are indistinguishable from real data was impossible. Now machines generate information, like entire videos that are real but fake. This can, and has already (look at facebook and fake profiles), led to big problems and in a few years the world will change quite drastically because of fur…

I don't think I underestimate the speed nor velocity of the advancement, just being realistic about the scope of the advancements. I follow conferences and read ML papers (DL & DL/ML application in Robotics) regularly so I think I have a some grasp on the advancement in the field. GAN sure have been a big step, and have found a great usage. (deepfake etc) but I'd say it is still highly overblown. Like GPT3 it will se…

I don't think ML over promises anything, its just a new way to help create software to solve problems that can't really be solved with traditional programming.

Re: Talk about “AI” in the press these days tends to conflate two things

#70

Post by kerng » Sat, Dec 12, 2020, 7:30 PM UTC

dvfjsdhgfv wrote: post #45

kerng wrote: post #26

Earlier quoted context omitted.

You might underestimate the speed and velocity of advancement in the field. A few years ago creating artificial data or images that are indistinguishable from real data was impossible. Now machines generate information, like entire videos that are real but fake. This can, and has already (look at facebook and fake profiles), led to big problems and in a few years the world will change quite drastically because of fur…

I appreciate your enthusiasm but we're still not there. The videos look almost perfect, with "almost" still oscillating around the uncanny valley. Self-driving cars are almost totally self-driving and are almost not killing people. GPT3 uses fluent English and can be used to generate almost meaningful content in most contexts. Speech recognition progressed enormously and can be used to transcribe most well-pronouncin…

The interesting thing is that in many ways we are already there - but it's difficult to see because we are part of the shift.

I really enjoyed (think it was Joscha Bach) take on the idea that companies are basically AI with humans in the loop.

The more our life and work is online and digital the more computers and algorithms will control what we see and experience, and what opinions we have.

A company has a cost function to optimize (like revenue), something that machines can optimize towards as well.

So maybe AI is just like a company but with humans out of the loop. It might also be that the human way of "intelligence" is not the most optimal approach.

I mean we cant even define what it means to be intelligent, which is why we are having these discussions.