Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind: A Generalist Agent

deepmind.com

Post reply on HN
Print view View original

261–270 of 345 posts

Re: DeepMind: A Generalist Agent

#261

Post by atleta »

joe_the_user wrote: post #247
sinenomine wrote: post #211

Earlier quoted context omitted.

The human extinction due to would be "hard takeoff" of an AGI should be understood as a thought experiment, conceived in a specific age when the current connectionist paradigm wasn't yet mainstream. The AI crisis was expected to come from some kind of "hard universal algorithmic artificial intelligence", for example AIXItl undergoing a very specific process of runaway self-optimization. Current-generation systems aka…

We should think more about the Human alignment problem. Absolutely this The possibility of a thing being intentionally engineered by some humans to do things considered highly malevolent by other humans seems extremely likely and has actually been common through history. The possibility of a thing just randomly acquiring an intention humans don't like and then doing things humans don't like is pretty hypothetical and…

I wouldn't say the latter is hypothetical, or at least unlikely. We know from experience that complex systems tend to behave in unexpected ways. In other words, the complex systems we build usually end up having surprising failure modes, we don't get them right the first time. It's enough to think about basically any software written by anyone. But it's not just software.

I've just watched a video on YT about nuclear weapons, which included their history. The second ever thermonuclear weapon experiment (with a new fuel type) ended up with 2.5x the yield predicted, because there was a then unknown reaction that created additional fusion fuel during the explosion. [1]

[1] https://en.wikipedia.org/wiki/Castle_Bravo

Re: DeepMind: A Generalist Agent

#262

Post by ajmurmann »

Hellicio wrote: post #143
ajmurmann wrote: post #32

Earlier quoted context omitted.

> a world where AI is indistinguishable, or far superior, to human intelligence I think the part about being "indistinguishable from human intelligence" is potentially a intellectual trap. We might get to it being far superior while still underperforming at some tasks or behaving in ways that don't make sense to a human mind. An AI mind will highly likely work completely differently from humans and communicating with…

Only if you don't assume that consciousness comes from complexity. The physical ability of an animal to see better/different/faster doens't matter as we do not compare / seperate us from animals by those factors. We seperate us by consciousness and it might get harder and harder to shut down a PC on which a ML model is running which begs you not to do it.

I think we are talking past each other on a few things.

1. I should have been cheater. I want necessarily referring to physical ability, but cognitive. This might need navigation skills or pattern recognition, certain types of memory. A sibling comment linked an interesting article that had concrete experiments in which chimps outdid humans in certain games.

2. I thought this discussion was about general AI not consciousness. I suspect these two are related, but don't know. I suspect that anything that processes information might be conscious, but don't know. I'm even open to the idea that plants, my laptop or even a photo cell might be conscious. I would be shocked of we found that mammals aren't. But this is now closer to religion than anything else, since we cannot disproof that something has consciousness.

Re: DeepMind: A Generalist Agent

#263

Post by temp13890130 »

kromem wrote: post #260
sinenomine wrote: post #211

Earlier quoted context omitted.

The human extinction due to would be "hard takeoff" of an AGI should be understood as a thought experiment, conceived in a specific age when the current connectionist paradigm wasn't yet mainstream. The AI crisis was expected to come from some kind of "hard universal algorithmic artificial intelligence", for example AIXItl undergoing a very specific process of runaway self-optimization. Current-generation systems aka…

If anything, AGI seems to be the sole deus ex machina that can avert the inevitable tragedy we're on track for as a result of existing human misalignment. "Oh no, robots are going to try to kill us all" has to get in line behind "oh no, tyrants for life who are literally losing their minds are trying to measure dicks with nukes" and "oh no, oil companies are burning excess oil to mine Bitcoin as we approach climate c…

> We're already in a Greek tragedy. The machines really can't make it much worse, but could certainly make it much much better.

Except that, when true AGI arrives, we're all obsolete and the only things that will have any value are certain nonrenewable resources. No one has described a good solution for the economic nightmare that will ensue.

Re: DeepMind: A Generalist Agent

#264

Post by throwaway6532 »

viksit wrote: post #182

(Former AI researcher / founder here) It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI. For a) - I see 2 main blockers, 1) A way to build second/third order reasoning systems that rely on intuitio…

>In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).

And for this to develop in machines, machines would have to be subject to many mistakes along the way leading to all kinds of outcomes that we hold humans accountable for by fining them, sending them to jail, some of them dying etc. I think that would be so wholly unpalatable to man kind they'd cut that experiment short before it ever reached any sort of scale.

I agree with your conclusion that enough of the rules can't be encoded by us as we don't even know them and for machines to acquire them the traditional way is, I believe, fundamentally disagreeable to humans.

Re: DeepMind: A Generalist Agent

#265

Post by sdenton4 »

kromem wrote: post #260
sinenomine wrote: post #211

Earlier quoted context omitted.

The human extinction due to would be "hard takeoff" of an AGI should be understood as a thought experiment, conceived in a specific age when the current connectionist paradigm wasn't yet mainstream. The AI crisis was expected to come from some kind of "hard universal algorithmic artificial intelligence", for example AIXItl undergoing a very specific process of runaway self-optimization. Current-generation systems aka…

If anything, AGI seems to be the sole deus ex machina that can avert the inevitable tragedy we're on track for as a result of existing human misalignment. "Oh no, robots are going to try to kill us all" has to get in line behind "oh no, tyrants for life who are literally losing their minds are trying to measure dicks with nukes" and "oh no, oil companies are burning excess oil to mine Bitcoin as we approach climate c…

The machines will be the Greek chorus singing us to our doom.

Re: DeepMind: A Generalist Agent

#266

Post by sdenton4 »

hervature wrote: post #259
ben_w wrote: post #220

Earlier quoted context omitted.

Goals in the context of AI aren’t the type of thing you’re arguing against here. AI can absolutely have goals — sometimes in multiple senses at the same time, if they’re e.g. soccer AIs. Other times it might be a goal of “predict the next token” or “maximise score in Atari game”, but it’s still a goal, even without philosophical baggage about e.g. the purpose of life. Those goals aren’t necessarily best achieved by h…

What the parent is saying is that an AI (that is, AGI as that is what we are discussing) gets to pick its goals. For some reason, humans have a fear of AI killing all humans in order to to achieve some goal. The obvious solution is thus to achieve some goal with some human constraint. For example, maximize paperclips per human. That actually probably speeds up human civilization across the universe. No, what people a…

> maximize paperclips per human

Kill all humans, make one paperclip, declare victory.

Re: DeepMind: A Generalist Agent

#267

Post by cscurmudgeon »

brador wrote: post #213
justinpombrio wrote: post #208

Earlier quoted context omitted.

I am quite scared of human extinction in the face of AGI. I certainly didn't jump on it, though! I was gradually convinced by the arguments that Yudkowsky makes in "Rationality: from AI to Zombies" ( https://www.readthesequences.com/ ). Unfortunately they don't fit easily into an internet comment. Some of the points that stood out to me, though: - We are social animals, and take for granted that, all else being equal…

AI can’t have goals since the universe is logically meaningless. Our desire for purpose is a delusion.

If the universe is "logically meaningless", is your comment (which happily lives inside the universe) true or false?

Re: DeepMind: A Generalist Agent

#268

Post by sdenton4 »

croddin wrote: post #198
viksit wrote: post #182

(Former AI researcher / founder here) It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI. For a) - I see 2 main blockers, 1) A way to build second/third order reasoning systems that rely on intuitio…

I think of it as System 1 vs System 2 thinking from 'Thinking, Fast and Slow' by Daniel Kahneman.[1] Deep learning is very good at things we can do without thinking, and is in some cases superhuman in those tasks because it can train on so much more data. If you look at the list of tasks in System 1 vs System 2, SOTA Deep learning can do almost everything in System 1 at human or superhuman levels, but not as many in…

In biological history, system two was an afterthought at best. It likely didn't exist before spoken language, and possibly barely before written language. And to the extent that system two exists, it's running on hardware almost entirely optimized for system one thinking.

Re: DeepMind: A Generalist Agent

#269

Post by gmadsen »

viksit wrote: post #182

(Former AI researcher / founder here) It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI. For a) - I see 2 main blockers, 1) A way to build second/third order reasoning systems that rely on intuitio…

I'm not so sure its impossible. the 40 year semantic map project Douglas Lenat: Cyc is truly astounding. I think in the next decade we will see really interesting integration of state of the art deep learning with something like Cyc

Re: DeepMind: A Generalist Agent

#270

Post by yarg »

kromem wrote: post #258

It's interesting I so often see the discussion of AI in the context of what a singular model can or can't do. This seems myopic. Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for. And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task. Where…

On the topic of Broca's area, families with mutations to the FOXP2 gene are known to have trouble with vocalisations.

https://pubmed.ncbi.nlm.nih.gov/16437554/

There are a number of structures related to specific functionality - facial recognition is similarly hardwired.

Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.

Post reply on HN