Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

171–180 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#171

Post by ke88y »

arisAlexis wrote: post #143
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Fake news. 350 people including the top scientists from the top labs signed the petition. Your local univ AI researcher rolls his eyes. Not the guys working at OpenAI or anthropic or deep mind.

> Not the guys working at OpenAI or anthropic or deep mind.

A lot of them do. A huge percent of people who aren't speaking out are rolling their eyes. But what are you supposed to do? Contradict your boss's boss's boss?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#172

Post by jrflowers »

ethanbond wrote: post #166
jrflowers wrote: post #158

Earlier quoted context omitted.

> The AI skeptics don't need to know which outcome the future holds. But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.

Person observing Trinity test: "Wow this is really dangerous and could wipe out a city" Very astute response: "An atomic weapon has never wiped out a city"

Watching the first dot matrix printer output a page of text: “This might print out text that summons Cthulhu”

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#173

Post by [deleted] »

jameshart wrote: post #150
gamegoblin wrote: post #129

Earlier quoted context omitted.

GPT can give a single Yes/No answer that indicates a fair amount of intelligence for the right question. No iteration there. Just a single pass through the network. Hofstadter is surprised by this.

Well, no, it can produce a probability distribution over all possible tokens, among which ‘yes’ or ‘oui’ or ‘はい’ or ‘totally’ or ‘no’ or ‘nein’ or the beginning of ‘as a large language model I am unable to answer that question’ are all represented. Which is either more or less impressive than just being able to answer ‘yes or no’ depending on your priors I guess. There’s maybe an interesting philosophical question of…

[deleted]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#174

Post by biggoodwolf »

zone411 wrote: post #102
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

Read the parent comment again, even taking the survey at face value:

"EXPERTS DECLARE EXPERTS' FIELD IS MOST IMPORTATN!!!!"

No news, only snooze

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#175

Post by ethanbond »

jrflowers wrote: post #172
ethanbond wrote: post #166

Earlier quoted context omitted.

Person observing Trinity test: "Wow this is really dangerous and could wipe out a city" Very astute response: "An atomic weapon has never wiped out a city"

Watching the first dot matrix printer output a page of text: “This might print out text that summons Cthulhu”

Is all that money flowing into AI development and deployment in order to produce pieces of paper with text on them?

Nope, it's to produce society-scale transformation.

So even the optimists wouldn't buy your analogy.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#176

Post by hollerith »

echelon wrote: post #106
zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

We are serious about stopping you. We judge that at the present time, the main hope is that the US and British governments will ban large training runs and hopefully also shut down most or all of the labs. We judge that unlawful actions like torching data centers make it less likely we will realize the main hope.

Chinese society is much more likely to suddenly descend into chaos than the societies most of the people reading this are most familiar with, to briefly address a common objection on this site to the idea that a ban imposed by only the US and Britain will do any good. (It would be nice if there were some way to stop the reckless AI research done in China as well as that done in the US and Britain, but the fact that we probably cannot achieve that will not discourage us from trying for more achievable outcomes such as a ban in the US or Britain. I am much more worried about US and British AI research labs than I am of China's getting too powerful.)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#177

Post by firebirdn99 »

> But very soon they're going to be, they may very well be more intelligent than us and far more intelligent than us. And at that point, we will be receding into the background in some sense. We will have handed the baton over to our successors, for better or for worse.

Hinton also said, it's like we are a passing phase in evolution, where we created these immortal beings.

Being that we are so bad at predicting the future, and taking precautionary measures. See pandemic. Even all the alarm bells sounding, we won't be able to do anything concrete here. It's like we are mostly a reactive species, we don't have terribly good incentives to act in foresight.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#178

Post by wilg »

ke88y wrote: post #171
arisAlexis wrote: post #143

Earlier quoted context omitted.

Fake news. 350 people including the top scientists from the top labs signed the petition. Your local univ AI researcher rolls his eyes. Not the guys working at OpenAI or anthropic or deep mind.

> Not the guys working at OpenAI or anthropic or deep mind. A lot of them do. A huge percent of people who aren't speaking out are rolling their eyes. But what are you supposed to do? Contradict your boss's boss's boss?

> But what are you supposed to do? Contradict your boss's boss's boss?

Yes? Obviously?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#179

Post by jebarker »

yldedly wrote: post #113
jebarker wrote: post #98

Earlier quoted context omitted.

I was surprised he said this too. Even without the autoregressive part of GPT models you have a deep transformer with attention, so even a single forward pass can modify its own intermediate outputs.

What do you mean? Attention is just matrix multiplication and softmax, it's all feed forward.

I wasn't disputing that it's feed-forward. I just meant that stacked transformer layers can be thought of as an iterative refinement of the intermediate activations. Not the same as an autoregressive process that receives previous outputs as inputs, but far more expressive than a single transformer layer.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#180

Post by throwway120385 »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No thought was given to the effect of those decisions, only to the fact that this mysterious and "all-knowing" computer system told them it was a good decision. We're seeing the same thing with Facebook and "algorithms": "It's not us! It's the algorithm! We don't know why it does this." Which completely belies the fact that they wrote the algorithm and they should be held responsible for its effects.

We are about to enter a couple of decades of people using these pithy model systems to make real decisions that impact lots of people, and if I've learned anything in the past 20 years its that the impacts that technologists and "emininent minds" are predicting are nothing like what will actually happen. But terrible, banal things will be done at the behest of these systems and nobody is talking about it.

Post reply on HN