Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#141

Post by llamaimperative » Mon, Jul 03, 2023, 6:24 PM UTC

jrflowers wrote: post #114

hackinthebochs wrote: post #77

Earlier quoted context omitted.

Seeing how confidently and obtusely people dismiss the risks of AI makes me so much more fearful of what's to come. Human hubris knows no limits.

It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re…

If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that.

Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry risk.

That's a perfectly sensible position. The optimist position isn't even internally consistent. See Andreessen on Sam Harris's podcast: AI will produce consumer utopia and drive prices down. Also, there are no downside risks because AI will be legally neutered from doing much of anything.

Is it legally neutered or is it transformative? The skeptical case doesn't rely on answering this question: to the extent it's effective, powerful, and able to do good things in the world, it will also be effective, powerful, and able to do bad things in the world. The AI skeptics don't need to know which outcome the future holds.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#142

Post by ke88y » Mon, Jul 03, 2023, 6:26 PM UTC

petermcneeley wrote: post #127

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

They are building consensus and finding alignment. The problem is power bends truth. This is all about access to a new powerful tool. They want to concentrate that access in the hands of those that already have control. The end goal here is the destruction of the general purpose computer.

Perhaps, but I doubt it. Never attribute to malice what can instead be attributing to stroking massive egos.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#143

Post by arisAlexis » Mon, Jul 03, 2023, 6:26 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Fake news. 350 people including the top scientists from the top labs signed the petition. Your local univ AI researcher rolls his eyes. Not the guys working at OpenAI or anthropic or deep mind.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#144

Post by zone411 » Mon, Jul 03, 2023, 6:26 PM UTC

startupsfail wrote: post #105

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield. You actually need someone with vision and track record of doing right predictions and placing right technology bets. Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of…

I agree. If you speak with ML engineers, many simply haven't deeply considered longer-term risks and things like self-improvement until recently. I think we'll see more concern, not less. For me personally, GPT-3 wasn't that surprising, but GPT-4 and code generation capabilities were.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#145

Post by arisAlexis » Mon, Jul 03, 2023, 6:27 PM UTC

ke88y wrote: post #134

zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

Loling something that serious with several top scientists as you said warning is at least very idiotic regardless where the truth lies. Let's start from that.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#146

Post by arisAlexis » Mon, Jul 03, 2023, 6:28 PM UTC

echelon wrote: post #106

zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

"If these people were so concerned, they'd by shouting from the hilltops"

they are, one by one

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#147

Post by pbw » Mon, Jul 03, 2023, 6:28 PM UTC

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong.

I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter".

Cockroaches are not an intelligent species, so they cannot understand our thoughts. But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea.

AI might eventually be able to think much quicker than us, to see patterns and make insights better and faster than us. But I don't think makes us cockroaches. I think if they are so smart, they are by definition smart enough to explain us any idea, and with effort we'll be able to understand it and contribute our own thoughts.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#148

Post by phillipcarter » Mon, Jul 03, 2023, 6:28 PM UTC

echelon wrote: post #138

zone411 wrote: post #128

Earlier quoted context omitted.

If you read the survey, you'll find that many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. So, this type of response wouldn't be rational if they're thinking that things will go fine most of the time. If these researchers are thinking logically, they would also realize that this kind of reaction has little chance of success, especially if research continues in places like China. It…

> If you read the survey Not a nice way to engage in debate. I've spent more time listening to and refuting these arguments than most. Including debating Eliezer. > many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. A 10% chance of an asteroid hitting the earth would result in every country in the world diverting all of their budgets into building a means to deflect it. > So, this t…

> Not a nice way to engage in debate.

So...you didn't read the survey.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#149

Post by petermcneeley » Mon, Jul 03, 2023, 6:29 PM UTC

ke88y wrote: post #142

petermcneeley wrote: post #127

Earlier quoted context omitted.

They are building consensus and finding alignment. The problem is power bends truth. This is all about access to a new powerful tool. They want to concentrate that access in the hands of those that already have control. The end goal here is the destruction of the general purpose computer.

Perhaps, but I doubt it. Never attribute to malice what can instead be attributing to stroking massive egos.

"People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public"

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#150

Post by jameshart » Mon, Jul 03, 2023, 6:29 PM UTC

gamegoblin wrote: post #129

jameshart wrote: post #123

Earlier quoted context omitted.

It only yields something that looks like intelligence when you update the context and iterate though.

GPT can give a single Yes/No answer that indicates a fair amount of intelligence for the right question. No iteration there. Just a single pass through the network. Hofstadter is surprised by this.

Well, no, it can produce a probability distribution over all possible tokens, among which ‘yes’ or ‘oui’ or ‘はい’ or ‘totally’ or ‘no’ or ‘nein’ or the beginning of ‘as a large language model I am unable to answer that question’ are all represented. Which is either more or less impressive than just being able to answer ‘yes or no’ depending on your priors I guess.

There’s maybe an interesting philosophical question of perspective there because if you think of the GPT as answering the question ‘if you had just read this, what token would you expect to read next?’ That doesn’t seem like a question that necessarily requires ‘intelligence’ so much as ‘data’. It’s just a classification problem and we’ve been throwing NNs at that for years.

But if you ask the question ‘if you had just written this, what token would you expect to output next?’ It feels like the answer would require intelligence.

But maybe they’re basically identical questions?