Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#281

Post by detourdog » Mon, Jul 03, 2023, 9:03 PM UTC

spaceman_2020 wrote: post #255

detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

> I also think that AR/VR is more scary than AI Absolutely agree. I was never bothered by Oculus, but Apple's Vision Pro demonstration was equal parts fascinating and terrifying. I can see the next generation getting completely lost in alternate realities. Smartphone addiction got nothing on what's about to come.

I only have experience with the Apple Vision demo and the terrifying/emotional part for me was the beautiful people using the device in beautiful places.

Everything was so clean and stress free that it was obviously artificial. I could only imagine stressed out people using it in squalor. The whole demo seemed geared to keep that thought far away.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#282

Post by JimtheCoder » Mon, Jul 03, 2023, 9:03 PM UTC

flangola7 wrote: post #269

JimtheCoder wrote: post #227

Earlier quoted context omitted.

Doesn't this only work if everyone is stupid and believes everything they see/read/hear on the internet? You have to have a little more faith in humanity than that...

With respect: *Have you been living under a rock?*

No, I have not.

I do not judge humanity as a whole based on a very vocal minority...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#283

Post by [deleted] » Mon, Jul 03, 2023, 9:03 PM UTC

pseudonom- wrote: post #272

ke88y wrote: post #134

Earlier quoted context omitted.

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

> Ie even lower than my guess of 90% above. These are not comparable numbers. You're comparing "fraction of people" vs "fraction of outcomes". Presumably an eye-roller assigns ~0 probability to "extremely bad" outcomes (or has a shockingly cavalier attitude toward medium-small probabilities of catastrophe).

[deleted]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#284

Post by gundmc » Mon, Jul 03, 2023, 9:03 PM UTC

A very interesting conversation, but the article really makes the reader work to understand what his previous criticisms were and how they have now changed. It feels like the author assumes the reader has been closely following academic discourse on the subject. Maybe that's a fair assumption for their typical readers but it does make the article less accessible for newer readers.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#285

Post by Mouthfeel » Mon, Jul 03, 2023, 9:06 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

[dead]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#286

Post by mquander » Mon, Jul 03, 2023, 9:07 PM UTC

echelon wrote: post #106

zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

What do you mean, you haven't seen anything else like it? At least O(100) people are working full-time on research programs trying to mitigate AI x-risks. The whole founding team of Anthropic left OpenAI to do their own thing specifically because they thought that's what they needed to do to pursue their safety agenda.

Isn't that taking the problem at least as seriously as quitting a job at Google?

It sounds like you think that the main way to act on a concern is by making a maximum amount of noise about it. But another way to act on a concern is to try to solve it. Up until very recently, the population of people who perceive the risk are mostly engineers and scientists, not politicians or journalists, so they are naturally inclined towards the latter approach.

In the end, if people aren't putting their money (really or metaphorically) where their mouth is, you can accuse them of not really caring, and if people are putting their money where their mouth is, then you can accuse them of just talking their book. So reasoning from whether they are acting exactly how you think they should, is not going to be a good way to figure out how right they are or aren't.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#287

Post by jurgenaut23 » Mon, Jul 03, 2023, 9:08 PM UTC

zerocrates wrote: post #232

mlyle wrote: post #59

Earlier quoted context omitted.

I think there's a number of just crappy scenarios that can come with LLMs and other generative AI: - Further trashing our public discourse: making truth even more uncertain and valuable information even harder to find. We're not doing great with social media, and it's easy to envision that generative AI could make it twice as bad. - Kneecapping creative work by commoditizing perhaps half of it. There's going to be a…

Yeah these effects seem hugely foreseeable (and/or are currently happening already) but tend to not be the type of thing "AI risk" people are talking about.

Exactly! I am positive there will be a ton of negative impact, like with most (all?) of tech. And it might be worse than anything we’ve seen (although beating the Snapchat algo might prove tricky). But that’s NOT what Tegmark and Hofstadter are talking about… their concern is existential and somewhat philosophical in the case of Hofstadter, as if GPT-4 questioned his very nature. To me, that doesn’t make sense.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#288

Post by dlkf » Mon, Jul 03, 2023, 9:09 PM UTC

startupsfail wrote: post #105

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield. You actually need someone with vision and track record of doing right predictions and placing right technology bets. Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of…

> You actually need someone with vision and track record of doing right predictions and placing right technology bets.

Hofstadter does not exactly fit this description.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#289

Post by anigbrowl » Mon, Jul 03, 2023, 9:10 PM UTC

tyre wrote: post #164

ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

This is a great way to put it. The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes. But I don't think that matters. I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate…

I think LLMs are very close to AGI, lacking only the real-time feedback element (easily enough replicated by running them off batteries). I'm also more sanguine about the existential risks because I have somewhat more confidence in the rationality of AGI than I do in that of humans.

I don't think it takes a huge amount of faked primary material (generated phone calls, fuzzy video, etc.) that's massively amplified until it becomes "true enough" to drive a Chinese invasion of Taiwan, a Russian tactical nuclear strike in Ukraine, an armed insurrection in the United States.

This I agree with 100%. Modern information warfare is about constructing reliable viral cascades, and numerous influencers devote themselves to exactly that for various mixes of profit an ideology. Of your 3 scenarios the third seems most likely to me, and is arguably already in progress. The other two are equally plausible, but imho dictatorships tend to centralize control of IW campaigns to such a degree that they lack some of the organic characteristics of grassroots campaign. Incumbent dictators' instinct for demagoguery is often tempered with a desire for dignity and respectability on the world stage, which might be a reason than civil strife and oppression tends to be more naked and ruthless in less developed countries where international credibility matters less.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#290

Post by sanderjd » Mon, Jul 03, 2023, 9:11 PM UTC

phillipcarter wrote: post #157

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

I'm not sure if it's coincidental and ironic or entirely intended by you, but I found it funny that my introduction to this crowd was a literal fanfiction written by Eliezer Yudkowsky.