Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#251

Post by dist-epoch » Mon, Jul 03, 2023, 8:07 PM UTC

petermcneeley wrote: post #240

dist-epoch wrote: post #219

Earlier quoted context omitted.

I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.

Perhaps that is Nassim's argument because he is an expert in risk. The actual issue with experts monitoring experts is the bias induced by self interest (see previous sentence). Nassim sort of gets this when it comes to his discussion of "skin in the game". I dont want to malign Nassim too much but he seems like the kind of person that "sorta gets it" when it comes to everything that matters.

AI researchers also have a vested interest in saying their technology is safe.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#252

Post by jmopp » Mon, Jul 03, 2023, 8:09 PM UTC

bobcostas55 wrote: post #197

phillipcarter wrote: post #157

Earlier quoted context omitted.

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

[flagged]

I love how you jumped straight from "people weirdly into eugenics should be disavowed" to "dysgenics is desirable". That is the epitome of black-and-white thinking if I ever saw it.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#253

Post by petermcneeley » Mon, Jul 03, 2023, 8:10 PM UTC

dist-epoch wrote: post #251

petermcneeley wrote: post #240

Earlier quoted context omitted.

Perhaps that is Nassim's argument because he is an expert in risk. The actual issue with experts monitoring experts is the bias induced by self interest (see previous sentence). Nassim sort of gets this when it comes to his discussion of "skin in the game". I dont want to malign Nassim too much but he seems like the kind of person that "sorta gets it" when it comes to everything that matters.

AI researchers also have a vested interest in saying their technology is safe.

Does Douglas Hofstadter?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#254

Post by Gibbon1 » Mon, Jul 03, 2023, 8:17 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> Why is that?

Because of deep silo myopia. Meaning they have no idea what terrible things the pointy haired bosses and grifters are going to use this stuff for.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#255

Post by spaceman_2020 » Mon, Jul 03, 2023, 8:18 PM UTC

detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

> I also think that AR/VR is more scary than AI

Absolutely agree. I was never bothered by Oculus, but Apple's Vision Pro demonstration was equal parts fascinating and terrifying. I can see the next generation getting completely lost in alternate realities.

Smartphone addiction got nothing on what's about to come.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#256

Post by YeGoblynQueenne » Mon, Jul 03, 2023, 8:22 PM UTC

kalkin wrote: post #248

kalkin wrote: post #241

Earlier quoted context omitted.

Is there a way that you'd recommend somebody outside the field assess your "90%" claim? Elsewhere in the thread you're dismissive of that one survey - which I agree is weak evidence in itself - and you also deny that the statements of leaders at OpenAI, DeepMind and Anthropic are representative of researchers there, which again may be the case. But how is someone who doesn't know you or your network supposed to asses…

Relatedly, it'd be helpful if you could point to folks with good ML research credentials making a detailed case against x-risk. The prominent example of whom I'm aware is Yann LeCun, and what I've seen of his arguments take place more in the evopsych field (the stuff about an alleged innate human drive for dominance which AIs will lack) than the field in which he's actually qualified.

(Not the OP) In this article Le Cun argues in very concrete technical terms about the impossibility of achieving AGI with modern techniques (yes, all of them):

Meta's AI guru LeCun: Most of today's AI approaches will never lead to true intelligence

https://www.zdnet.com/article/metas-ai-guru-lecun-most-of-to...

Edit: on second thought, he gets maybe a bit too technical at times, but I think it should be possible to follow most of the article without specialised knowledge.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#257

Post by orlp » Mon, Jul 03, 2023, 8:23 PM UTC

pbw wrote: post #198

wslh wrote: post #165

Earlier quoted context omitted.

> A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter". I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from…

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case. Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas…

> but if AIs invent a new type of math, they would be able to teach it to us

There are already math proofs made by humans on this very day that are hundreds upon hundreds of pages of lemmas that are highly advanced and building on other advanced results. Understanding such a proof is an undertaking that literally takes years. An AI might end up doing it in minutes. But what an AI could cook up in years could take a human... several lifetimes to understand.

As another example, take the design and fabrication of a modern microprocessor. There are so many layers of complexity involved, I would bet that no single person on this planet has all the required knowledge end-to-end needed to manufacture it.

As soon as the complexity of an AI's knowledge reaches a certain point, it essentially becomes unteachable in any reasonable amount of time. Perhaps smaller sub-parts could be distilled and taught, but I think it's naive to assume all knowledge is able to be sliced and diced to human-bite-sized chunks.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#258

Post by ke88y » Mon, Jul 03, 2023, 8:23 PM UTC

kalkin wrote: post #222

ke88y wrote: post #185

Earlier quoted context omitted.

> Yes? Obviously? I mean, I agree, obviously. My point is that most people don't. And I think for two reasons. The first and more important reason is that most people aren't involved in The Discourse and don't want to be involved in The Discourse. That's probably 99.9% of the Silent Majority -- they simply don't want to talk about anything on HN or Twitter. They view it as a waste of time or worse. and they aren't wr…

It's a bit cheeky of you to be complaining about personal insults while the substance of your OP was the assertion that x-risk worriers are motivated by ego rather than real thought.

I considered that when replying. I don't think I am making a personal criticism just for the sake of it, or an ad hom. And I do think the observation I am making is interesting. I don't mean it as a personal attack. Really.

Suppose what I am saying is true -- that relatively unknown people rolling their eyes or laughing and relatively known people being very earnestly concerned. And that these are people with otherwise similar credentials, at least as far as assessing x-risk is concerned.

Maybe you disagree, and that's okay, and there are other threads where that discuss is ongoing. But here let's assume it's true, because I think it is and that's relevant to your fair criticism.

Like, it is a weird thing, right? Normally famous scientists and CEOs are not so far out ahead of the field on things like this. More often than not it's the opposite. To have that particular set of people so far out of stride isn't particularly normal.

I think the common thread that differentiates similarly-senior people on the x-risk question is not experience, or temperament, or scope of responsibility. Or even necessarily the substance of what they believe if you sit down and listen and probe what they really mean when thy say there is or isn't x-risk from AI! The difference is mostly amount of Ego and how much they want to be in The Discourse.

Also: I don't think that having a large ego is necessarily a character flaw, any more than having a strong appetite or needing more/less sleep. It's just how some people are, and that's okay, and people with big egos can be good or bad people, and circumstantially ego can be good or bad. But people who have bigger egos do behave a bit differently sometimes.

Anyways, I'm not trying to assassinate anyone's character or even necessarily mount an ad hom dismissal of x-risk. I'm observing something which I think is true, and doing it in as polite a way as I can even though it's a bit of an uncomfortable thing to say.

I guess what I'm trying to say is that "maybe this personality trait explains a weird phenomenon of certain types of experts clustering on an issue", and it's worth saying if you think it might be true, even if that personality trait has (imo perhaps overly) negative connotations.

And in any case this is substantially different from "you're an idiot because I disagree with you".

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#259

Post by sanp » Mon, Jul 03, 2023, 8:26 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

The warnings are an admission from the “Great Minds and a great Thinkers” that someone other than them has created something previously thought impossible or, at the very least, several years / decades out. So, I am not sure ego is at play here. Perhaps someone close to the problem (your actual scientists on the ground) is not capable / unwilling to accept the issues raised as that has direct (negative) impact on what they have created.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#260

Post by leftcenterright » Mon, Jul 03, 2023, 8:29 PM UTC

tasuki wrote: post #237

lo_zamoyski wrote: post #206

Earlier quoted context omitted.

> The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. Except they're not philosophizing, not in any real sense of the word. They're terrible at it. Most of them are frauds, quacks, and pseudo-intellectual con artists (like Harari) who adore the limelight offered to them by the media and a TED Talks-watching segment of t…

> frauds, quacks, and pseudo-intellectual con artists (like Harari) Uh, mind elaborating? Why is Harari that? Do you have any examples of non-frauds and actual intellectuals? > a TED Talks-watching segment of the public who are, frankly, intellectually out of their depth, but enjoy the delusion and feeling of participating in something they think is "intellectual" I'm afraid that would be me.

I believe at least part of the reason for calling Harari pseudo-intellectual is the use of misleading statements in favor of storytelling in the books and also building up on others' work with better storytelling.

More on this: https://www.currentaffairs.org/2022/07/the-dangerous-populis...