Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

211–220 of 374 posts

Re: Among the A.I. doomsayers

#211

Post by serf »

sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

given who the people are that signed it really just comes off more like an attempt at creating a regulatory moat around the territory they got to first.

"This is extinction level important, so all you people who aren't us need to be careful meddling with the stuff we're meddling with for profit."

Re: Among the A.I. doomsayers

#212

Post by mitthrowaway2 »

i5heu wrote: post #199
mitthrowaway2 wrote: post #184

Earlier quoted context omitted.

Given that you don't think the CEO of Anthropic is qualified to comment on this matter, what kind of "data that is relevant" do you imagine we might be able to have, prior to an AGI existing? I am indeed uncomfortable with someone saying that we can't worry about a gas leak until after someone strikes a match and notices an explosion or the lack thereof. If the sad reality is that we can't detect the gas leak any oth…

I really wonder why there is such a fixation with these CEOs. I have proven to you logically that they cannot have more knowledge then anyone else, but you continue to present their authority as knowledge. Pls prove to me that they know more about the effects of AGI on humanity or that they have some magical knowledge we normal ppl are unable to comprehend. If you cannot do this, then why are you bringing up these pp…

I wasn't the one who brought that person up. I only kept with that example because you didn't answer my question when I asked you what would qualify someone to have an opinion, or what evidence you imagined could exist, and I didn't want to put words in your mouth by assuming that you also think that eg. Yoshua Bengio or Geoff Hinton or Max Tegmark (or take your pick of anybody) don't have informed opinions either. Or that they have well-reasoned arguments, rather than authoritative opinions, if you think that reason is applicable.

I'm only asking for a standard that falls slightly short of "already having built Schrodinger's doomsday weapon and observed whether humanity dies as a result". You can set the bar wherever else you like, please!

Re: Among the A.I. doomsayers

#213

Post by serf »

fragmede wrote: post #207
giantrobot wrote: post #202

Earlier quoted context omitted.

We don't give AP tests to humans to determine whether or not they are in fact an intelligent being. We give them to humans to show that over the course of about a decade and a half that human went from not realizing it had hands to ingesting a bunch of knowledge and had the capability to recall it answer questions. ChatGPT was trained on hundreds of terabytes of information. I would hope that with all that effort it…

what's your definition of intelligence that you're using here. there are various takes on what it means, so I'd like to know your take on it. That it's able to spit out essays makes it something , intelligence is too loosely defined to describe it for everybody, so we need to develop new words and meanings and new language for what it does. It doesn't think, but it does dot products to tensors to get create its outpu…

intelligence in animal species, roughly, is how well the animal deals with novel situations. Corvid tool use is a common example of animal intelligence.

It is interesting to consider AIs that way given that almost every AI will fail catastrophically when confronted with an area of expertise that is outside of their training corpus, although I guess Corvid tool use isn't exactly out of the animals' range of expertise.

Re: Among the A.I. doomsayers

#214

Post by parineum »

mitthrowaway2 wrote: post #175
i5heu wrote: post #171

Earlier quoted context omitted.

No, my position is that you can not extrapolate on something that does not exist and that’s “category” ever never existed. We don’t know any significant other intelligent then humans, not even aliens, so we have just no trace of direction that is rooted in reality rather then fantasy. This is especially obvious since opinion differ so much about the effects of AGI on humanity and since you cannot prove them right or…

It still seems to me that you're saying that we won't possibly be able to declare AGI an existential threat to humanity until after it has already been built. At that point, we can presumably settle the question by seeing if humanity goes extinct. This poses something of a paradox to those of us who prefer existential threats to humanity not get built in the first place.

I think you're interpretating the statement as "nobody can have an opinion" from "AI programming doesn't give someone a more qualified opinion on AI".

Plenty of people have interesting or valuable opinions and insights on AGI, programming LLMs doesn't have any impact into that though.

It'd be like expecting the developers of friendster to have a good idea of what the impact of social media was going to be in 2020.

AI isn't AGI.

Re: Among the A.I. doomsayers

#215

Post by otabdeveloper4 »

reducesuffering wrote: post #157
i5heu wrote: post #151

Earlier quoted context omitted.

They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.

What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?

Google search can solve AP tests too. (And before that library indexes did the same.) This isn't some new technology.

Re: Among the A.I. doomsayers

#216

Post by semi-extrinsic »

statuslover9000 wrote: post #203
antod wrote: post #162

Earlier quoted context omitted.

Part of me wonders if these people are intentionally framing the debate around ethics and potential risks as longer term extinction level problems to distract from the nearer term damage caused by them accelerating the economic inequality of the AI have-nots while they make themselves even richer.

The short-term effect is a harbinger of the long-term risk, since capitalism doesn’t inherently care for people who don’t provide economic value. Once superintelligent AI arises, none of us will have value within this system. Even the largest current capital holders will have a hard time holding on to it with an enormous intelligence disadvantage. The logical endpoint is the subjugation or elimination of our species,…

There are a lot of assumptions going on here. One of them is that superintelligent AI will arise. We have no reason to believe this will happen in our lifetimes. I posit that we are about as close to superintelligent AI as James Watt was to nuclear fusion.

The other assumption is that wealth and power are distributed according to intelligence. This is obviously false, wealth and power are largely distributed according to who you or your father plays golf with. As long as AIs don't play golf and don't have fathers, we are quite safe.

Re: Among the A.I. doomsayers

#217

Post by living_room_pc »

1vuio0pswjnm7 wrote: post #102

Always all-or-nothing thinking from these folks. Like what they are working on can never be just another boring thing that nerds find entertaining. No, it has to be "world-changing". Gonna "change the world" (for the better or the worse?) while sitting behind a keyboard. Except they do not know how to write. Overlook the important details and exaggerate, communicating in hyperbolic, know-it-all nerd gibberish. (Grant…

Its the AI hype train. There is so much money flowing around this topic right now, everyone wants their share of the pie. I would guess that anyone writing articles on the topic has some stake in AI as well either as an investor or a employee.

What I find is interesting, is that the negative news regarding AI safety is adding to the hype as well since it seems to capture a lot of attention.

Re: Among the A.I. doomsayers

#218

Post by otabdeveloper4 »

reducesuffering wrote: post #155
tivert wrote: post #148

Earlier quoted context omitted.

So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman ( https://blog.samaltman.com/the-merge ) I would say it is.

Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people! Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of c…

Let's hope Jezos gets transduced first, inshallah.

Re: Among the A.I. doomsayers

#219

Post by ActorNightly »

drooby wrote: post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Look at China.

You have arguably more STEM talent in China through a much more rigorous selection process, as well as a lot more centralized direction under a more authoritarian government, with a large amount of funding.

Yet China hasn't managed to take over the world. You would think that they if they wanted to become the worlds primary economic power, and they could throw enough compute at the problem (like an AGI would), they would have figured out how to do it by now.

Re: Among the A.I. doomsayers

#220

Post by roarcher »

hyperadvanced wrote: post #209
roarcher wrote: post #168

Earlier quoted context omitted.

I call it an LLM, not AGI. Consider this scenario: - Ask ChatGPT to produce some output meeting certain criteria - ChatGPT responds, but the output does not meet the criteria in one or more obvious ways - Point this out and ask for a corrected version - ChatGPT says "I apologize, here's the corrected version:" and spits out a nearly identical answer with the same obvious mistake. - Repeat steps 3 and 4 ad nauseam. I…

I’d say that LLM’s inability to handle DSL’s in particular is an interesting question on the topic of knowledge and language. Should a generally intelligent entity be able to quickly figure out a new dialect with little background? If I were to air-drop you into an extremely foreign nation, how long would it take you to organically decode the local language and be able to meaningfully craft expressions in it? I’m not…

I agree with your points in general, though I wasn't talking about DSL generation or any specific task. I was talking about ChatGPT's general tendency to cheerfully apologize for mistakes, explain exactly what the mistake was, and then present the same mistake while claiming that it's been corrected.

You can ask ChatGPT what it means to be asked to correct a mistake, and it will give you a perfectly thorough and eloquent answer. But it is often unable to apply this concept to its own behavior. If asked, it can explain back to you exactly what correction you want it to make. You can even make it pledge to correct the mistake in exactly the manner that it just described. And then it will completely fail to do it. It reminds me of that "repeat after me" meme from Friends [1].

This makes me lean in the direction of "stochastic parrot" when I think about what LLMs are. As impressive as it is, ChatGPT demonstrably lacks a sense of self. It talks as if it understands that it's an agent in control of its behavior, but then fails to control or even recognize its own actions.

[1] https://en.meming.world/images/en/c/c2/Friends-_Repeat_After...

Post reply on HN