Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

341–350 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#341

Post by mitthrowaway2 »

Imnimo wrote: post #35

I feel like if I actually believed that the death of all biological life was imminent and that the only way to forestall it was to bomb data centers, I'd spend a lot more time planning how to bomb data centers and a lot less time curating Shoggoth memes on my web forum. On the other hand, if I were more concerned with self-aggrandizement, writing doomsaying thinkpieces for Time is exactly what I'd spend my time on.

Bombing data centers would be really ineffective. How many data centers do you think you could take down before you got caught? How much time would that really set back AI training? Plus it would ensure that nobody respectable ever associates with your cause, whether they privately agree with it or not.

Between that and advocating for an international treaty that treats GPUs the way the world currently treats plutonium? I think the latter has a higher chance of success, however small it may be.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#342

Post by mitthrowaway2 »

kromem wrote: post #178
jacknews wrote: post #168

I don't accept the premise that biological life is doomed by superintelligent machines, why would it be? But there is risk, obviously, which needs to be addressed, yet, quite predictably, in order of concerns, competition for profits or geopolitical advantage is coming first. And second, third, ...

Because hand wave . The real answer is that most of these folks started thinking about this stuff in a day and age very far from the one we are in, established a picture they clung to since, and have in some cases built entire careers around it. It's called anchoring bias. His comments on "atoms repurposed" echo the 70s paperclip problem, when so far humanity is doing just fine at not knowing when to stop making crap…

That's an unfair and low-effort dismissal of a lot of very well-thought-out and carefully reasoned arguments.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#343

Post by etienne618 »

gonehome wrote: post #334
etienne618 wrote: post #332

Earlier quoted context omitted.

I disagree with your argument, especially for point 1: these systems are massively constrained. The hardware they run on is fragile requiring massive amounts of power and tightly controlled environments. They don't have any means of replicating themselves (it can't run on arbitrary systems). The datacenters also have massive bandwidth between nodes - even if you could run 'it' on all the personal computers and phones…

It's a narrower argument that I'm making - it's not about LLMs or implementation (can put those details aside), it's about possibility that a superintelligent AGI could be created, there's nothing magical about biological intelligence that would prevent it. How or how difficult, or when are all questions that follow from that first premise (that it is possible). I don't really make strong claims about any of the impl…

I hear you - its just that I dont think that it neccesarily follows that an AGI in whatever form will necesarily be unconstrained or free to improve itself. I also happen to think that the biological systems are in-fact rather special and hard to replicate - especially in terms of efficiency and resillience. Just because some implementation of a universal turing machine can simulate intelligence doesn't mean it can do it well enough to survive the real world.

Finally, to me, nuclear reactions are kind of the opposite of AGI: I think it's vastly easier to blow something up (increase entropy) than to create an intelligence capable of understanding and improving itself (decreasing entropy - possibly at an accelerating rate).

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#344

Post by righttoolforjob »

_skel wrote: post #329
gonehome wrote: post #308

Earlier quoted context omitted.

I'd argue the dismissals are a lot more handwavy. The simplified argument is: - Superintelligent AGI that can modify itself in pursuit of a goal is possible. - If that AGI is not aligned with human goals it very likely ends in the end of humanity. - We have no idea how to align an AGI or even really observe what it's true state/goals are. Without this capability if we stumble upon creating an AGI capable of improving…

Point 2 does not logically follow from point 1. That's the entire problem here. Between point 1 and 2 there must be quite a few other steps, causally linked, otherwise it's just a massive imaginary leap based on assumptions that nobody is explaining. > The dismissals from Tyler Cohen and Pinker are mostly just relying on heuristics which are often right, but even if they're right 999 out of 1000 times, if that 1 in 1…

Who said it followed? That's a strawman. To dumb it down even further for you, I guess...

There are brains, therefore there can be created an artificial super-brain. Artificial super-brain might have goals which don't align with human brain and we will have no way to understand or control the situation.

Two unrelated facts, which together mean that we should be careful with experimenting with the science working towards super-brains. Just a single super-brain could end us.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#345

Post by righttoolforjob »

etienne618 wrote: post #343
gonehome wrote: post #334

Earlier quoted context omitted.

It's a narrower argument that I'm making - it's not about LLMs or implementation (can put those details aside), it's about possibility that a superintelligent AGI could be created, there's nothing magical about biological intelligence that would prevent it. How or how difficult, or when are all questions that follow from that first premise (that it is possible). I don't really make strong claims about any of the impl…

I hear you - its just that I dont think that it neccesarily follows that an AGI in whatever form will necesarily be unconstrained or free to improve itself. I also happen to think that the biological systems are in-fact rather special and hard to replicate - especially in terms of efficiency and resillience. Just because some implementation of a universal turing machine can simulate intelligence doesn't mean it can d…

Nobody says that it necessarily follows, but it's a quite trivial step once you have the AGI, hence pointless that you bring up it as an argument.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#346

Post by famouswaffles »

labrador wrote: post #313
edgyquant wrote: post #276

Earlier quoted context omitted.

So home schooled and/or religious people shouldn’t be allowed to have their voice heard? Shouldn’t be allowed to put forth ideas?

Of course they should. I just like them to be up front about them if they are far enough outside the mainstream that they might be somewhat surprising and we should consider how those beliefs inform the opinions they put forth. Peter Thiel is open about his and I respect him more for that. I worry about hidden agendas. https://www.google.com/search?q=peter+thiel+religious+belief...

What an absurd paragraph i've just read lol. Many, Many things influence a person's opinions, religious or not. Why don't you tell me all about your life so i don't know you have some hidden agenda ?

Like lol what exactly are you expecting here ? Why should that be some special announcement call you have to make with every article instead of the myriad of things that may influence a person's opinions ?

If you don't like listening to the opinion of religious people then say that and get this over with. and nobody is imposing anything on you here.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#347

Post by righttoolforjob »

sterlind wrote: post #339
gonehome wrote: post #305

Earlier quoted context omitted.

Take a smart human and speed up their thinking a billion times, you get the time for free. What takes humanity a decade of effort could happen in a day (or faster) that's the nature of thinking faster. People often mistakenly model a smart human compared to a dumb human, but the actual spectrum of capability is way wider than that. Lack of knowledge could be another way of saying "not aligned" which is the core issue…

that's the thing though, I don't believe superintelligence can possibly work like that. I think it's a logical fallacy to treat superintelligences like hypercomputers. they're constrained by existing on classical computers. like, as a trivial counterexample, I can speak at 500 wpm. let's say that ChatGPT can generate 500 words per second for a single thread of conversation - I think that's a generous overestimate. no…

What is the limit of intelligence? What limits the human intelligence? How many more times intelligent would an AGI have to be to have the capacity to take over the world?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#348

Post by gonehome »

righttoolforjob wrote: post #345
etienne618 wrote: post #343

Earlier quoted context omitted.

I hear you - its just that I dont think that it neccesarily follows that an AGI in whatever form will necesarily be unconstrained or free to improve itself. I also happen to think that the biological systems are in-fact rather special and hard to replicate - especially in terms of efficiency and resillience. Just because some implementation of a universal turing machine can simulate intelligence doesn't mean it can d…

Nobody says that it necessarily follows, but it's a quite trivial step once you have the AGI, hence pointless that you bring up it as an argument.

While I appreciate you find my argument persuasive - imo this style of comment will do more to alienate those replying to me that disagree than to persuade them.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#349

Post by gsala »

layer8 wrote: post #107

Well, he’s right about one thing: > On Feb. 7, Satya Nadella, CEO of Microsoft, publicly gloated that the new Bing would make Google “come out and show that they can dance.” “I want people to know that we made them dance,” he said. > This is not how the CEO of Microsoft talks in a sane world. Certainly this is not the kind of childish priorities the leader of a company as powerful as Microsoft should have.

I think you need the full context. That's from the interview from the Decoder podcast

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#350

Post by righttoolforjob »

kvetching wrote: post #217
oifjsidjf wrote: post #8

Earlier quoted context omitted.

This is the one moment he has been waiting his entire life for. He literally waited his entire life for this moment, obviously he's gonna milk it to every drop.

The thing is, there's no evidence that the current path will lead to superintelligence. It could be the case that despite feeding a trillion dollars in compute / data to a model, that we still need to hold it's hand to do anything useful. He jumped the gun and he's really tarnished his reputation. How can anyone take him seriously after the insane rhetoric and hyperbole of this article?

That's besides the point. The argument still holds. We don't want to find ourselves with our foot on an exploding mine before we stop walking across the minefield. At that point it is too late.
Post reply on HN