Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

131–140 of 280 posts

Re: AI 2040 and the cult of intelligence

#131

Post by ben_w »

majormajor wrote: post #102
skybrian wrote: post #34

Earlier quoted context omitted.

"Consistently wrong" seems a bit much. Seems like being directionally right early that AI would be a Big Deal and scary should count for something? It doesn't mean any details or other predictions are right, though.

It's not a particularly novel claim since at least Terminator (and even moreso Terminator 2) made it EXTREMELY mainstream. It was a topic in less-mainstream sci-fi well before that. And some more mainstream stuff like Star Trek TOS. Frankly it seems more common than not in the last 40 years. I don't really remember a big wave of claims "Terminator is silly, no sort of AI could ever be malevolent!"

One thing I find weird is that despite the stock photo of the Terminator's shiny endoskeleton holding a gun being used to illustrate so many AI news stories, now we have ChatGPT people keep saying "how can AI possibly hurt us?"

We've even had the dichotomy of some Doctor Who episodes where the self-driving car crashes itself to kill the occupant to silence them, against repeated real-life news stories about self-driving cars killing people, and yet the connection isn't getting made that software controls hardware.

Re: AI 2040 and the cult of intelligence

#132

Post by [deleted] »

piloto_ciego wrote: post #115
Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

The cynicism and misanthropy is nonsense and a direct result of the internet/social media landscape. It's BS. In the current media ecosystem, what gets you "social cache" or whatever? Let's call them "cool points." Well, right now, the current metric is basic "likes" or "retweens" or "upvotes" or whatever. And to get those, you have to make a claim and then that claim has to be evaluated by others. But the evaluation…

[deleted]

Re: AI 2040 and the cult of intelligence

#135

Post by ben_w »

thrw045 wrote: post #78
skybrian wrote: post #52

Earlier quoted context omitted.

It's common sense now . It hasn't always been common sense that AI alignment is an important problem.

It depends on what you mean by alignment. If you mean the sense it's used now, meaning bias in models, security and safety for users and so on, then that's a way more mundane version of alignment than what Yudkowsky and ilk promoted. They are talking about a superintelligent being possibly destroying humanity / the earth. And it wasn't that hard to predict. We have many examples of more technological / "advanced" peo…

> When GPT-4 came out, Yudkowsky said it might be conscious. I think he has written interesting stuff but let's be real

I don't expect GPT-4 to be conscious, it would be quite remarkable if we've stumbled upon whatever evolution did that made us conscious.

But how would you even tell, given we don't know the mechanism or physical process that un-thinking and un-planning natural selection happened to stumble upon to give it to us?

Re: AI 2040 and the cult of intelligence

#136

Post by hirako2000 »

charcircuit wrote: post #22
ThrowawayR2 wrote: post #3

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…

It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.

Safetensors don't execute code. A call would need function calling, as other comment say, it would quickly be noticed.

Re: AI 2040 and the cult of intelligence

#137

Post by 8bitsout »

paul7986 wrote: post #39
Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

He talks about AI cutting through popups, but if you follow that line of thinking further, screens and traditional websites likely fade in importance. It becomes more talking and texting to AI and visuals on the topic at hand appear in smart glasses, lock-screen–centric AI phones (where website visits dwindle), digital picture frames, TVs, etc. I sorta gather (just a hunch) this is the type of device Open AI is worki…

I'm curious how the value of your website would be determined in your model of the future and why an AI (at least one of the cloud provider models) wouldn't just persist your data after accessing it once. In other words, the way I am understanding it is that your data has a package value. An AI, if it wanted to access the entire contents, pays the whole package value. And every time it does this, you get paid. So, to really benefit, your data needs some sort of minimum value that is accessed at an appropriate frequency.

Also, in this model Cloudflare basically becomes the centralized gate keeper of you getting paid. And to me, this kind of sucks. It's not a very productive line of thought on my end -- I could see something like this happening as the internet trends toward centralization over time -- but it just feels bad.

Re: AI 2040 and the cult of intelligence

#138

Post by drdaeman »

Jtarii wrote: post #107
jchw wrote: post #77

Earlier quoted context omitted.

That's a bit literal of an interpretation. Even acknowledging freedom as a spectrum that can never truly be absolute, there's gotta be a threshold you cross where you'd say "OK, we no longer have basic freedoms." And not only are we definitely screaming towards that threshold, but people seem thrilled about it, for a variety of reasons. All I can say to that is I guess I hope everyone enjoys the world they think they…

The article seems to be taking a very literal interpretation of "freedom" as in any restriction on what a local LLM can do is totalitarian dystopia. I don't think it should be legal to get an LLM to research how to make a bioweapon in your basement, that seems like a bad idea.

And there’s where some people disagree.

IMHO, one should be able to research absolutely anything, for any reason. “Forbidden knowledge” shouldn’t exist as a category, freedom of thought should be protected, even if that thought is intended to cause harm.

What shouldn’t be possible (and obviously not legal as well) is building that bioweapon.

Re: AI 2040 and the cult of intelligence

#139

Post by Planktonne »

ks2048 wrote: post #128

I, for one, think the ChatGPT response to, "Hey, I just killed my wife ..." is not bad and preferable to it helping him.

For society to function, we remove some freedoms. "Getting support with murdering women" feels like a reasonable thing to curtail.

Re: AI 2040 and the cult of intelligence

#140

Post by c1ccccc1 »

Planktonne wrote: post #17

> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming. I really think we need to stop giving credence to people who have 1) Been consistently wrong with all their predictions 2) Demonstrated an endless spiteful cynicism Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk…

> they things they believe in are unlikely to happen.

Sorry, but most times I've talked to someone who says this (about AI completely replacing humanity), they don't have anything to say about why it's unlikely to happen other than:

- "well, it's just such an extreme outcome, it must be improbable"

- "humanity has survived near scrapes with extinction before"/"all the previous doomsday predictions have been wrong"

There is a cliche that all teenagers deep down believe that they are invincible. It seems to me that humanity is still a teenager in this respect: We don't take seriously the possibility of our own extinction. While one might think that the invention of nuclear weapons would serve as a wake-up call, if anything it has done the opposite.

I'm willing to hear arguments besides the two above, if you have them. (And to be clear, being replaced by AI doesn't necessarily mean being replaced by LLMs in particular. They are a relatively new development.)

Post reply on HN