Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

151–160 of 280 posts

Re: AI 2040 and the cult of intelligence

#151

Post by paul7986 »

8bitsout wrote: post #137
paul7986 wrote: post #39

Earlier quoted context omitted.

He talks about AI cutting through popups, but if you follow that line of thinking further, screens and traditional websites likely fade in importance. It becomes more talking and texting to AI and visuals on the topic at hand appear in smart glasses, lock-screen–centric AI phones (where website visits dwindle), digital picture frames, TVs, etc. I sorta gather (just a hunch) this is the type of device Open AI is worki…

I'm curious how the value of your website would be determined in your model of the future and why an AI (at least one of the cloud provider models) wouldn't just persist your data after accessing it once. In other words, the way I am understanding it is that your data has a package value. An AI, if it wanted to access the entire contents, pays the whole package value. And every time it does this, you get paid. So, to…

Thank you for reading and just one idea from one human. But our content keeps Facebook, Google and the digital world spinning. Further in our daily lives we all create content which parts of it or a lot of it is content that Facebook, Google, etc make multiples of billions off of. Why are we not making that money off it ourselves is my thinking. It might not be a lot of money but it would be one way of many that humans get paid for keeping AI relevant.

Though this human would be fine if the universe just unplugged AI and I could go back to working as a well paid UX Engineer/Designer where jobs were plentiful and it was easy to get a job (not have to go through 15 interviews). Thanks AI, but as a startup person I am deep into vibe coding now all my crazy ideas and without having to lean on others (so far). That's pretty cool!

Re: AI 2040 and the cult of intelligence

#152

Post by thrw045 »

ben_w wrote: post #135
thrw045 wrote: post #78

Earlier quoted context omitted.

It depends on what you mean by alignment. If you mean the sense it's used now, meaning bias in models, security and safety for users and so on, then that's a way more mundane version of alignment than what Yudkowsky and ilk promoted. They are talking about a superintelligent being possibly destroying humanity / the earth. And it wasn't that hard to predict. We have many examples of more technological / "advanced" peo…

> When GPT-4 came out, Yudkowsky said it might be conscious. I think he has written interesting stuff but let's be real I don't expect GPT-4 to be conscious, it would be quite remarkable if we've stumbled upon whatever evolution did that made us conscious. But how would you even tell, given we don't know the mechanism or physical process that un-thinking and un-planning natural selection happened to stumble upon to g…

Sure, we can't tell, but that's a really bad argument to make the claim that it _is_ conscious. I just have a really hard time believing that a transformer can be conscious, but yes I can't prove it.

Re: AI 2040 and the cult of intelligence

#153

Post by ben_w »

thrw045 wrote: post #152
ben_w wrote: post #135

Earlier quoted context omitted.

> When GPT-4 came out, Yudkowsky said it might be conscious. I think he has written interesting stuff but let's be real I don't expect GPT-4 to be conscious, it would be quite remarkable if we've stumbled upon whatever evolution did that made us conscious. But how would you even tell, given we don't know the mechanism or physical process that un-thinking and un-planning natural selection happened to stumble upon to g…

Sure, we can't tell, but that's a really bad argument to make the claim that it _is_ conscious. I just have a really hard time believing that a transformer can be conscious, but yes I can't prove it.

Agreed. I push back equally against all who are confident, regardless of which way.

Re: AI 2040 and the cult of intelligence

#154

Post by johnfn »

fasterik wrote: post #146
johnfn wrote: post #142

Earlier quoted context omitted.

The discussion isn't about whether it's an "original or interesting thought", it's about whether Yudkowsky is "consistently wrong with all their predictions". You keep shifting the topic.

I was mainly responding to the claim that he was "saying that AI was going to be a serious problem decades before anyone else was even considering it as a remote possibility." But we can also look at the specific predictions he's made. https://www.longtermwiki.com/wiki/E643 Some that turned out to be wrong include: singularity by 2021, singularity by 2025, 70% chance of human extinction between 2003-2015, and that hi…

I was responding to someone who said Yudkowsky was "consistently wrong with all their predictions". Yes, fair enough, Yudkowsky wasn't the literal first person ever to say that AI might be bad. The point was that the common academic response at the time if you were to say AI could be bad was to laugh you out of the room. IMO, you get a lot of points for making a prediction when almost everyone else in the world disagrees with you. The reason those movies you cited were blockbusters was because very view people believed that they were realistic. No one made a movie about a pandemic in 2020.

I don't think there's much to be gained out of hashing out whether every prediction Yudkowsky has said was right or wrong: I likely directionally agree with you there, as I also find some of his more extreme predictions to be inaccurate. I mostly take issue with "consistently wrong". The results in longtermwiki are not "consistently wrong". He's definitely wrong sometimes. But consistently?

Re: AI 2040 and the cult of intelligence

#155

Post by trescenzi »

majormajor wrote: post #90
pas wrote: post #81

Earlier quoted context omitted.

sorry, could you explain what Calvinism is for someone who (is a programmer and heard about EA and) grew up in a post-Soviet atheist metallurgy town? thanks!

Calvinism is a variant of Christianity with the following tenets ( https://en.wikipedia.org/wiki/Five_Points_of_Calvinism ): - everyone is damned and depraved - god chose some people to save, at his own whim, for no reasons related to their actions - jesus' death was a sacrifice to atone for those people, not for everyone - the chosen can't opt-out (I'm merging two into one here) I don't really see the connection to…

There’s two kinds of Effective Altruism(EA). The kind Peter Singer and others promoted for decades and the kind that’s now part of Silicon Valley and finance. The Peter Singer kind basically just says “reduce the most suffering you can” and the cheapest way to maximize that is stuff like mosquito nets. That was all well and good.

When EA met Silicon Valley the basics of utilitarianism, minimize suffering, merged with the standard exponential curve that Silicon Valley loves. It went from mosquito nets prevent malaria for pennies to the future is infinite therefore there will always be more people in the future than the present so minimizing future peoples suffering is key. This is where it starts to combine with super intelligence. If there is a super intelligence that destroys humanity that would cause _maximum suffering_. Therefore the most effective way to reduce suffering is actually to prevent said super intelligence from destroying/torturing humanity. Again because there’s more future people than current people.

This is how it becomes Calvinist. The present has no value. Only the chosen, those working in AI, can save the future. And you must do everything you can to prevent said future.

It can go both ways. Either you believe ASI will always be bad so you believe work must be done to prevent it. Or you believe that ASI can be made good and the good kind will maximize utility. They are both the same though fundamentally and are a belief in a higher power, ASI, which either provides ultimate damnnation and must be averted or provides ultimate salvation and must be brought into existence.

If you’re donating mosquito nets or reading up on the best charities in your area that’s not Calvinist. If you’re devoting your life to AI because you believe that’s how you can be most altruistic it starts to look something like a religion and Calvinism is an imperfect but useful analog.

Re: AI 2040 and the cult of intelligence

#156

Post by 8bitsout »

paul7986 wrote: post #151
8bitsout wrote: post #137

Earlier quoted context omitted.

I'm curious how the value of your website would be determined in your model of the future and why an AI (at least one of the cloud provider models) wouldn't just persist your data after accessing it once. In other words, the way I am understanding it is that your data has a package value. An AI, if it wanted to access the entire contents, pays the whole package value. And every time it does this, you get paid. So, to…

Thank you for reading and just one idea from one human. But our content keeps Facebook, Google and the digital world spinning. Further in our daily lives we all create content which parts of it or a lot of it is content that Facebook, Google, etc make multiples of billions off of. Why are we not making that money off it ourselves is my thinking. It might not be a lot of money but it would be one way of many that huma…

Ok, I hear you :). I share similar sentiments myself.

Re: AI 2040 and the cult of intelligence

#157

Post by plorkyeran »

johnfn wrote: post #109
fasterik wrote: post #108

Earlier quoted context omitted.

No, they didn't say the same thing. Identifying potential risks is not the same thing as confidently predicting the extinction of humanity.

We agree the guy roughly said AI will be dangerous. We agree AI is dangerous. Not sure what more there is to say here.

He made specific predictions about what problems AI would cause, not just a vague "AI will be dangerous". Those specific predictions were entirely incorrect; they both said that things would happen that didn't, and failed to foresee any of the actual problems with AI that have happened.

Re: AI 2040 and the cult of intelligence

#158

Post by slopinthebag »

Jtarii wrote: post #48

>Like we either live in a world with freedom or we don’t This is something I would expect a 12 year old to say. Constraints on your freedom are literally everywhere in every interaction you have with every part of society.

When people say this they’re most likely referring to negative liberty - freedom from external coercion and interference. This is not the same as positive liberty - “I can do whatever I want”.

I think it’s self-evident there is a difference between having the “freedom” to punch random people on the sidewalk, and having the freedom to walk down the sidewalk without being randomly punched. Now replace walking down the sidewalk with using an uncensored LLM and it might be clearer what freedom means in this context.

Re: AI 2040 and the cult of intelligence

#159

Post by jchw »

Jtarii wrote: post #107
jchw wrote: post #77

Earlier quoted context omitted.

That's a bit literal of an interpretation. Even acknowledging freedom as a spectrum that can never truly be absolute, there's gotta be a threshold you cross where you'd say "OK, we no longer have basic freedoms." And not only are we definitely screaming towards that threshold, but people seem thrilled about it, for a variety of reasons. All I can say to that is I guess I hope everyone enjoys the world they think they…

The article seems to be taking a very literal interpretation of "freedom" as in any restriction on what a local LLM can do is totalitarian dystopia. I don't think it should be legal to get an LLM to research how to make a bioweapon in your basement, that seems like a bad idea.

I just don't subscribe to the belief that knowledge itself is dangerous. Well, that's a bit of a cop-out; a lot of things aren't inherently dangerous but that doesn't mean they must never ever be regulated, but regulating knowledge and information under the belief that it is essentially inherently dangerous to have access to it is truly as cut-and-dry as it comes. We already know where that ends.

Re: AI 2040 and the cult of intelligence

#160

Post by Aurornis »

mlinsey wrote: post #51

I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction…

> The AI can and should refuse it.

This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes.

Which is precisely what we’re going through right now with frontier models and it’s terrible.

These proposals always assume some perfect mechanism for identifying the thought crime with triggering on the normal requests. The people who want to commit crimes just persist until they jailbreak the guardrails while the rest of us suffer with denials.

Also if you can’t imagine these guardrails being used by governments to control inconvenient speech, you probably need to think a little harder about the realities of how these will be used.

Post reply on HN