Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040 and the cult of intelligence

geohot.github.io

Post reply on HN
Print view View original

81–90 of 280 posts

Re: AI 2040 and the cult of intelligence

#81

Post by pas »

reinitctxoffset wrote: post #49
drc500free wrote: post #32

Earlier quoted context omitted.

If you replace "AI" with "Adonai" in EY's framing, it reduces to biblical parables from his childhood about arguing with God to negotiate a new covenant. He's clearly a bright guy, but a lot of his work seems to be trying to reconcile Old Testament narrative patterns with atheism, and simply slotting an omnipotent AI in as a replacement.

I once heard TESCREAL/EA/etc. called "Calvinism for programmers". Having been raised substantially by a Calvinist grandmother, oh yeah. It's a super weird religion.

sorry, could you explain what Calvinism is for someone who (is a programmer and heard about EA and) grew up in a post-Soviet atheist metallurgy town? thanks!

Re: AI 2040 and the cult of intelligence

#83

Post by reinitctxoffset »

gilrain wrote: post #56
reinitctxoffset wrote: post #41

Earlier quoted context omitted.

It'll be like DNS or dynamic linking or torrents of media or watching TLS on your own box: only people motivated to be free from corporate surveillance will be. By default a megacorp will get everything you see online or ask online, they will run code on your computer underneath you without telling you, they'll collaborate with arbitrarily totalitarian governments, we already know what they do by default. But we also…

> … like those ICE guys terrorizing all the Latino guys. … and the white guys they don’t like, like nurses with empathy. Don’t you dare think you’re safe because of your color. Unless you’re a monster or willing to become one, they want you oppressed.

Most people are neither exceptionally good nor exceptionally bad. This sounds like a tautology but it's worth printing some type signatures even if they could be inferred.

If you look at any truly large scale Holocaust, whether in Rwanda in 92 or the Third Reich, you find that a few people will sacrifice themselves to defend others, a few will be monsters at the first opportunity, and most will be steered by whether the leaders are the first or second kind of people.

Re: AI 2040 and the cult of intelligence

#84

Post by emp17344 »

piker wrote: post #26

What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”

This is really only meaningful if AI imparts knowledge beyond what you could learn from a simple Google search. It’s not clear that this is the case.

Re: AI 2040 and the cult of intelligence

#85

Post by Dylan16807 »

skybrian wrote: post #52
gilrain wrote: post #46

Earlier quoted context omitted.

> Seems like being directionally right early that AI would be a Big Deal and scary should count for something? Only if it was an uncommon prediction; otherwise, it’s just evidence of common sense.

It's common sense now . It hasn't always been common sense that AI alignment is an important problem.

If we do a survey of stories about AI from the last 50 years, it seems like it's a worry everyone knows about, and that people generally take seriously in proportion to how much they think an AI will actually exist.

Re: AI 2040 and the cult of intelligence

#86

Post by emp17344 »

andy99 wrote: post #45

It will be interesting to see how the discussion shakes out, this is all stuff that would get you down modded and talked down to if you said if on HN, so far the reactions are positive but I think there’s a massive authoritarian / safetyism contingent here that will disagree hard with almost everything.

Way too many rationalist-adjacent cultists here who worship Yudkowsky as a prophet.

Re: AI 2040 and the cult of intelligence

#87

Post by tancop »

jdw64 wrote: post #35

I sometimes wonder what freedom really is. Should the freedom to harm others also be guaranteed as freedom? Individual computational sovereignty versus the prevention of social harm is always contentious, and claiming that something is always right is always extremist. If it is "intelligence," is it not natural to reject that? I also think local models should adapt to me when it comes to safety issues, but people bri…

> Should the freedom to harm others also be guaranteed as freedom?

everyone should have the ability to harm others, with the understanding that if they actually use it for evil there will be consequences. as in "i can guarantee freedom of speech, but not freedom after speech."

its the same principle as guns in america. you can own a gun and shoot it as much as you want but if you (try to) kill someone you go to prison unless you prove it was self defense.

the difference is with a physical weapon you can do things like registration or red flag laws to reduce the risk, but a llm is made of information that can be copied without anyone knowing. that means any laws to regulate ai at the level of models are unenforceable without totalitarian control over all computer hardware. that would kill free speech and real criminals would still find a way.

i think the right answer is to give up control over model capabilities and regulate uses. leave individual people and communities out of it. create an absolute safe harbor for open source and self hosting.

as you said, we need to focus on the ways ai concentrates power in the hands of those who already have a lot of it - big business and the state. that is where the biggest, and maybe the only, real harm comes from.

Re: AI 2040 and the cult of intelligence

#88

Post by drdaeman »

skybrian wrote: post #52
gilrain wrote: post #46

Earlier quoted context omitted.

> Seems like being directionally right early that AI would be a Big Deal and scary should count for something? Only if it was an uncommon prediction; otherwise, it’s just evidence of common sense.

It's common sense now . It hasn't always been common sense that AI alignment is an important problem.

Why? Science fiction talked about technology’s behavior and human expectations of it even way before computers became mechanical.

Stories about golems going rogue and genies requiring careful wishing aren’t modern era inventions.

Re: AI 2040 and the cult of intelligence

#89

Post by kristjansson »

ben_w wrote: post #67
mlinsey wrote: post #51

I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction…

> It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc. I'm neither American nor a lawyer. Is "conspiracy" protected under the first amendment? If you discuss a crime with someone to learn about it, does that count as "conspiracy"?

GP is correct, books describing, advocating, even instructing crime in the abstract are almost surely protected speech.

Re: AI 2040 and the cult of intelligence

#90

Post by majormajor »

pas wrote: post #81
reinitctxoffset wrote: post #49

Earlier quoted context omitted.

I once heard TESCREAL/EA/etc. called "Calvinism for programmers". Having been raised substantially by a Calvinist grandmother, oh yeah. It's a super weird religion.

sorry, could you explain what Calvinism is for someone who (is a programmer and heard about EA and) grew up in a post-Soviet atheist metallurgy town? thanks!

Calvinism is a variant of Christianity with the following tenets ( https://en.wikipedia.org/wiki/Five_Points_of_Calvinism ):

- everyone is damned and depraved

- god chose some people to save, at his own whim, for no reasons related to their actions

- jesus' death was a sacrifice to atone for those people, not for everyone

- the chosen can't opt-out (I'm merging two into one here)

I don't really see the connection to effective altruism at all. The way I grew up understanding it is actually more the opposite?

Post reply on HN