Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040: Plan A

ai-2040.com

Post reply on HN
Print view View original

521–530 of 567 posts

Re: AI 2040: Plan A

#521

Post by huhkerrf »

taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

> I grew up in evangelical christianity, and to them the end of the world is just around the corner

I grew up in evangelical Christianity, I'm still an evangelical Christian, and you're wrong to paint with a broad brush like that. Are there evangelicals that are millenarian? Yep, of course.

But that is not my experience across multiple evangelical churches across decades.

Re: AI 2040: Plan A

#522

Post by oliculipolicula »

ToValueFunfetti wrote: post #320
JumpCrisscross wrote: post #317

Earlier quoted context omitted.

> GP just dismissed it as religion without engaging with a word of the material Fair enough. I didn’t see anything novel in the article. So treating it as a motif within the abovequoted “Superintelligence: The Idea That Eats Smart People” context is fair and a real argument. > Cynicism Cynicism isn’t the opposite of blind optimism. Nihilism is. I’m not seeing a rejection of the article as being baseless as cynical or…

I hate to get bogged down in semantics, but with the hope that one of the stronger top-level critiques makes it into the top slot here and this conversation gets buried: Cynicism is defined as >An attitude of scornful or jaded negativity, especially a general distrust of the integrity or professed motives of others. I'm not saying that cynicism is automatically wrong, just that I once could trust that, when HN is wro…

>..Nothing in life ... has any chance of succeeding without strenuous practice; and this is capable of overcoming anything..

---Diogenes (L) on Diogenes, arch-Cynic, VI.2.71

Afaik not formalised in Lean4, but sounds like optimism

Most of D's other stuff imho is optimism taken to its clear-eyed extreme. ya know. the way a very best version of us could always unironically RELY on HN [,after much practice]

>so those whose training has been of the opposite kind derive more pleasure from despising pleasure than from the pleasures themselves.

Might be why professional Nihilists admire Cynics (ie, for their ... unaffected comedy)

https://penelope.uchicago.edu/Thayer/E/Roman/Texts/Diogenes_...

Re: AI 2040: Plan A

#523

Post by tim333 »

beshrkayali wrote: post #285
skrebbel wrote: post #268

Earlier quoted context omitted.

It's nuts how well "Superintelligence: The Idea That Eats Smart People"[0] aged. That talk is a decade old by now and still hits just as hard as it did back then, despite the incredible advances made in AI in the meantime. [0] https://idlewords.com/talks/superintelligence.htm

This is hilariously true > AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel co…

AI is "completely inaccessible to experiment given our current technology"?

I think experiments have happened and even been mentioned on HN occasionally and effected RAM prices.

Re: AI 2040: Plan A

#524

Post by tim333 »

skrebbel wrote: post #268
taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

It's nuts how well "Superintelligence: The Idea That Eats Smart People"[0] aged. That talk is a decade old by now and still hits just as hard as it did back then, despite the incredible advances made in AI in the meantime. [0] https://idlewords.com/talks/superintelligence.htm

I always thought it a bit pessimistic on the upsides like robot to build us palaces, immortality, conquer the galaxy type stuff along the lines of the "Grandiosity" bit in the essay. It's a shame we have amazing tech possibilities and most people are either saying either it's rubbish and won't work or it'll bring doom. Aside from being boring they are probably factually wrong.

Bit like saying early medical knowledge was rubbish or will doom us by summoning demons while really life expectancy at birth went from twenty something in Roman times to eighty odd now.

Re: AI 2040: Plan A

#525

Post by pnt12 »

Aurornis wrote: post #471
bwanab wrote: post #456

Earlier quoted context omitted.

The other thing about this article is that it contains falsifiable claims that are, well, false. Consider the following direct quote: "The 2028 election cycle is heated, as usual. AI is the biggest topic." No poll of the American public has AI as one of the top concerns except in polls where the question is specifically about AI. The general polls show topics like healthcare costs, inflation/affordability, government…

The AI 2040 writing and the AI 2027 before it both sprang from within the rationalist communities. Scott Alexander (of the popular rationalist Slate Star Codex / Astral Codex Ten blog) played a role in both articles, though he says he chose to leave his name off of this one. AGI and AI doom scenarios have been evergreen topics in rationalism for years. Long before we had enough compute power to do anything at all use…

First I was intrigues by the rationalist movement, but I was deeply disappointed. To put it bluntly, seems to me like geeks LARPing as philosophers, and not with rational argument as they promise, but rationalizing their sci fi fanfictions.

Re: AI 2040: Plan A

#526

Post by nradov »

fwipsy wrote: post #520
nradov wrote: post #478

Earlier quoted context omitted.

Right, the so-called "AI safety researchers" are worthless clowns and grifters with nothing of value to contribute. We would all be better off if they spent their time on something productive like writing entertaining sci-fi stories. They certainly seem to have the rich imagination necessary for that occupation.

Now this is an unserious comment, just throwing insults without explaining your reasoning at all. Do you believe that AI itself is worthless? Or that it's just incapable of causing harm? Do you resent the safeguards placed on AI because they are personally inconvenient to you? When introducing a new technology at this scale, don't you think it's worth looking before leaping?

So called "AI safety researchers" are themselves unserious so of course I don't take them seriously. LLMs are extremely useful, I use them constantly for a wide variety of tasks. LLMs don't cause harm: humans cause harm. There's no real need to intentionally cripple or restrict the models in the name of "safety"; that's just stupid and done only to prevent corporate embarrassment.

Re: AI 2040: Plan A

#527

Post by stalfie »

nadir_ishiguro wrote: post #510
stalfie wrote: post #509

Earlier quoted context omitted.

No one knows how LLMs work. We know how the architecture works, but almost nothing about why. Saying "statistical next token prediction" tells you about as much about LLMs as saying "action potential thresholds" tells you about the brain. A true fact that explains very little. And I'm sorry, but you're not up to date about interpretability literature, or for that matter philosophical discourse, if you think you have…

"And I'm sorry, but you're not up to date about interpretability literature, or for that matter philosophical discourse," I see no citations referring to the current philsophical discourse, unless you mean to imply anthropic's paid people are to be considered to be part of that. That they "see structures in Claude akin to those we think are associated with human consciousnesses experience in the brain" is if anything…

Sure, here are some citations:

https://arxiv.org/abs/2401.03910?utm_source=chatgpt.com https://www.frontiersin.org/journals/psychology/articles/10.... https://arxiv.org/pdf/2408.04666 https://arxiv.org/abs/2402.00901 https://ar5iv.labs.arxiv.org/html/2407.11015 https://ar5iv.labs.arxiv.org/html/2202.05262 https://www.sciencedirect.com/science/article/abs/pii/S13646...

My point was not to argue one point or another about LLM intelligence, but to push against the notion that you have to be "delusional" to even argue that it is possible that LLMs can qualify as intelligent (although the op prefaced it with "actually"). That's mainly what irked me about the original comment, the arrogance of dismissing everyone even having the discussion as insane, as if there's no legitimate argument to be made.

And this was mainly the point I was arguing. However since we're on the topic, I also happen to think it's intellectually lazy to dismiss the Anthropic interpretability teams work as "delusional" simply because they have a conflict of interest. Of course, that is not an irrelevant fact, but much of their original work has since been replicated by independent entites (eg. https://arxiv.org/abs/2510.01246). Until they publish something that turns out to be fraudulent, I think it's reasonable to consider Anthropic's paid people a very relevant, and in fact excellent part of interpretability discourse.

Dismissing all opinions where there is a perceived conflict of interest is a pleasant cognitive bias to have, but reality is often more nuanced than that.

Re: AI 2040: Plan A

#528

Post by nonadhocproblem »

sajithdilshan wrote: post #290
nonadhocproblem wrote: post #248

Earlier quoted context omitted.

Are you trolling? OpenAI's models resolved the Cycle Double Conjecture yesterday, which has been an open problem for 50 years.

Do you even know what Cycle Double Conjecture is? Or what even a conjecture means in general? It’s a perfect use case for autocomplete with existing knowledge

Please seek help. These levels of copium can't possibly be healthy.

Re: AI 2040: Plan A

#529

Post by angoragoats »

tim333 wrote: post #523
beshrkayali wrote: post #285

Earlier quoted context omitted.

This is hilariously true > AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel co…

AI is "completely inaccessible to experiment given our current technology"? I think experiments have happened and even been mentioned on HN occasionally and effected RAM prices.

> AI is "completely inaccessible to experiment given our current technology"?

No, that’s not what the article/talk said. It said AI _risk_ is inaccessible (as the poster you’re replying to correctly includes in their quote). The point in the original talk is clear if you read it; that everyone coming up with their own pet theories for how superintelligent AI could destroy us as a species is building their theory on speculation on top of speculation.

Note that the talk is from 2016, before transformers and LLMs were invented. Though I would argue that the vast majority of it, including the parts we’re discussing here, are still completely valid.

Re: AI 2040: Plan A

#530

Post by ToValueFunfetti »

oliculipolicula wrote: post #522
ToValueFunfetti wrote: post #320

Earlier quoted context omitted.

I hate to get bogged down in semantics, but with the hope that one of the stronger top-level critiques makes it into the top slot here and this conversation gets buried: Cynicism is defined as >An attitude of scornful or jaded negativity, especially a general distrust of the integrity or professed motives of others. I'm not saying that cynicism is automatically wrong, just that I once could trust that, when HN is wro…

>..Nothing in life ... has any chance of succeeding without strenuous practice; and this is capable of overcoming anything.. ---Diogenes (L) on Diogenes, arch-Cynic, VI.2.71 Afaik not formalised in Lean4, but sounds like optimism Most of D's other stuff imho is optimism taken to its clear-eyed extreme. ya know. the way a very best version of us could always unironically RELY on HN [,after much practice] > so those wh…

Ah, this clarifies things. I suppose I brought this on myself by putting it at the start of the sentence, but I do mean little-c cynicism, not the ancient philosophy. I was surprised to be called out on it, as GP is a shining example, and I was confused by the mention of nihilism, but I see now where wires were crossed. Thanks!
Post reply on HN