Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040: Plan A

ai-2040.com

Post reply on HN
Print view View original

491–500 of 567 posts

Re: AI 2040: Plan A

#491

Post by gabriel-uribe »

gamblor956 wrote: post #482
gabriel-uribe wrote: post #196

Earlier quoted context omitted.

Wasn't writing about major companies. That's obviously next, if we follow the trend lines.

Most companies have already given up on AI. It was a bigger disappointment than big data. The only companies sticking with AI at this point are the major players who have chasing their masssive overinvestments to the bitter end, crappy coding shops, and for some reason Starbucks and they'll all suffering badly in a year their customers begin demanding massive amounts of human coding to repair all the issues with the…

We're still in the early adoption phase. Betting against the internet wasn't a great idea, and betting against AI doesn't look like one either ;)

Re: AI 2040: Plan A

#492

Post by m4rtink »

0xDEAFBEAD wrote: post #436
beshrkayali wrote: post #285

Earlier quoted context omitted.

This is hilariously true > AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel co…

Theorizing about nuclear winter is somewhat similar, in the sense of being inaccessible to experiment. Does that mean we should disregard the possibility of nuclear winter?

As with using the Orion drive for launch and landing on Erth is possible, test nuclear winter is certainly possible. But as with the former, you won't be very popular among the survivors (if any).

Re: AI 2040: Plan A

#493

Post by sudb »

27183 wrote: post #469
sudb wrote: post #451

Earlier quoted context omitted.

> To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. This is not at all what I was saying. I think you've already conceded that LLMs demonstrate emergent behaviour but you dismissed it as a "close enough facsimile to intelligence". I was saying that the emergent behaviour is reliably replicable, in response to your following statement: > Figure out how to make it happen reliably a…

> I think you've already conceded that LLMs demonstrate emergent behaviour No. Please don't put words in my mouth. What I said is that an LLM compresses a bunch of information into a semantic embedding space and then does sort of a stochastic search in that embedding space. Any similarity to "intelligence" is accidental. You may look at the results of that process and "see" thinking or reasoning or something, but it…

> but it ain't there

On what grounds? I don't think you've provided any evidence other than LLMs can't "adapt" or "learn" to show that LLMs do not show intelligence in any way. I think it's clear that there must be some emergent form of intelligence over words from just the agentic coding ability alone. I am not claiming that LLMs are intelligent, only that they display aspects of what we understand as intelligence.

> I don't think LLMs need to be intelligent to be (at least narrowly) useful

I agree! But they are more than narrowly useful, and they absolutely do not belong in the same category as random forests or genetic algorithms!

> Go collect your Nobel prize then! This is no longer a discussion grounded in reality

Once again you are being condescending while misrepresenting my position. The emergent aspects of "intelligence" have been replicated by virtue of independent LLM vendors training their own models - I am not making a stronger claim, you have misunderstood me.

Thanks for participating.

Re: AI 2040: Plan A

#494

Post by m4rtink »

0xDEAFBEAD wrote: post #438
zdragnar wrote: post #433

Earlier quoted context omitted.

Fable 5 doesn't represent anything new, other than scale and some refinement techniques, over the original LLMs. In the chase for AGI specifically, LLMs are a dead end, just like all the other AI technologies that died in the AI winter. What priors should be updated?

>Fable 5 doesn't represent anything new, other than scale and some refinement techniques, over the original LLMs. Yet it is generating billions in revenue which Eliza did not. Perhaps all we need is scale and some refinement techniques to eat a big fraction of the economy. If unimpressive inputs lead to impressive outputs, that should make you more worried, not less.

And possibly trilions in running costs, not mentioning all the shady training data sources.

Re: AI 2040: Plan A

#495

Post by throwaway27448 »

brookman64k wrote: post #270
throwaway27448 wrote: post #220

Earlier quoted context omitted.

It's not clear that "super intelligence" is a meaningful concept. This presumes that our concept of intelligence can continue to grow beyond human capacity as opposed to asymptotically approaching what humans recognize as "very intelligent". Perhaps, for instance, how we evaluate intelligence is bounded not by some quantitative capacity but rather our inability to agree on basic concepts/values. And do we even have t…

The Orthogonality Thesis (by Nick Bostrom) says that intelligence and ultimate goals are independent. Those non-instrumental goals can’t be stupid nor right or wrong. Increasing intelligence will not change the goals only the capability to reach them.

Ok so what is the point of neurotically obsessing about intelligence? Would it not be more fruitful to emphasize computation?

Re: AI 2040: Plan A

#496

Post by heyts »

fwipsy wrote: post #381
taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was…

> they were effective altruists who chose AI safety research as the most effective way to improve the world

This actually made me laugh, I’m sorry

Re: AI 2040: Plan A

#497

Post by defgeneric »

iugtmkbdfil834 wrote: post #473
defgeneric wrote: post #448

Earlier quoted context omitted.

It's a form of denial. We're getting another "de-thronement of man" on the order of Copernicus and Darwin. Some get excited, others turn away in horror. Negation is the outward expression of the desire to keep human intelligence wrapped in its mystical veil. One popular idea is that these systems will asymptotically approximate human intelligence because they're trained on mostly human-written texts. Not only is that…

Can you elaborate on this? I am clearly not aware of this line of thinking and the related contradiction.

Approximate as a limit and not surpass, where the hidden hope actually seems to be that they don't surpass. There are lots of variations floating around, from simple metaphors like "LLM as librarian that speaks to you," to even Sutton's remark that rather than being a case where we've taken the "bitter lesson" to heart, LLMs may be a yet another case where we're limited by it.

Re: AI 2040: Plan A

#498

Post by ThrowawayR2 »

Aurornis wrote: post #471
bwanab wrote: post #456

Earlier quoted context omitted.

The other thing about this article is that it contains falsifiable claims that are, well, false. Consider the following direct quote: "The 2028 election cycle is heated, as usual. AI is the biggest topic." No poll of the American public has AI as one of the top concerns except in polls where the question is specifically about AI. The general polls show topics like healthcare costs, inflation/affordability, government…

The AI 2040 writing and the AI 2027 before it both sprang from within the rationalist communities. Scott Alexander (of the popular rationalist Slate Star Codex / Astral Codex Ten blog) played a role in both articles, though he says he chose to leave his name off of this one. AGI and AI doom scenarios have been evergreen topics in rationalism for years. Long before we had enough compute power to do anything at all use…

If I recall correctly, the text of Yudkowsky's response to Roko's Basilisk used to be part of the Wikipedia page but was removed and not even a reference link was provided even though the response was mentioned in the body of the article. For posterity's sake, from https://rationalwiki.org/wiki/Roko%27s_basilisk/Original_pos... here it is in all its hilariously unhinged glory:

"One might think that the possibility of CEV punishing people couldn't possibly be taken seriously enough by anyone to actually motivate them. But in fact one person at SIAI was severely worried by this, to the point of having terrible nightmares, though ve wishes to remain anonymous. I don't usually talk like this, but I'm going to make an exception for this case.

Listen to me very closely, you idiot.

YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL.

There's an obvious equilibrium to this problem where you engage in all positive acausal trades and ignore all attempts at acausal blackmail. Until we have a better worked-out version of TDT and we can prove that formally, it should just be OBVIOUS that you DO NOT THINK ABOUT DISTANT BLACKMAILERS in SUFFICIENT DETAIL that they have a motive to ACTUALLY BLACKMAIL YOU.

If there is any part of this acausal trade that is positive-sum and actually worth doing, that is exactly the sort of thing you leave up to an FAI. We probably also have the FAI take actions that cancel out the impact of anyone motivated by true rather than imagined blackmail, so as to obliterate the motive of any superintelligences to engage in blackmail.

Meanwhile I'm banning this post so that it doesn't (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (I'm not sure I know the sufficient detail.)

You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends. This post was STUPID.

(For those who have no idea why I'm using capital letters for something that just sounds like a random crazy idea, and worry that it means I'm as crazy as Roko, the gist of it was that he just did something that potentially gives superintelligences an increased motive to do extremely evil things in an attempt to blackmail us. It is the sort of thing you want to be EXTREMELY CONSERVATIVE about NOT DOING.)"

Re: AI 2040: Plan A

#499

Post by defgeneric »

pibaker wrote: post #312
taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

When I learned that this website is by the same people who gave us AI 2027, I immediately thought about the Wikipedia page on doomsday predictions. https://en.wikipedia.org/wiki/List_of_dates_predicted_for_ap... Notice the many times when a prediction failed and the so called prophet would come back in a few years and give you a new date. And they would tell you, well, this time it's real. I do find it ironic that ma…

Like all forms of eschatology it's not actually concerned with the future to come--it's real aim is to have influence now.

It's just a form of rhetoric.

To dismiss them, saying, "ah ha! they're making the same error as all failed prophets!" misses the point.

What they are really talking about is AI governance.

Re: AI 2040: Plan A

#500

Post by derektank »

mekael wrote: post #442
derektank wrote: post #380

Earlier quoted context omitted.

>My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets. I’d be curious to hear you expand on this. What binds the EA community together, from the shrimp welfare enthusiasts and wild animal initiative, to the longtermist lightcone obsessive, to the people funding vitamin A supplementation, is…

As a society we already know how to make the lives of everyone better: 1. Stable housing 2. Access to safe drinking water 3. Access to food 4. Access to healthcare 5. Access to education 6. Stable governments The EA community has so many "ideas" about what would help, when all they need to do is focus on those six and the world would be as close to a utopia as you or I could hope to see in our lifetimes. I legitimate…

There are tradeoffs to make between those six items. Money spent on stable housing is money not spent on safe drinking water is money not spent on education. There is a limited amount of money spent on charitable giving every year, and we obviously cannot afford to provide all of those things to everyone with the funds currently available. If you’re arguing that people should simply be donating more money, I (and I think every EA) is right there with you. But given the world exists as it is, I would rather see money that’s currently being donated diverted from Catholic missionary schools and good governance NGOs to GiveWell’s top charities because I’m very confident that it would mean fewer dead kids.
Post reply on HN