AI 2027 • Hacker News Board

Re: AI 2027

#411

Post by bshacklett » Fri, Apr 04, 2025, 1:55 PM UTC

TheDong wrote: post #265

ivraatiems wrote: post #219

Though I think it is probably mostly science-fiction, this is one of the more chillingly thorough descriptions of potential AGI takeoff scenarios that I've seen. I think part of the problem is that the world you get if you go with the "Slowdown"/somewhat more aligned world is still pretty rough for humans: What's the point of our existence if we have no way to meaningfully contribute to our own world? I hope we're wr…

> What's the point of our existence if we have no way to meaningfully contribute to our own world? For a sizable number of humans, we're already there. The vast majority of hacker news users are spending their time trying to make advertisements tempt people into spending money on stuff they don't need. That's an active societal harm. It doesn't contribute in any positive way to the world. And yet, people are fine to…

I was hoping someone would bring up Bullshit Jobs. There are definitely a lot of people spending the majority of their time doing "work" that doesn't have any significant impact to the world already. I don't know that some future AI takeover would really change much, except maybe remove some vale of perception around meaningless work.

At the same time, I wouldn't necessarily say that people are currently fine getting dopamine hits from social media. Coping would probably be a better description. There are a lot of social and societal problems that have been growing at a rapid rate since Facebook and Twitter began tapping into the reward centers of the brain.

From a purely anecdotal perspective, I find my mood significantly affected by how productive and impactful I am with how I spend my time. I'm much happier when I'm making progress on something, whether it's work or otherwise.

Re: AI 2027

#412

Post by cheevly » Fri, Apr 04, 2025, 1:58 PM UTC

kubb wrote: post #300

esafak wrote: post #119

Earlier quoted context omitted.

It's already replacing artists; that's why they're up in arms. People don't need stock photographers or graphic designers as much as they used to. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4602944

I know that artists don’t like AI, because it’s trained on their stolen work. And yet, AI can’t create a sprite sheet for a 2d game. This is because it can steal a single artwork but it can’t make a collection of visually consistent assets.

Bro what are you even talking about? ControlNet has been able to produce consistent assets for years.

How exactly do you think video models work? Frame to frame coherency has been possible for a long time now. A sprite sheet?! Are you joking me. Literally churning them out with AI since 2023.

Re: AI 2027

#413

Post by eob » Fri, Apr 04, 2025, 2:03 PM UTC

An aspect of these self-improvement thought experiments that I’m willing to tentatively believe.. but want more resolution on, is the exact work involved in “improvement”.

Eg today there’s billions of dollars being spent just to create and label more data, which is a global act of recruiting, training, organization, etc.

When we imagine these models self improving, are we imagining them “just” inventing better math, or conducting global-scale multi-company coordination operations? I can believe AI is capable of the latter, but that’s an awful lot of extra friction.

Re: AI 2027

#414

Post by treis » Fri, Apr 04, 2025, 2:05 PM UTC

bsenftner wrote: post #340

stego-tech wrote: post #279

It’s good science fiction, I’ll give it that. I think getting lost in the weeds over technicalities ignores the crux of the narrative: even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. The problems it raises - alignment, geopolitics, lack of societal safeguards - are all real, and happening now (just replace “AGI” with “corpo…

Whatever the future is, it is not American, not the United States. The US's cultural individualism has been Capitalistically weaponized, and the educational foundation to take the country forward is not there. The US is kaput, and we are merely observing the ugly demise. The future is Asia, with all of western culture going down. Yes, it is not pretty, The failed experiment of American self rule.

People said the same thing about Japan but they ran into their own structural issues. It's going to happen to China as well. They've got demographic problems, rule of law problems, democracy problems, and on and on.

Re: AI 2027

#415

Post by fmap » Fri, Apr 04, 2025, 2:05 PM UTC

gwd wrote: post #393

fmap wrote: post #366

Earlier quoted context omitted.

> even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. I agree that it's good science fiction, but this is still taking it too seriously. All of these "projections" are generalizing from fictional evidence - to borrow a term that's popular in communities that push these ideas. Long before we had deep learning there were people l…

> Someday we will have a machine simulate a cat, then the village idiot... This isn't how LLMs work. I think you misunderstood that argument. The simulate the brain thing isn't a "start from the beginning" argument, it's an "answer a common objection" argument. Back around 2000, when Nick Bostrom was talking about this sort of thing, computers were simply nowhere near powerful enough to come even close to being smart…

> The idea is, if we don't think of anything more efficient, we'll at least be able to simulate a cat, and then an idiot, and then Einstein, and then something smarter. And since we almost certainly will think of something more efficient than "simulate a human brain", we should expect superintelligence to come much sooner.

The problem with this argument is that it's assuming that we're on a linear track to more and more intelligent machines. What we have with LLMs isn't this kind of general intelligence.

We have multi-paragraph autocomplete that's matching existing texts more and more closely. The resulting models are great priors for any kind of language processing and have simple reasoning capabilities in so far as those are present in the source texts. Using RLHF to make the resulting models useful for specific tasks is a real achievement, but doesn't change how the training works or what the original training objective was.

So let's say we continue along this trajectory and we finally have a model that can faithfully reproduce and identify every word sequence in its training data and its training data includes every word ever written up to that point. Where do we go from here?

Do you want to argue that it's possible that there is a clever way to create AGI that has nothing to do with the way current models work and that we should be wary of this possibility? That's a much weaker argument than the one in the article. The article extrapolates from current capabilities - while ignoring where those capabilities come from.

> And, even if you think A or B are unlikely, doesn't it make sense to just consider the possibility that they're true, and think about how we'd know and what we could do in response, to prevent C or D?

This is essentially https://plato.stanford.edu/entries/pascal-wager/

It might make sense to consider, but it doesn't make sense to invest non-trivial resources.

This isn't the part that bothers me at all. I know people who got grants from, e.g., Miri to work on research in logic. If anything, this is a great way to fund some academic research that isn't getting much attention otherwise.

The real issue is that people are raising ridiculous amounts of money by claiming that the current advances in AI will lead to some science fiction future. When this future does not materialize it will negatively affect funding for all work in the field.

And that's a problem, because there is great work going on right now and not all of it is going to be immediately useful.

Re: AI 2027

#416

Post by whiplash451 » Fri, Apr 04, 2025, 2:08 PM UTC

fmap wrote: post #366

stego-tech wrote: post #279

It’s good science fiction, I’ll give it that. I think getting lost in the weeds over technicalities ignores the crux of the narrative: even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. The problems it raises - alignment, geopolitics, lack of societal safeguards - are all real, and happening now (just replace “AGI” with “corpo…

> even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. I agree that it's good science fiction, but this is still taking it too seriously. All of these "projections" are generalizing from fictional evidence - to borrow a term that's popular in communities that push these ideas. Long before we had deep learning there were people l…

> there are exciting applications which will end up underfunded after the current AI bubble bursts

Could you provide examples? I am genuinely interested.

Re: AI 2027

#417

Post by [deleted] » Fri, Apr 04, 2025, 2:09 PM UTC

827a wrote: post #91

Readers should, charitably, interpret this as "the sequence of events which need to happen in order for OpenAI to justify the inflow of capital necessary to survive". Your daily vibe coding challenge: Get GPT-4o to output functional code which uses Google Vertex AI to generate a text embedding. If they can solve that one by July, then maybe we're on track for "curing all disease and aging, brain uploading, and coloni…

[deleted]

Re: AI 2027

#418

Post by whiplash451 » Fri, Apr 04, 2025, 2:10 PM UTC

fmap wrote: post #366

stego-tech wrote: post #279

It’s good science fiction, I’ll give it that. I think getting lost in the weeds over technicalities ignores the crux of the narrative: even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. The problems it raises - alignment, geopolitics, lack of societal safeguards - are all real, and happening now (just replace “AGI” with “corpo…

> even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. I agree that it's good science fiction, but this is still taking it too seriously. All of these "projections" are generalizing from fictional evidence - to borrow a term that's popular in communities that push these ideas. Long before we had deep learning there were people l…

There is no need to simulate Einstein to transform the world with AI.

A self-driving car would already be plenty.

Re: AI 2027

#419

Post by luxardo » Fri, Apr 04, 2025, 2:15 PM UTC

wkat4242 wrote: post #215

pydry wrote: post #182

Earlier quoted context omitted.

>History says that actually when this happens, an entire generation is yeeted on to the streets History hasnt had to contend with a birth rate of 0.7-1.6. It's kind of interesting that the elite capitalist media (economist, bloomberg, forbes, etc) is projecting a future crisis of both not enough workers and not enough jobs simultaneously.

I don't really get the American preoccupation with birth rates. We're already way overpopulated for our planet and this is showing in environmental issues, housing cost, overcrowded cities etc. It's totally a great thing if we start plateauing our population and even reduce it a bit. And no we're not going extinct. It'll just cause some temporary issues like an ageing population that has to be cared for but those iss…

We are most certainly not "overpopulated" in any way. Usage per person is what the issue is.

And no society, ever, has had a good standard of living with a shrinking population. You are advocating for all young people to toil their entire lives taking care of an ever-aging population.

Re: AI 2027

#420

Post by whiplash451 » Fri, Apr 04, 2025, 2:19 PM UTC

Aurornis wrote: post #210

torginus wrote: post #143

Much has been made in its article about autonomous agents ability to do research via browsing the web - the web is 90% garbage by weight (including articles on certain specialist topics). And it shows. When I used GPT's deep research to research the topic, it generated a shallow and largely incorrect summary of the issue, owning mostly to its inability to find quality material, instead it ended up going for places li…

This story isn’t really about agents browsing the web. It’s a fiction about a company that consumes all of the web and all other written material into a model that doesn’t need to browse the web. The agents in this story supersede the web. But your point hits on one of the first cracks to show in this story: We already have companies consuming much of the web and training models on all of our books, but the reports t…

In my opinion, the real breakthrough described in this article is not bigger models to read the web, but models that can experiment on their own and learn from these experiments to generate new ideas.

If this happens, then we indeed enter a non-linear regime.