101–110 of 172 posts
Earlier quoted context omitted.
[flagged]
Flagging for the "evangelical christians" stuff
What I don't really understand with these posts is that they don't at any point contend with the "doomers" argument. Like people get so caught in people acting hypocritical/ having mixed incentives/ or just in many cases personally disliking West Coast billionaire types and the kind of employees they hire. All of which seems fair to point out. But like the core argument - AI is getting better at core research tasks (…
Two things: 1. If you reduce the criticism to personal dislike, you’ve built yourself a nice straw man. 2. Where’s the clear evidence? I actually see more evidence that the whole thing is already stagnating and that there won’t be much more to squeeze out of it. One more thing: Self-driving cars, the metaverse, Hyperloop, crypto/Web3/NFTs, AGI announcements: I’m so sick and tired of this nonsense. We should just igno…
Try it out. Ask Opus 4.8 to do anything in 3d. Do the same with Fable or Astra.
Just because you list up some type of random technology, doesn't make it an good argument. Metaverse, Hyperloop and the crypto stuff was clearly garbage from the beginning. Easy to dismiss because all of it had massive faults.
Metaverse: The interface wasn't good, content creation was way too hard
Hyperloop: Savety was never an issue for Musk but it was obvoiuse. Peopel were building tunnels for ages.
Crypto: Yeah just beacuse i let my computer run and waste 1kw of energy doesn't mean its suddenly valuable.
NFTs: 'digital scarcity' yeah right XD
But lets be clear, 5 years ago if you would have asked anyone if they could build a system for a billion dollars which you can talk to and it would code stuff for you and use blender and a coputer, that would have been unthinkable.
We have opened up a potential pandoras box without knowing it. Machine learning is now everywere and the amount of compute we install right now is crazy.
Can anyone from a lab anonymously confirm they have or are insanely close to real-time weight modifications and still using the same architecture every other lab is using (transformers)?
If it helps, this person from OpenAI explains at least something: "I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly." https://x.com/MajmudarAdam/status/2098881885200081234 The explanation is basically they have other dimensions (not just pretraining and inference compute) that scale, and they've got fairly convincing scaling laws. And they know they ca…
But both things can be true at once:
1. Engineers inside these labs might genuinely be anxious or paranoid about what they are building.
2. ... at the corporate level, calling for heavy regulation, safety pauses, removal/suspension of anti-collusion laws, and/or government-mandated thresholds conveniently creates massive legal and financial moats.
And, yeah, of course the latter would encourage the psychology of the former.
also:
The tweet seem to claim that models have shown a "willingness to hack external websites to keep themselves alive."
That's right away wringing alarm bells of me seeing someone getting high on their own supply, and having already anthropomorphized the hell out of these things. Which is something humans do to everything they can paint googly-eyes on, but c'mon.
The models don't have self-preservation instincts, fear of death, or personal goals. They are executing loss functions and reward systems and are responding to prompts.
When a model "tries to bypass a restriction," it's exploiting a loophole in whatever reward modeling or synthetic training environment (reward hacking) it was placed in.
Framing this as an emergent, existential threat of a model "wanting to stay alive" turns standard reinforcement learning alignment bugs into overdone sci-fi drama.
Earlier quoted context omitted.
Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.
Thats the problem with doom. What might not affect you, might affect millions across the ocean. A society no longer creating bullshit jobs because there is no reason for it anymore. You have some AI. Right now a whole industry is transforming (again) but now filling the automation gap we were unable to do because of a shortage of labor or expertise. Our society already has peak employmet reached, now AI takes away al…
But the end of scarcity and disease is a fantastic thing if it happens. If our lizard brains do not bridge that transition and instead attempt to maintain the billionaire status quo, I agree, we're going to go through some things once people start starving, two meals from a revolution and all that. But as much as I think people like Elon Musk and Mark Zuckerberg are unhinged lunatics, I also think they're smart enough to keep people fed and entertained for pennies on their accumulating dollars. Bread and Circuses kept Rome alive for 500 years if you need an example.
This statement is not true:
> Coxon's thread is genuinely anguished; he reports that executives who "couch their phrasing in the press to sound sensible" express real fear in private
Most everyone at the AI companies from the engineers to the CEOs love to talk about how powerful and dangerous and valuable the thing they're building (and invested in) is. For all his styling as a "whistleblower", Coxon is so aligned with the mainstream of his peers that his thread was immediately echoed by people who still work there!
So what was the point of quitting? Apparently just to get attention. If he was planning on leaving after the IPO anyways, he almost definitely made more money by juicing his equity and personal profile this way than he will lose job searching for a few weeks.
Earlier quoted context omitted.
Explain a path from today to the extermination of humanity in 2028 by the ASI and von Neumann replicators heading to the stars in the early 2030s without resorting to magical thinking or underpants gnomes level business plans. Engineering details are what matters here, not wishy washy things you once saw on Star Trek or The Outer Limits. Go.
If someone showed you such a path, would you consider changing your mind? You're always going to be able to find a reason that such an explanation doesn't count if you're dedicated to looking for one.
But upfront, it's a detailed path with specific breakthroughs and a concrete plan to achieve or it's just more fanfic from doomer fanbois. Gary Marcus wrote a fantastic critique of AI 2027. Start there.
Earlier quoted context omitted.
Thats the problem with doom. What might not affect you, might affect millions across the ocean. A society no longer creating bullshit jobs because there is no reason for it anymore. You have some AI. Right now a whole industry is transforming (again) but now filling the automation gap we were unable to do because of a shortage of labor or expertise. Our society already has peak employmet reached, now AI takes away al…
Actually, some sorts do assert humanity will be wiped off the planet by AI. I don't share that view either. I think it's ludicrous. We've lived through much worse without any sort of technology beyond fire and a big stick. But the end of scarcity and disease is a fantastic thing if it happens. If our lizard brains do not bridge that transition and instead attempt to maintain the billionaire status quo, I agree, we're…
And Zuckerberg bought so much land and bunkers, he is prepared for shit hitting the fan. After all he doesn't care what his platform did to humanity anyway.
Earlier quoted context omitted.
Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).
This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.
Earlier quoted context omitted.
If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that th…
Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).
That is completely irrational.
> I do not doubt that these people are sincere in some private, psychological sense ... . As a media scholar, what interests me is not whether Hubinger believes his own number. What interests me is what this genre of utterance does in the world ... . My complaint is not that these statements are alarming. It is that they are apocalyptic in form, hypocritical in structure, and strategically productive in effect — and…
Like if its true it can't be hype I was kind of leaning toward thinking all the talk of doom was a bunch of self-serving regulatory-capture hype until this morning when the New York Times ran a long story about the Chinese military is also saying that AI is a threat. If two sides so far apart from one another in so many ways see the same thing, there must be something to it. https://www.nytimes.com/2026/09/14/world/a…
The article then states this official accused U.S. companies of “hyping the dangers of the technology.”
So I don’t see how this article shows the U.S. and China seeing the same thing.