191–200 of 941 posts
Earlier quoted context omitted.
Haven't people learned about the peril of assuming, "Line go up" yet?! TRENDS HAVE FEEDBACK
Trends don't go on forever, but the market can stay irrational longer than you can stay solvent. There's no good rule of thumb for this, other than maybe the Lindy effect.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…
Humans weren't built to handle long term risks. We just weren't. For basically all of our evolutionary history, we were almost overwhelmingly concerned with the short term. What will you eat today, How will you sleep tonight. Problems on the order of days or weeks. At best, the next season. Our intelligence evolved to disregard super long term risks because it simply didn't matter (what use is worrying about 5 years…
I’m curious what the downsides are of taking statements like these seriously. There seems to be universal eye rolling that happens in each and every one of these cases, and it comes down to usually one reason: “If they really believed it they would be whistleblowing etc..” Completely forgetting that working at Los Alamos was basically the highlight of your life if you were a physicist in 1940. It’s no different here…
What are the downsides of taking what amounts to unsubstantiated gossip seriously?
Earlier quoted context omitted.
I read this. I still think it's complete bullshit. The person posting this may very well believe in all this crap, I don't dispute that. People believe in all sorts of shit.
So, let’s quantify things: what’s the chance they’re right? And what’s the cost of that chance happens?
how could they do it (not kill everyone) 1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear. 2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications…
Assuming those events happen in that order, then the prior might solve the latter.
“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...
Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…
“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
Earlier quoted context omitted.
> What's the most dangerous thing that's happened with an LLM so far? I don't know, maybe a mass shooting? https://www.npr.org/2026/09/02/nx-s1-5953021/openai-tumbler-... Oh, and let's just forget the uncountable early deaths from the environmental disaster of the Datacenter buildout. It's not as sexy and doesn't make headlines, so those deaths don't really count or matter do they?
I did know about the mass shooting but failed to mention it here. I’d put it in the “unlikely to be a widespread problem” category. If we’re in the “one AI driven mass shooting every four years” world for example it’s fair to call it a rare issue. The environmental impact seems either very overblown (e.g., water usage just isn’t that high) and the part that isn’t overblown is totally abatable (e.g., noise and emissio…
> The people building AI earnestly believe that it could kill us all by the end of the decade. I think he is being over dramatic. In the space of about four years, LLMs progressed from mediocre high school student to Ph.D. graduate in every field. That's impressive, but there is no evidence yet they can outperform or outsmart humans. Their biggest advantage for tasks such as proving theorems or long coding sessions i…
I mean, unless you see clear reasons for them to stop getting better _right now_, this is not very comforting.