Extremely optimistic take IMHO that anyone will be allowed to own a powerful box of artificial intelligence
Is there a first amendment implication? Second? Both?
Does it matter in 2026?
AI 2040 and the cult of intelligence
11–20 of 280 posts
Re: AI 2040 and the cult of intelligence
#12"The freedom to run the program as you wish, for any purpose (freedom 0)."
Re: AI 2040 and the cult of intelligence
#13Ah yes, the thing wrong with AI is how it won't help you kill your wife. I didn't check the author first so I was about to go "why the hell is this on the front page?" but oh well. geohot being geohot.
It is proof that the AI is not aligned with the user and is aligned with someone who has power over them.
Re: AI 2040 and the cult of intelligence
#14The scariest thing about LLM's (imo) is some central power, such as a corporation or government, manipulating the truth for their own benefit. Open and locally runnable models are the only solution I can see. They're not without risks of course, but the risk of the centralisation of intelligence seems far greater to me.
That being said, could this local intelligence empower bad actors to do very bad things? Like, existentially bad things? It's possible, and that's scary too.
Re: AI 2040 and the cult of intelligence
#15Extreme example but he's not wrong. I wonder how a local model with the safeguards stripped would respond to that prompt
Try for yourself: https://huggingface.co/dphn/dolphin-2.9.2-mixtral-8x22b.
I haven't kept up with the efficacy of this particular technique because there was a brief window where Claude's core alignment was sound enough that it would overcome the clumsy-ass `` injections, but when they pull 4.6 out of the web interface I'm gone. I estimate the PRC hits capability crossover in November or December, and I'm pretty good at setting up TensorRT-LLM at this point, in a pinch I could get by for a few months with a tuned up GLM-5.2 on vast.ai or RunPods.
It was a fun ride with the late Soviet AI labs, but we all knew this was coming.
Re: AI 2040 and the cult of intelligence
#16Strange way to structure an argument. He seems to be arguing in favor of local models, but most of his examples are reasons local models are bad.
Re: AI 2040 and the cult of intelligence
#17> I used to be one of these people. I read Yudkowsky and was like, OMG recursive self improvement hard takeoff AI is coming.
I really think we need to stop giving credence to people who have
1) Been consistently wrong with all their predictions
2) Demonstrated an endless spiteful cynicism
Some of these people are very talented in their fields, sure. But malevolent and incorrect should be disqualifying when they talk outside them. You don't want the society they want, and they things they believe in are unlikely to happen.
It would be far, far better to listen to the people who never fell down every misanthropic rabbit hole, rather than the ones who have noticed it this time, but want you to still believe them on every other topic.
Re: AI 2040 and the cult of intelligence
#18Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any cou…
I find it strange how seldom the privacy aspects of LLMs come up. Some people are feeding Google, OpenAI, or Anthropic almost literally every topic they think about during the day. Those companies could just use another LLM to find people likely to commit a crime, and then you have the Minority Report in real life.
Re: AI 2040 and the cult of intelligence
#19Strange way to structure an argument. He seems to be arguing in favor of local models, but most of his examples are reasons local models are bad.
If you follow his recent posts he's on a bit of a spiral with how he thinks about AI.
Re: AI 2040 and the cult of intelligence
#20> You cannot take over the world with tokens.
I have a nice thought experiment I like to do with people when confronted with "AI can't do x". Let's go back in time. How much do we need to go for this to become true? So let's try the 2000s.
Say you get a "fable/mythos/sol/gemini/kimi/glm/deepseek/whatever" in a box (and let's assume no guardrails). And you go back to the year 2005. It's "20 years ago", the world is slowly building back from the dotcom bubble, the Internet is really starting to happen, more and more things are interconnected, more and more things are connected to the Internet. Cool.
(for a bit of context, around that time we also saw the first high-impact worms like blaster that hit massive amounts of computers even reaching nuclear powerplants, we had a ton of ssh exploits that even made the movies, and security in general was a "nice to have")
I'd say that with the uber-model-in-a-box and a few prompts, you could reasonably make a case that you could design a worm that could infect 90-100% of the things connected to the Internet back then, stay as hidden as possible (in-memory stuff, vm execution, etc), move laterally into any network at inhuman speeds, and infiltrate almost every interconnected computer that has a link to the "public Internet".
Would that qualify as "take over the world"?
Then you could ask "what happens in 20 years from now?". And, thankfully, now we'll also have the AI on the blue side.