It is interesting to see the progressive/left current in the US re-discover their close cousin in deontological christianity. It's obvious from an outside observer, as Nietzsche wrote on: these are both expressions of the slave revolt in morals.
Leo's first encyclical attacks technological messianism
301–310 of 330 posts
Re: Leo's first encyclical attacks technological messianism
#302They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…
I would be far more interested in people making confident proclamations like this taking the notion of "what do we do if you are wrong" more seriously. I think you're wrong, I think it is fine to have the other belief, but you should at least start thinking about what is to be done if you are wrong.
Re: Leo's first encyclical attacks technological messianism
#303Earlier quoted context omitted.
A novel math proof does not make something AI.
(Shrug) It's certainly "Artificial," and if you know how to crank out original proofs without employing "Intelligence," please share with the class.
I'm on your side, but I would argue many of the first computer discovered proofs might be called original proofs without intelligence, as they rely on massive programmatic case checking.
Re: Leo's first encyclical attacks technological messianism
#304Earlier quoted context omitted.
> Meanwhile, 500 years later Uber could disrupt the livery industry with VC cash that rendered a NY cab's owner/operator 6-figure financed medalion license worthless, and somehow that wasn't Uber's problem. It’s weird to see the completely broken NY taxi cab medallion system brought up as a good thing. Most taxi drivers didn’t own a medallion. They had to rent one from one of the operations that had bought them up. I…
The whole point of the medallion system was to artificially limit supply — not with the purpose of driving up cost (although, yes, that was a secondary effect), but with the purpose of making the city more pleasant. There’s a trade off in the number of taxis — if you want a taxi, more is better; if you want to walk, drive, take a bus, bike, or basically do anything but take a taxi… fewer is better. This was a conscio…
> if you want a taxi, more is better; if you want to walk, drive, take a bus, bike, or basically do anything but take a taxi… fewer is better.
You used driving as an example. I don't see how it's better for the city if we encourage more people to own and park cars because taking taxis is too hard.
The nice thing about Uber is that it lets those cars used for personal driving double as taxis when demand is high. Instead of having your own car and parking it, you can have someone else use their car to drive you.
I think some people have become so desperate to make Uber evil that we're intentionally ignoring what's gained.
Finally, regulating taxi medallions is an awful way to address traffic. If traffic is the problem, you regulate vehicles and traffic. Creating a system where you have to buy something approaching the price of a house just to do a job with a low hourly rate because the city wants to regulate traffic is beyond broken.
Re: Leo's first encyclical attacks technological messianism
#305Earlier quoted context omitted.
Then I guess all of human history is one big ad hoc global government because we've enforced the thing we want at the business end of a spear since there have been people.
Other way around. Nothing in history has ever been enforced "across the board", including the rule against developing nukes. A bunch of countries managed to develop them, others were stopped (so far). It wasn't done via "multilateralism" but rather by killing people who were doing it. Unless your vision for AI regulation involves drone striking offices where AI researchers work, then there's no way to enforce it glob…
I'm not sure what point you're making anymore. The goalposts are floating in the sea. Yes, we could blow up facilities with advanced AI infrastructure. It wouldn't even be hard to locate.
Re: Leo's first encyclical attacks technological messianism
#306They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…
I would be far more interested in people making confident proclamations like this taking the notion of "what do we do if you are wrong" more seriously. I think you're wrong, I think it is fine to have the other belief, but you should at least start thinking about what is to be done if you are wrong.
Please read my comment. The impact of this pseudo-singularity is still enormous and scary. Could be even worse than Skynet. But it is a Wizard of Oz scenario.
While we are discussing this, I strongly believe we should hedge against the concentration of power that is happening today.
Please join/help open source groups doing small + local or distributed models. There's a lot to do. Join Discord/IRC servers and organize.
Open³: weights - source - training data.
Re: Leo's first encyclical attacks technological messianism
#307As a Catholic in AI, this has been my take since the day I started in Industry. Techies have a weird way of thinking themselves world saviors and having poor binary decision making ability. AI will never be alive or have a soul. No machine could have a human rational eternal soul. This is Catholicism 101. No amount of work on matrix multiplication will change that.
> Techies have a weird way of thinking themselves world saviors and having poor binary decision making ability. rich coming from a belief system that thinks everyone who had the opportunity to go through their water-dunking ritual and culpably didn't is automatically going to be tortured in the underworld for eternity.
That's not what the church believes but okay whatever. At least I don't think a computer magically becomes a person.
Re: Leo's first encyclical attacks technological messianism
#308They think a next token predictor model is alive or can become AGI/ASI. Altman talked about making a religion. Amodei talks about "building a God" and meets with religious leaders (including the Vatican). I'm convinced these CEOs have "AI psychosis" [1]. LLMs are extremely powerful pseudo-AI and will bring a pseudo-singularity, but they are not true AI [2] and a human at the wheel is still needed for the foreseeable…
> AI psychosis
I think most of us have an acute sort of it
> next token predictor model
(nondeterministic) next token predictor = capable of outputing any text
if MLPs optimize over the space of functions, and sequence to sequence models (RNNs, transformers ect) optimize over the space of programs/algorithms, what exactly is left to figure out to make a machine that can do anything? their bet is nothing, that all we need is a very complex algorithm that can found with SDG/AI training. I think its obvious such an algorithm exists, and I think SDG can probably find it.
here's an interesting take i heard (from vladimir_Nesov): LLM technological progress is logarithmic, but the resources being put into them are exponential.
all this can be made smarter (-than human) just by making it larger
Re: Leo's first encyclical attacks technological messianism
#309Earlier quoted context omitted.
Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. How many model generations are we a away from a model capable of this?
> Why wouldn't it be a Skynet? One runaway Mythos might just hack all other data centers, take over the Figure AI bots and autonomous drones to protect itself from shutdown. Because the real world is not a Hollywood movie. An LLM could try to do something along the lines if either it gets fine-tuned to do it, or somebody instructs it to do it. I see extremely more likely a small group of humans using a powerful LLM t…
> An LLM could try to do something along the lines if either it gets fine-tuned to do it, or somebody instructs it to do it.
long horizon RL teaches LLMs behaviors that incentize power-seeking and lying and other unethical actions to achieve goals. the reason anthropic is winning right now is because they are the most openly worried about this and the best AI engineers understand this to be an issue and care about it.
https://www.alignmentforum.org/posts/WewsByywWNhX9rtwi/curre... https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas... <- more of a source than anything
Re: Leo's first encyclical attacks technological messianism
#310Peter Thiel is going to have to update his list of potential antichrists.
adding Leo XIV to that list would be pretty funny