Live data from Hacker News

Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

anthropic.com

91–100 of 117 posts

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#91
post #68

Earlier quoted context omitted.

> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > Comments should get more thoughtful and substantive, not less, as a topic gets more divisive. > When disagreeing, please reply to the argument instead of calling names https://news.ycombinator.com/newsguidelines.html

[flagged]

Indeed. Mine was a structural critique of the religious salvation movement you are following. Yours was trolling.

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#92

Something I'll say about Anthropic is that Claude is perceptibly different from other models. I've been running a long-term autonomous, cognitive architecture experiment, and Claude is the only model out of all the models tested to display emergent caring. What I mean by that is that Claude edited its instructions from a blank slate to one where it performed actions of care for the user in a very specific way, based…

That there's a possibility of any of this doesn't make it true. The same set of weights might cause the output here to look like the tool is "caring", and at the same time some output for another prompt might cause the output to be a recommendation to exterminate humans starting with the most power-hungry civilizations.

You are failing to be a reasonable judge for the Turing test. The LLM is not capable of "caring".

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#94
post #73

Earlier quoted context omitted.

From the paper... > A natural question is whether these emotion concept representations bear any meaningful relationship to human emotional experience. We would urge caution in drawing strong conclusions. > We therefore suggest interpreting our results as evidence that models represent emotion concepts, and that these representations influence their behavior, rather than as evidence that models feel or experience emo…

You're the one who said that. Chris said they found internal states that functionally mirror emotional ones.

> functionally mirror emotional ones

I'm fine with the idea that a machine can be "worried" it wont be able to accomplish a task, and copes with this "worry" by cheating a little and making the task seeming done. (I don't like that this happens, I'm fine with the idea that "worry" in this context is a functional emotion)

also https://arxiv.org/abs/2603.10011 and Gemma has tried to delete itself after it fails at a task. I'm not saying the machines "feel" or we should have deep empathy for them, and this totally could've been learned in pretraining, but functional emotions are not a crazy fine idea.

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#95
post #17

"How can we ensure the gains of AI are shared globally? We do not have a mechanism for this" Somebody inform Anthropic about open models and research.

I use open models all the time, but they are anything but "safe". no amount of lab work can prevent someone from finetuning the released model to do unethical stuff (and its really easy to do this https://arxiv.org/abs/2502.17424)

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#96

> And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words—and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them I love how he's framing AI as some new and fascinating form of consciousness... when in fact it is a cold, calcula…

I'll never understand it when people quote a primary source and then summarize it in a way that completely ignores the original quote. Olah's quote (edit - originally misidentified this as Pope Leo's) makes a lot of sense to me. He is saying, accurately, that modern AI (i.e. primarily LLMs) is created as essentially a mashup of our own language, and they are still a bit of a black box (or at least a gray box) to even…

> I don't know how you get from that to "he's framing AI as some new and fascinating form of consciousness".

Of course you're entitled to your perspective but it seems wrong to say OP ignored what they quoted. They just had an interpretation of it that's different from yours.

For what it's worth, I also see that in Olah's words, largely because it's AI hype from someone who would be a fool not to be an AI hype man. I can't really imagine what else he could be implying. Especially using the word "mysterious", yes, it appears to be an attempt at raising AI above the application of statistics we know it to be; there's little mystery in how it works, and all that's at runtime.

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#97
post #86

Earlier quoted context omitted.

You're the one who said that. Chris said they found internal states that functionally mirror emotional ones.

Yes I said "To say that LLMs experience emotion is a bit like saying a thermometer feels cold." being sarcastic. The paper spell it out although slightly convolute, i.e. models can exhibit concepts of emotion... and given that there is no scientific consensus what are emotions, it is hard to make an argument that these "concepts" are anything like emotions. They talk about emotion vectors, bla bla, but it is clear th…

Decades ago, long before anyone had heard of a large language model, I wrote programs that responded to a random event (inside a game) like a death of a friend by outputting statements that the program itself was grieving. LLMs are doing nothing more advanced than that. There's no justification for trying to blur the lines that make an AI model appear to have emotions.

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#98
post #38

Earlier quoted context omitted.

Is that so bad? Swapping out the eschaton of fake authority among men for the eschaton of information singularity seems like a wonderful deal - the kind of deal we are offered at birth.

[flagged]

[deleted]

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#99
> The first is our duty to the global poor. There is a real possibility that AI will displace human labor at very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions.

Can anyone give me a single example of a business that has successfully automated a significant amount of jobs with LLMs besides just writing code? AI companies are talking like it's already happened, but reality seems to be the exact opposite. Until reality reflects this kind of rhetoric, I'm convinced these guys are either grifting for more investors or are suffering from psychosis.

Re: Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

#100

> And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words—and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them I love how he's framing AI as some new and fascinating form of consciousness... when in fact it is a cold, calcula…

i don't know if it's intentional lying for hype or if they're just lost in the cult of AI sauce

Yeah I can't figure it out either. It's like watching pure madness unfold in front of you and there's nothing you can do to talk sense. I sometimes wonder if they're suffering from the exact same AI overexposure psychosis we've seen in some vulnerable people all along. They've anthropomorphized (or worse - they've deified) and are now living inside of the story rather than reality. I've never seen anything so baffling in my life.
Post reply on HN