Earlier quoted context omitted.
Is it this fuzzy blob of ineffable experiences, aesthetics, qualia, self-reference, dualism, spirituality and other things? I don't need to reject all this to notice that its fuzziness makes it impossible to operationalize.
No... again a misunderstanding of the word, and a major one! A soul - and I mean this in the most atheistic, nonreligious sense you can imagine - is what you operationalize every single day when you're faced with hardship or any situation where you'd prefer not to be. It's also the dynamic that makes you feel joy in doing something beyond simple pleasure or happiness. You know SQL? Think of "soul" as a view of the ta…
What can you say besides "I don't feel a soul inside an LLM"? You don't feel it? OK, good for you, whatever.