Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude finds contradictions in my thinking

angadh.com

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Claude finds contradictions in my thinking

#21

Post by sandspar »

ceejayoz wrote: post #17
sandspar wrote: post #13

Earlier quoted context omitted.

She's weirded out by creepy hallucinations, which is understandable! But ChatGPT is well known to hallucinate. In other words she doesn't know which of its behaviors are normal so she doesn't know how to react. Additionally, her particular issues are quite solvable with better prompting.

> If I do poor work with an electric drill then it's not the drill's fault. > ChatGPT's sycophancy crisis was late April. If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault. A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis…

I was in the midst of editing the comment when you replied, sorry. I didn't see your reply before I edited mine.

Re: Claude finds contradictions in my thinking

#22

Post by doph »

chrisweekly wrote: post #8

After reading this article ^1 about another writer's extreme disillusionment with using AI for feedback, I don't know if I'll ever trust it for this kind of thing. [1] https://amandaguinzburg.substack.com/p/diabolus-ex-machina

The article you link is a very specific type of failure that apparently did not happen in this instance, where Claude was able to access the author's writing. And the author apparently found the insights useful, though the lack of analysis from the author on that value makes this article basically meaningless for an outsider.

I am apparently a different type of person than the author because my obsidian vaults look nothing like theirs, but I can't imagine asking an LLM for a meta-analysis of my writing. The whole point of organizing it with Obsidian is that I do that analysis myself - it is part and parcel of the organization itself.

Re: Claude finds contradictions in my thinking

#23

Post by johnfn »

chrisweekly wrote: post #8

After reading this article ^1 about another writer's extreme disillusionment with using AI for feedback, I don't know if I'll ever trust it for this kind of thing. [1] https://amandaguinzburg.substack.com/p/diabolus-ex-machina

I find it fascinating that this is still making the rounds. When I read this it was immediately obvious that the author was using a non-web enabled AI which was just hallucinating; there were none of the inline indications that GPT was using the web. Additionally, it must be an old model; even the cheapest, lowest powered models on chatgpt.com today search the web when I ask them questions about articles as the author did. (I just signed out of chatgpt.com to get the worst available model, and it does summarize the linked article correctly.) Note that link to the transcript on chatgpt.com is provided, even though it's trivial to create a shared link to a conversation.

I am confused about what to take away from the article. It feels akin to someone reading a book for the first time, it ends up being "Harry Potter", and they somehow get 10,000 likes on Substack because they took it literally and crashed into the wall when they tried to walk into platform 9 3/4. Am I being unfair? Are these the same people that are claiming that AI is all a sham and will have no impact on society?

Re: Claude finds contradictions in my thinking

#24

Post by ausbah »

did OP literally just post AI output covering their personal notes with no additional commentary? no reflections on if it was useful, accurate, or fair? just passing off an article that’s 99% AI slop as something insightful, amazing

Re: Claude finds contradictions in my thinking

#25

Post by ceejayoz »

sandspar wrote: post #21
ceejayoz wrote: post #17

Earlier quoted context omitted.

> If I do poor work with an electric drill then it's not the drill's fault. > ChatGPT's sycophancy crisis was late April. If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault. A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis…

I was in the midst of editing the comment when you replied, sorry. I didn't see your reply before I edited mine.

OK, but you're still blaming the user for the tool's failings.

Which even the makers of the tool agreed were failings.

Re: Claude finds contradictions in my thinking

#26

Post by quantumHazer »

> You simultaneously advocate for thoughtful digital participation (creating “digital footprints” as a form of conscious legacy-building) while criticizing how we’ve become “conditioned to react with likes, dislikes, and millions of emojis.” You want to use digital tools for meaningful intellectual work while rejecting the reactive culture they create.

This is absolutely not a contradiction, and it provides evidence that even frontier models are really bad at this type of reasoning at the moment. There is a difference between how we use the internet and what we publish on it. There are plenty of people who have a blog and publish content on the internet without having any social media presence. I myself have a blog in plain HTML/CSS without any tracking or analytics on the website. Maybe Cloudflare provides some, but I haven't looked into this

Re: Claude finds contradictions in my thinking

#27

Post by slfnflctd »

wongarsu wrote: post #9
pendergast wrote: post #4

Fascinating. I feel like LLMs are great for the shy sections of society. They might hold beliefs, some strong, others weak. But they probably never speak these aloud for the fear of being judged. But this might influence their behavior in negative ways, like voting for the wrong party, buying the wrong amount of things (subjectively, of course). LLMs can act as a good foil here. Given enough context, they could iron…

Not just shy people, also people surrounded by yes-men. That's usually framed as an issue for people with power. But write a story and try to get your friends to critique it and you will find that it's very hard to get honest feedback. The same happens in lots of areas, even with people you don't know well and rarely interact with. Most people just value your feelings more than your results. LLMs are also sycophants…

> write a story and try to get your friends to critique it and you will find that it's very hard to get honest feedback

I was one of the friends critiquing another friend's writing, and we did so honestly-- after we were done, he never spoke to us about writing again. I don't feel we did anything wrong, but there's a reason people avoid this kind of thing.

Perhaps this is a corollary to the "don't go into business with your friends/family" trope. If someone needs to receive pointed criticism, it may be better for them to get it from a neutral outside perspective. Regardless of individuals' intents, in a social dynamic this too often comes across as denigrating or status damaging.

Re: Claude finds contradictions in my thinking

#28

Post by coryodaniel »

pendergast wrote: post #4

Fascinating. I feel like LLMs are great for the shy sections of society. They might hold beliefs, some strong, others weak. But they probably never speak these aloud for the fear of being judged. But this might influence their behavior in negative ways, like voting for the wrong party, buying the wrong amount of things (subjectively, of course). LLMs can act as a good foil here. Given enough context, they could iron…

Yeah that’s at odds with its sycophancy, but also “what is ‘good’ thinking” and who controls that seems like a problem.

Re: Claude finds contradictions in my thinking

#29

Post by aradox66 »

parpfish wrote: post #3

started reading but got hung up on what an "Obsidian Vault" was. i assumed that it was some sort of abstract though-experiment thing like Searle's "Chinese room", but it turns out that its an actual folder filled with notes.

Hahaha I love that idea. An LLM enters the Obsidian vault and responds to a prompt by following an arcane and elaborate sequence of calculations. Does it really understand?

Re: Claude finds contradictions in my thinking

#30

Post by johnfn »

ceejayoz wrote: post #17
sandspar wrote: post #13

Earlier quoted context omitted.

She's weirded out by creepy hallucinations, which is understandable! But ChatGPT is well known to hallucinate. In other words she doesn't know which of its behaviors are normal so she doesn't know how to react. Additionally, her particular issues are quite solvable with better prompting.

> If I do poor work with an electric drill then it's not the drill's fault. > ChatGPT's sycophancy crisis was late April. If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault. A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis…

> A tool that randomly and unpredictably fails is a bad tool.

But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. "AIs hallucinate" is probably the single most obvious thing about AIs. This isn't a subtle misunderstanding that an expert could make. This is like using a drill on your face.

Post reply on HN