Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cursor IDE support hallucinates lockout policy, causes user cancellations

old.reddit.com

Post reply on HN
Print view View original

631–635 of 635 posts

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#631

Post by learningstud »

nerdjon wrote: post #63

There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore and then a company that would benefit from that narrative gets directly hurt by it. Which of course they are going to try to brush it all away. Better than admitting that this problem very much still exists and isn’t going away anytime soon.

People hallucinate all the time out of pressure or habit. We don't need AI for that. It's hard to tell most people from AI. Most people would fail Turing tests as subjects.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#632

Post by learningstud »

emn13 wrote: post #437
lynguist wrote: post #400

Earlier quoted context omitted.

https://www.anthropic.com/research/tracing-thoughts-language... The section about hallucinations is deeply relevant. Namely, Claude sometimes provides a plausible but incorrect chain-of-thought reasoning when its “true” computational path isn’t available. The model genuinely believes it’s giving a correct reasoning chain, but the interpretability microscope reveals it is constructing symbolic arguments backward from…

While some of what you say is an interesting thought experiment, I think the second half of this argument has, as you'd put it, a low symbolic coherence and low plausibility. Recognizing the relevance of coherence and plausibility does not need to imply that other aspects are any less relevant. Redefining truth merely because coherence is important and sometimes misinterpreted is not at all reasonable. Logically, a f…

Validity is not soundness. Wonder why people are just beginning to realize what logicians have been studying for more than a century. This goes to show that most programming was never based on logic but vibes. People have been vibe coding with themselves before AI became prominent.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#633

Post by learningstud »

CodesInChaos wrote: post #439
lynguist wrote: post #400

Earlier quoted context omitted.

https://www.anthropic.com/research/tracing-thoughts-language... The section about hallucinations is deeply relevant. Namely, Claude sometimes provides a plausible but incorrect chain-of-thought reasoning when its “true” computational path isn’t available. The model genuinely believes it’s giving a correct reasoning chain, but the interpretability microscope reveals it is constructing symbolic arguments backward from…

> The model genuinely believes it’s giving a correct reasoning chain, but the interpretability microscope reveals it is constructing symbolic arguments backward from a conclusion. Sounds very human. It's quite common that we make a decision based on intuition, and the reasons we give are just post-hoc justification (for ourselves and others).

Exactly, most of us behave in almost the same as AI does. We finally have a mirror to reflect upon.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#634

Post by learningstud »

ryandrake wrote: post #212
schmichael wrote: post #168

Earlier quoted context omitted.

> You're not supposed to trust the tool This is just an incredible statement. I can't think of another development tool we'd say this about. I'm not saying you're wrong, or that it's wrong to have tools we can't just, just... wow... what a sea change.

Imagine if your compiler just randomly and non-deterministically compiled valid code to incorrect binaries, and the tool's developer couldn't really tell you why it happens, how often it was expected to happen, how severe the problem was expected to be, and told you to just not trust your compiler to create correct machine code. Imagine if your calculator app randomly and non-deterministically performed arithmetic in…

Edsgar Dijkstra!

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#635

Post by SpanishBrowne »

mntruell wrote: post #272

(Cursor cofounder) Apologies - something very clearly went wrong here. We’ve already begun investigating, and some very early results: * Any AI responses used for email support are now clearly labeled as such. We use AI-assisted responses as the first filter for email support. * We’ve made sure this user is completely refunded - least we can do for the trouble. For context, this user’s complaint was the result of a r…

cofounder or another bot stringing letters together?
Post reply on HN