Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cursor IDE support hallucinates lockout policy, causes user cancellations

old.reddit.com

Post reply on HN
Print view View original

421–430 of 635 posts

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#421

Post by jmaker »

cryptoegorophy wrote: post #180
nerdjon wrote: post #63

There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore and then a company that would benefit from that narrative gets directly hurt by it. Which of course they are going to try to brush it all away. Better than admitting that this problem very much still exists and isn’t going away anytime soon.

I think that’s why Apple is very slow at rolling out AI if it ever actually will. Downside is way too big than the upside.

Even the iOS and macOS typing correction engine has been getting worse for me over the past few OS updates. I’m now typing this on iOS, and it’s really annoying how it injects completely unrelated words, replaces minor typos with completely irrelevant words. Same in Safari on macOS. The previous release felt better than now, but still worse than a couple years ago.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#422

Post by miyuru »

arkh wrote: post #379
babypuncher wrote: post #64

Earlier quoted context omitted.

Given how incredibly stingy tech companies are about spending any money on support, I would not be surprised if the story about it being a rogue AI support agent is 100% true. It also seems like a weird thing to lie about, since it's just another very public example of AI fucking up something royally, coming from a company whose whole business model is selling AI.

> how incredibly stingy tech companies are about spending any money on support Which is crazy. Support is part of marketing so it should get the same kind of consideration. Why do people think Amazon is hard to beat? Price? nope. Product range? nope. Delivery time? In part. The fact if you have a problem with your product they'll handle it? Yes. After getting burned multiple times by other retailers you're gonna pay…

In my products, if a user has payed me, their support tickets get high priority, and I get notified immediately.

Other tickets get replied within the day.

I am also running it by myself; I wonder why big companies with 50+ employees like cursor cheaps out with support.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#423

Post by Tainnor »

forgetfreeman wrote: post #260
theonething wrote: post #224

Earlier quoted context omitted.

Ok, here I thought requiring PR review and approval before merging was standard industry best practice. I guess all the places I've worked have been doing it wrong?

There's a lot of shit that has become "best practice" over the last 15 years, and a lot more that was "best practice" but fell out of favor because reasons. All of it exists on a continuum of what is actually reasonable given the circumstances. Reviewing pull requests is one of those things that is reasonable af in theory, produces mediocre results in practice, and is frequently nothing more than bureaucratic overhea…

Code review is actually one of the few practices for which research does exist[0] which points in the direction of it being generally good at reducing defects.

Additionally, in the example you share, where only one person knows the context of the change, code review is an excellent tool for knowledge sharing.

[0]: https://dl.acm.org/doi/10.1145/2597073.2597076, for example

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#425

Post by porridgeraisin »

I disagree with but understand selling AI as a hot thing that can reliably do things to other companies. False marketing has been around since "buh buh" sold a deer corpse to "bah bah" in exchange for his pretty leaf mat by telling him it's actually a cow.

But drinking the kool aid yourself? That demonstrates a new low in human mental facility.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#426

Post by TylerE »

jmaker wrote: post #421
cryptoegorophy wrote: post #180

Earlier quoted context omitted.

I think that’s why Apple is very slow at rolling out AI if it ever actually will. Downside is way too big than the upside.

Even the iOS and macOS typing correction engine has been getting worse for me over the past few OS updates. I’m now typing this on iOS, and it’s really annoying how it injects completely unrelated words, replaces minor typos with completely irrelevant words. Same in Safari on macOS. The previous release felt better than now, but still worse than a couple years ago.

It’s not just you. iOS auto correct has gotten damn near malicious. E seen it insert entire words out of nowhere

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#427

Post by timewizard »

xvector wrote: post #325
timewizard wrote: post #294

Earlier quoted context omitted.

> "and it has accelerated my output massively." The folly of single ended metrics. > but to avoid LLMs out of principal? That's just luddite. Do you double check that the LLM hasn't magically recreated someone else's copyrighted code? That's just irresponsible in certain contexts. > in case you feel it relevant. Of course it's relevant. If a 19 year old with 1 year of driving experience tries to sell me a car using t…

> Do you double check that the LLM hasn't magically recreated someone else's copyrighted code? I frankly do not care, and I expect LLMs to become such ubiquitous table-stakes that I don't think anyone will really care in the long run.

> and I expect LLMs to become such ubiquitous table-stakes

Unless they develop entirely new technology they're stuck with linear growth of output capability for input costs. This will take a very long time. I expect it to be abandoned in favor of better ideas and computing interfaces. "AI" always seems to bloom right before a major shift in computing device capability and mobility and then gets left behind. I don't see anything special about this iteration.

> that I don't think anyone will really care in the long run.

There are trillions of dollars at stake and access to even the basics of this technology is far from egalitarian or well distributed. Until it is I would expect people who's futures and personal wealth depends on it to care quite a bit. In the meanwhile you might just accelerate yourself into a lawsuit.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#428

Post by tomaskafka »

lynguist wrote: post #400
nerdjon wrote: post #63

There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore and then a company that would benefit from that narrative gets directly hurt by it. Which of course they are going to try to brush it all away. Better than admitting that this problem very much still exists and isn’t going away anytime soon.

https://www.anthropic.com/research/tracing-thoughts-language... The section about hallucinations is deeply relevant. Namely, Claude sometimes provides a plausible but incorrect chain-of-thought reasoning when its “true” computational path isn’t available. The model genuinely believes it’s giving a correct reasoning chain, but the interpretability microscope reveals it is constructing symbolic arguments backward from…

[flagged]

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#429

Post by tomaskafka »

throwawaysleep wrote: post #419
SCdF wrote: post #380

Earlier quoted context omitted.

> * Any AI responses used for email support are now clearly labeled as such. We use AI-assisted responses as the first filter for email support. Don't use AI. Actually care. Like, take a step back, and realise you should give a shit about support for a paid product. Don't get me wrong: AI is a very effective tool, *for doing things you don't care about*. I had to do a random docker compose change the the other day. I…

The amount paid is still pretty trivial. I wouldn’t expect much human support for most SaaS products costing $20 a month.

No idea why you're downvited. If anyone wants a human support handholding, that's a territory of $200 or $2000/mo products.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#430

Post by dspillett »

mntruell wrote: post #272

(Cursor cofounder) Apologies - something very clearly went wrong here. We’ve already begun investigating, and some very early results: * Any AI responses used for email support are now clearly labeled as such. We use AI-assisted responses as the first filter for email support. * We’ve made sure this user is completely refunded - least we can do for the trouble. For context, this user’s complaint was the result of a r…

> Any AI responses used for email support are now clearly labeled as such.

Because we all know how well people pay attention to such clear labels, even seasoned devs not just “end users”⁰.

Also, deleting public view of the issue (locking & hiding the reddit thread) tells me a lot about how much I should trust the company and its products, and as such I will continue to not use them.

--------

[0] though here there the end users are devs

Post reply on HN