I fear the day when legal matters and other impactful decisions are decided and similarly hallucinated by AI.
Cursor IDE support hallucinates lockout policy, causes user cancellations
571–580 of 635 posts
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#572Earlier quoted context omitted.
Yes, so in my first example in the GP, this happens first. Humans do the work. The calculator double checks and gives me a list of all errors plus 5% of the non-errors, and I only need to double check that list. In my third example, the calculator does the hard work of dividing, and humans can validate by the simpler task of multiplication, only having to do extra work 5% of the time. (In my second, the unreliablity…
I'd like to second the point made to you in this thread that went without reply: https://news.ycombinator.com/item?id=43702895 It's true that we use tools with uncertainty all the time, in many domains. But crucially that uncertainty is carefully modeled and accounted for. For example, robots use sensors to make sense of the world around them. These sensors are not 100% accurate, and therefore if the robots rely on t…
>went without reply
Yeah, I was getting a little self-conscious about replying to everyone and repeating myself a lot. It felt like too much noise.
But my first objection here is to repeat myself- none of my examples are sensitive to this problem. I don't need to understand what conditions cause the calculator/IDE/medical test/LLM to fail in order to benefit from a 95% success rate.
If I write a piece of code, I try to understand what it does and how it impacts the rest of the app with high confidence. I'm still going to run the unit test suite even if it has low coverage, and even if I have no idea what the tests actually measure. My confidence in my changes will go up if the tests pass.
This is one use of LLMs for me. I can refactor a piece of code and then send ChatGPT the before and after and ask "Do these do the same thing". I'm already highly confident that they do, but a yes from the AI means I can be more confident. If I get a no, I can read its explanation and agree or disagree. I'm sure it can get this wrong (though it hasn't after n~=100), but that's no reason to abandon this near-instantaneous, mostly accurate double-check. Nor would I give up on unit testing because somebody wrote a test of implementation details that failed after a trivial refactor.
I agree totally that having a good model of LLM uncertainty would make them orders of magnitude better (as would, obviously, removing the uncertainty altogether). And I wouldn't put them in a pipeline or behind a support desk. But I can and do use them for great benefit every day, and I have no idea why I should prefer to throw away the useful thing I have because it's imperfect.
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#573This is "AGI's finest". It's what we all wanted. Replacing your human support team to be run exclusively by AI LLM bots whilst they hallucinate to their users. All unchecked. Now this bug has now turned into a multi-million dollar mistake and costed Cursor to lose millions of dollars overnight. What if this was a critical control system in a hospital or energy company and their AI support team (with zero humans) hall…
[deleted]
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#574Earlier quoted context omitted.
Code review is actually one of the few practices for which research does exist[0] which points in the direction of it being generally good at reducing defects. Additionally, in the example you share, where only one person knows the context of the change, code review is an excellent tool for knowledge sharing. [0]: https://dl.acm.org/doi/10.1145/2597073.2597076 , for example
Oh I have no doubt it's an excellent tool for knowledge sharing. So are mailing lists (nobody reads email) and internal wikis (evergreen fist fight to get someone, anyone, to update). Despite best intentions knowledge sharing regimes are little more than well-intentioned pestering with irrelevant information that is absolutely purged from headspace during any number of daily/weekly/quarterly context switches. As I sa…
You're free to believe whatever you want, but again, this is one of the few things that we actually empirically know to be working.
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#575(Cursor cofounder) Apologies - something very clearly went wrong here. We’ve already begun investigating, and some very early results: * Any AI responses used for email support are now clearly labeled as such. We use AI-assisted responses as the first filter for email support. * We’ve made sure this user is completely refunded - least we can do for the trouble. For context, this user’s complaint was the result of a r…
Why did you remove this thread?
https://old.reddit.com/r/cursor/comments/1jyy5am/psa_cursor_...
(For reference, here it is in reveddit https://www.reveddit.com/v/cursor/comments/1jyy5am/psa_curso... - text from post was unfortunately not saved)
It's already locked and with a stickied comment from a dev clarifying what happened
Did you remove it so people can't find about this screwup when searching Google?
Anyway, if you acknowledge it was a mistake to remove the thread, could you please un-remove it?
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#576Earlier quoted context omitted.
How do you define code quality in this case and what is your stack?
The definition of code quality is irrelevant to my argument as both human and AI written code are held to the same standard by the same measure (however arbitrary that measure is). 100 units of something vs 99 units of something is a 1 unit difference regardless of what the unit is. By the time the AI is actually writing code, I've already had it do a robust architecture evaluation and review which it documents in a…
> The definition of code quality is irrelevant to my argument
Understand. Nevertheless, human engineers may deliberately choose certain level of quality and accept certain risks (quality of output is not direct measure of professionalism, so the question wasn’t pointed at your skill) — it‘s good that AI is matching your expectations, but it’s important to understand what are they for your projects.
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#577Earlier quoted context omitted.
> Code that you can understand and fix later, is acceptable quality per my definition. This definition limits the number of problems you can solve this way. It basically means buildup of the technical debt - good enough for throwaway code, unacceptable for long term strategy (growth killer for scale-ups). >Either way, LLMs are actually high up the quality spectrum This is not what I saw, it’s certainly not great. But…
I'm curious were you in an existing code base or a greenfield project? I've found LLMs tend to struggle getting a codebase from 0 to 1. They tend to swap between major approaches somewhat arbitrarily. In an existing code base, it's very easy to ground them in examples and pattern matching.
Greenfield. It’s an interesting question though, if on today‘s project some model will perform better tomorrow because of more reference data. I would expect LLMs to lag behind on latest technology, simply because their reference data has more older examples and may not include latest versions of platforms or frameworks. I have seen LLMs breaking on basic CRUD tasks because of that.
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#578Earlier quoted context omitted.
> This is not a problem in my unreliable calculator use-cases; are you disputing that or dropping the analogy? If you use an unreliable calculator to sum a list of numbers, you then need to use a reliable method to sum the numbers to validate that the unreliable calculator's sum is correct or incorrect.
Yes, so in my first example in the GP, this happens first. Humans do the work. The calculator double checks and gives me a list of all errors plus 5% of the non-errors, and I only need to double check that list. In my third example, the calculator does the hard work of dividing, and humans can validate by the simpler task of multiplication, only having to do extra work 5% of the time. (In my second, the unreliablity…
In your first example, you appear to assume that for calculations where "each mistake could cost $millions or lives", engineers who calculated by hand typically didn't double-check by redoing the calculation, so a second check with a 95% accuracy tool is better than nothing. This assumption is false. I suggest you watch the 2016 film Hidden Figures to understand the level of safety at NASA when calculations were done by hand. You are suggesting lowering safety standards, not increasing them.
Your third example is unclear. No calculators can perform factoring of large numbers, because that is the expected ability of future quantum computers that can break RSA encryption. It is also unclear why multiplication and division have different difficulties, when dividing by n is equal to multiplying by 1/n.
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#579Earlier quoted context omitted.
You’ve promised a ton of people refunds that never got them. Others in this thread, and myself included Edit: he did refund 22 mins after seeing this
Yeah I got asked for feedback and offered a refund when I cancelled. Never got any reply after. Guess it was AI slop
The email I received in case anyone is wondering:
Hi ,
I'm Michael, the founder and CEO of Cursor. I noticed you recently canceled your subscription, and I wanted to check in. If we fell short for you, I want to learn why and make it right.
First, if you'd like me to refund your account, please just reply to this email to let me know. I'd be happy to.
Second, could you share a sentence or two on what you disliked about Cursor? Or perhaps a screenshot of where it performed poorly? This will help us improve the product for future users.
I'd be very grateful to understand your candid thoughts. I'm listening and eager to fix our experience for you. Wishing you the best in any case!
Best, Michael
And my reply which I never got a response to:
Hi Michael,
Thanks for reaching out. I have honestly found that recent updates to the app have been extremely detrimental to the DX and productivity. A couple big issues I have found:
1. Removing the floating component window and providing no way to get it back. As a dev who is often travelling and working from a laptop screen I found the floating window to be extremely handy and its removal essentially meant I just do not use the composer anymore.
2. Constantly overriding VSCode native shortcuts. This is the most detrimental thing I have experienced, personally. Shortcuts are crucial to productivity and are engrained in muscle memory over years. Overriding them is essentially removing years of learned behaviour (things like cmd + shift + l)
3. The floating completion windows. These often end up overlapping my code code and break my flow. I have to press escape to close it and the whole experience is just jarring
4. Making the cursor dance around the screen when suggesting completions. I get that completions can sometimes be handy but moving my cursor when I am in the flow just makes me rethink what I am doing so that I can read a guess as to what I want which is often incorrect.
5. Poor suggestions. In the past month (maybe two?) I have noticed the quality of prompts is not up to par. I often find that Cursor will do weird things like import `jest.Mock` in my unit tests when I have not used jest in any repo which I have been actively maintaining since using cursor.
As for the refund I will leave that decision for you. I knew I was entering beta software when I ordered accepted the terms so I wouldn't be annoyed as such. Though, saying that, I do find myself using cursor a lot less now and am going to most likely shift back to plain VSCode as a result of the above.
If I think of anything else I will let you know.
Thanks,
Love, Peace and Happiness,
Re: Cursor IDE support hallucinates lockout policy, causes user cancellations
#580LLM anything makes me queasy. Why would any self respecting software developer use this tripe? Learn how to write good software. Become an expert in the trade. AI anything will only dig a hole for software to die in. Cheapens the product, butchers the process and absolutely decimates any hope for skill development for future junior developers. I'll just keep chugging along, with debian, python and vim, as I always ha…
If you are like me (same vim, python, no LLM, no autocompletion, no syntax highlighting noise), LSP will make you a better developer: it makes navigating the codebase MUCH easier, including stdlib and 3rd party dependencies.
As a result, you don't lose flow and end up reading considerably more code than you would have otherwise.