Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cursor IDE support hallucinates lockout policy, causes user cancellations

old.reddit.com

Post reply on HN
Print view View original

261–270 of 635 posts

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#261

Post by marcus_holmes »

jgb1984 wrote: post #125

LLM anything makes me queasy. Why would any self respecting software developer use this tripe? Learn how to write good software. Become an expert in the trade. AI anything will only dig a hole for software to die in. Cheapens the product, butchers the process and absolutely decimates any hope for skill development for future junior developers. I'll just keep chugging along, with debian, python and vim, as I always ha…

I was with you 150% (though Arch, Golang and Zed) until a friend convinced me to give it a proper go and explained more about how to talk to the LLM.

I've had a long-term code project that I've really struggled with, for various reasons. Instead of using my normal approach, which would be to lay out what I think the code should do, and how it should work, I just explained the problem and let the LLM worry about the code.

It got really far. I'm still impressed. Claude worked great, but ran out of free tokens or whatever, and refused to continue (fine, it was the freebie version and you get what you pay for). I picked it up again in Cursor and it got further. One of my conditions for this experiment was to never look at the code, just the output, and only talk to the LLM about what I wanted, not about how I wanted it done. This seemed to work better.

I'm hitting different problems, now, for sure. Getting it to test everything was tricky, and I'm still not convinced it's not just fixing the test instead of the code every time there's a test failure. Peeking at the code, there are several remnants of previous architectural models littering the codebase. Whole directories of unused, uncalled, code that got left behind. I would not ship this as it is.

But... it works, kinda. It's fast, I got a working demo of something 80% near what I wanted in 1/10 of the time it would have taken me to make that manually. And just focusing on the result meant that I didn't go down all the rabbit holes of how to structure the code or which paradigm to use.

I'm hooked now. I want to get better at using this tool, and see the failures as my failures in prompting rather than the LLM's failure to do what I want.

I still don't know how much work would be involved in turning the code into something I could actually ship. Maybe there's a second phase which looks more like conventional development cleaning it all up. I don't know yet. I'll keep experimenting :)

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#262

Post by dgs_sgd »

seabird wrote: post #220
Pedro_Ribeiro wrote: post #182

Earlier quoted context omitted.

Is it just me or has there been a wave of delusional people on Hacker News completely neglecting new advancements in technology? The two most common technologies I see having this type of discourse are AI coding and containers. Either everyone here is a low level quantum database 5D graphics pipeline developer with a language from the future that AI hasn't yet learned, or some people are in denial.

I'm primarily an embedded firmware developer. Gas/electric power products. Ada codebase, so it's off the beaten path but nothing academic by any stretch of the imagination. I have a comprehensive reference manual that describes exactly how the language should be working, and don't need an LLM to regurgitate it to me. I have comprehensive hardware and programming manuals for the MCUs I program that describe exactly ho…

Software development is a spectrum and you're basically on the polar opposite end of the one AI is being used for: sloppy web dev.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#263

Post by poink »

cryptoegorophy wrote: post #180
nerdjon wrote: post #63

There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore and then a company that would benefit from that narrative gets directly hurt by it. Which of course they are going to try to brush it all away. Better than admitting that this problem very much still exists and isn’t going away anytime soon.

I think that’s why Apple is very slow at rolling out AI if it ever actually will. Downside is way too big than the upside.

Yet Apple has reenabled Apple Intelligence multiple times on my devices after OS updates despite me very deliberately and angrily disabling it multiple times

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#264

Post by shipp02 »

schmichael wrote: post #168
yodsanklai wrote: post #131

Earlier quoted context omitted.

You're not supposed to trust the tool, you're supposed to review and rework the code before submitting for external review. I use AI for rather complex tasks. It's impressive. It can make a bunch of non-trivial changes to several files, and have the code compile without warnings. But I need to iterate a few times so that the code looks like what I want. That being said, I also lose time pretty regularly. There's a le…

> You're not supposed to trust the tool This is just an incredible statement. I can't think of another development tool we'd say this about. I'm not saying you're wrong, or that it's wrong to have tools we can't just, just... wow... what a sea change.

In Mechanical Engineering, this is 100% a thing with fluid dynamics simulation. You need to know if the output is BS based on a number of factors that I don't understand.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#265

Post by stogot »

saintfire wrote: post #247
cryptoegorophy wrote: post #180

Earlier quoted context omitted.

I think that’s why Apple is very slow at rolling out AI if it ever actually will. Downside is way too big than the upside.

You say slowly, but in my opinion Apple made an out of character misstep by releasing a terrible UX to everyone. Apple intelligence is a running joke now. Yes they didn't push it as hard as, say, copilot. I still think they got in way too deep way too fast.

Fast!? They were two years slow and still fell face flat, and then rolled back the software

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#266

Post by ericwood »

pizza wrote: post #243
ericwood wrote: post #209

Earlier quoted context omitted.

I have a hard time being sold on “yea it’s wrong a lot, also you have to spend more time than you already do on code review.” Getting to sit down and write the code is the most enjoyable part of the job, why would I deprive myself of that? By the time the problem has been defined well enough to explain it to an LLM sitting down and writing the code is typically very simple.

The parts worth thinking about you still think about. The parts that you’ve done a million times before you delegate so you can spend better and greater effort on the parts worth thinking about.

This is where the disconnect is for me; mundane code can sometimes be nefarious, and I find the mental space I'm in when writing it is very different than reviewing, especially if my mind is elsewhere. The best analogy I can use is a self-driving car, where there's a chance at any point it could make an unpredictable and potentially fatal move. You as the driver cannot trust it but are not actively engaged in the act of driving and have a much higher likelihood of being complacent.

Code review is difficult to get right, especially if the goal is judging correctness. Maybe this is a personal failing, but I find being actively engaged to be a critical part of the process; the more time I spend with the code I'm maintaining (and usually on call for!) the better understanding I have. Tedium can sometimes be a great signal for an abstraction!

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#267

Post by AIPedant »

permo-w wrote: post #203
cs702 wrote: post #143

Yup, hallucinations are still a big problem for LLMs. Nope, there's no reliable solution for them, as of yet. There's hope that hallucinations will be solved by someone, somehow, soon... but hope is not a strategy. There's also hype about non-stop progress in AI. Hype is more a strategy... but it can only work for so long. If no solution materializes soon, many early-adopter LLM projects/trials will be cancelled. Sig…

trying to "fix hallucinations" is like trying to fix humans being wrong. it's never going to happen. we can maybe iterate towards an asymptote, but we're never going to "fix hallucinations"

No: an LLM that doesn't confabulate will certainly get things wrong in some of the same ways that honest humans do - being misinformed, confusing similar things, "brain" damage from bad programming or hardware errors. But LLM confabulations like the one we're discussing only occur in humans when they're being sociopathically dishonest. A lawyer who makes up a court case is not a "human being wrong," it's a human lying, intentionally trying to deceive. When an LLM does it, it's because it is not capable of understanding that court cases are real events that actually happened.

Cursor's AI agent simply autocompleted a bunch of words that looked like a standard TOU agreement, presumably based on the thousands of such agreements in its training data. It is not actually capable of recognizing that it made a mistake, though I'm sure if you pointed it out directly it would say "you're right, I made a mistake." If a human did this, making up TOU explanations without bothering to check the actual agreement, the explanation would be that they were unbelievably cynical and lazy.

It is very depressing that ChatGPT has been out for nearly three years and we're still having this discussion.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#268

Post by tevon »

schmichael wrote: post #168
yodsanklai wrote: post #131

Earlier quoted context omitted.

You're not supposed to trust the tool, you're supposed to review and rework the code before submitting for external review. I use AI for rather complex tasks. It's impressive. It can make a bunch of non-trivial changes to several files, and have the code compile without warnings. But I need to iterate a few times so that the code looks like what I want. That being said, I also lose time pretty regularly. There's a le…

> You're not supposed to trust the tool This is just an incredible statement. I can't think of another development tool we'd say this about. I'm not saying you're wrong, or that it's wrong to have tools we can't just, just... wow... what a sea change.

Stackoverflow is like this, you read an answer but are not fully sure if its right or if it fits your needs.

Of course there is a review system for a reason, but we frequently use "untrusted" tools in development.

That one guy in a github issue that said "this worked for me"

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#269

Post by gitremote »

ToValueFunfetti wrote: post #251
ryandrake wrote: post #212

Earlier quoted context omitted.

Imagine if your compiler just randomly and non-deterministically compiled valid code to incorrect binaries, and the tool's developer couldn't really tell you why it happens, how often it was expected to happen, how severe the problem was expected to be, and told you to just not trust your compiler to create correct machine code. Imagine if your calculator app randomly and non-deterministically performed arithmetic in…

If the only calculators that existed failed at 5% of the calculations, or if the only communication tools miscommunicated 5% of the time, we would still use both all the time. They would be far less than 95% as useful as perfect versions, but drastically better then not having the tools at all.

Absolutely not. We'd just do the calculations by hand, which is better than running the 95%-correct calculator and then doing the calculations by hand anyway to verify its output.

Re: Cursor IDE support hallucinates lockout policy, causes user cancellations

#270

Post by tevon »

ryandrake wrote: post #218
forgetfreeman wrote: post #199

Earlier quoted context omitted.

"Do you trust other developers to write good code without mistakes without getting it reviewed by others." Literally yes. Test coverage and QA to catch bugs sure but needing everything manually reviewed by someone else sounds like working in a sweatshop full of intern-level code bootcamp graduates, or if you prefer an absolute dumpster fire of incompetence.

I would accept mistakes and inconsistency from a human, especially one not very experienced or skilled. But I expect perfection and consistency from a machine. When I command my computer to do something, I expect it to do it correctly, the same way every time, to convert a particular input to an exact particular output, every time. I don't expect it to guess, or randomly insert garbage, or behave non-deterministicall…

This seems like a particularly limited view of what a machine is. Specifically expecting it to behave deterministically.
Post reply on HN