Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm tired of fixing customers' AI generated code

medium.com

Post reply on HN
Print view View original

301–310 of 345 posts

Re: I'm tired of fixing customers' AI generated code

#301

Post by OJFord »

falcor84 wrote: post #251
delusional wrote: post #116

Earlier quoted context omitted.

We hired a new guy at work. In one of his first tasks he had chosen to write some bash, and it was pure nonsense. I mean it contained things like: if [ -z "${Var}+x" ] Where I can see what the author was trying to do, but the code is just wrong. I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, b…

I agree that it's a waste of a learning opportunity, but from my experience it is still often rational. There were many times in my career when I had what I expected to be a one-off issue that I needed a quick solution for and I would look for a quick and simple fix with a tool I'm unfamiliar with. I'd say that 70% of the time the thing "just works" well enough after testing, 10% of the time it doesn't quite work but…

I feel similarly that some such learning opportunities are just going to be larger rabbit holes than the thing is worth, but in those cases I'll just prefer to do it a different way that I do know or is worth learning.

E.g. maybe it would be very 'elegant' or rather concise awk if I could overcome the learning opportunity, but like you I would probably decide not to; I'll do it with the sed I do know even if it means some additional piping and cutting or grepping or whatever that awk could've done in one, because I already know it and it's going to be clearer to me and probably anyone else I'm working with.

I think we're saying quite similar things, but my point is I wouldn't be deleting it, dismissing the idea, and disappointing the colleague ready to teach me about it - because I never would've been willing to blindly try broken AI generated (or however sourced) code that I didn't understand in the first place.

Re: I'm tired of fixing customers' AI generated code

#302

Post by davidthewatson »

yawnxyz wrote: post #258
alexeiz wrote: post #55

I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.

Claude gave me something similar, except these were both used, and somehow global variables, and it got confused about when to use which one. Asking it to refactor / fix it made it worse bc it'd get confused, and merge them into a single variable — the problem was they had slightly different uses, which broke everything I had to step through the code line by line to fix it. Using Claude's still faster for me, as it'd…

I'd refer you to a comment I made a few weeks ago on an HN post, to the same effect, which drew the further comment from gwern here:

https://news.ycombinator.com/item?id=40922090

LSS: metaprogramming tests is not trivial but straightforward, given that you can see the code, the AST, and associated metadata, such as generating test input. I've done it myself, more than a decade ago.

I've referred to this as a mix of literate programming (noting the traps you referred to and the anachronistic quality of them relative to both the generated tests and their generated tested code) wrapped up in human-computer sensemaking given the fact that what the AI sees is often at best a lack in its symbolic representation that is imaginary, not real; thus, requiring iterative correction to hit its user's target, just like a real test team interacting with a dev team.

In my estimation, it's actually harder to explain than it is to do.

Re: I'm tired of fixing customers' AI generated code

#303

Post by davidthewatson »

n4r9 wrote: post #157
Piskvorrr wrote: post #154

Earlier quoted context omitted.

We aren't. LLMs may have been useful for a moment in time, before the trick "it's now MY OWN creation, no IP strings attached - when it comes through the plagiarism machine" became apparent, and before the models started eating their own tail. Now they're just spiralling down, and it will IMNSHO take something else than an iterative "a future version will surely fix this, One Day, have faith."

There are signs of a decline in people asking and answering questions on sites like stack exchange: https://meta.stackexchange.com/questions/387278/has-stack-ex... So I hope you're right, but the evidence is currently that you're wrong. Let's see how it plays out, I suppose.

I upvoted your comment because I'm afraid you may be correct. I say, "afraid" because I can remember the day when a member of my team was fired for copy pasta from SO with little, if any understanding, into "production" code.

The problem, of course, is that this might work once in a while for low hanging fruit, until the web inherited things like DICOM and we now have medical imaging in the web browser (I've heard in Apple Vision Pro), where robotics implies the price of unforeseen bugs is not accidental death or dismemberment of one patient, but potentially many.

Re: I'm tired of fixing customers' AI generated code

#304

Post by gopher_space »

danenania wrote: post #298
withinboredom wrote: post #295

Earlier quoted context omitted.

As someone who uses LLMs on my hobby projects to write code, I’ve found the opposite. I usually fix the code, then send it in saying it is a refactor to clarify things. It seems to work well enough. If it is rather complex, I will paste the broken code into another conversation and ask it to refactor/explain what is going on.

Fixing the mistake yourself and then sending the code back is a positive example, since you're demonstrating the correct way rather than asking for a fix. But in my experience, if you continue iterating from that point, there's still a risk that parts of the original broken code can leak back into the output again later on since the broken code is still in context. Ymmv of course and it definitely depends a lot on th…

I’m attempting to keep the context ball rolling by reiterating key points of a request throughout the conversation.

The challenge is writing in a tone that will gently move the conversation rather than refocus it. I can’t just inject “remember point n+1” and hope that’s not all it’ll talk about in the next frame.

If nothing else, LLMs have helped me understand exactly why GIGO is a fundamental law.

Re: I'm tired of fixing customers' AI generated code

#305

Post by mhuffman »

jtbayly wrote: post #281
Piskvorrr wrote: post #169

Earlier quoted context omitted.

- Which might be a different matter: of specifically SE declining. (A very different, and long-running, tragedy, but one that began long before the current AI boom and prompted by very different, non-technical issues.) - That said, surely traffic will decline for Q&A sites. "How do I connect tab A into slot B" is something that people are likely to query LLMs for; the response will surely sound authoritative, and cou…

But what happens with the next generation of questions? The reason LLMs can answer how to right-align a paragraph in HTML is at least in part because it has been asked and answered publicly so many times. Now imagine that HTMZ comes along and people just go straight to asking how to full justify text in HTMZ for their smart bucket. What happens? I doubt we’ll get good answers. It feels like the test of whether LLMs c…

>Now imagine that HTMZ comes along and people just go straight to asking how to full justify text in HTMZ for their smart bucket. What happens? I doubt we’ll get good answers.

So, I think the answer is that since all useful data is already in a LLM somewhere all new data will be stolen/scraped and inserted in real time. So if real people are answering the question it will work as normal. The real question is what happens when people are trying to mine karma by answering questions using an LLM that is hallucinating. We have seen such with the Bug Bounty silliness going on.

Re: I'm tired of fixing customers' AI generated code

#306

Post by romwell »

aidos wrote: post #275
sa-code wrote: post #250

Earlier quoted context omitted.

Unless you're using Python and said variable was meant to be reassigned, but you used a different name instead. E.g. file_name = 1 filename = 2

Fairly sure the linters would catch that (unless you referenced both of them in later code).

>unless you referenced both of them in later code

Generative AI: hold my bear

Re: I'm tired of fixing customers' AI generated code

#307

Post by trees101 »

viraptor wrote: post #233
floydnoel wrote: post #230

Earlier quoted context omitted.

does aider have an executable installer yet? i tried installing it but the python experience is terrible. last time i messed with python installs on my mac everything worked like shit until o reinstalled the OS.

python -mvenv aider aider/bin/pip install aider-chat aider/bin/aider And you're done. There's also a docker version https://aider.chat/docs/install/docker.html Just don't mess with the system-wide installed version of python and it will be fine. This isn't a python specific issue though.

`pipx install aider-chat`

(https://aider.chat/docs/install/pipx.html will install it globally on your system within its own python environment. This way you can use aider to work on any python project, even if that project has conflicting dependencies.)

I run it with these settings: `aider --sonnet --no-auto-commits--cache-prompts`

if you want to run the most bleeding edge version at the same time without breaking anything,

`pipx install --suffix=@dev git+https://github.com/paul-gauthier/aider.git`

then run it with `aider@dev`

`aider@dev` ... with your desired settings

Re: I'm tired of fixing customers' AI generated code

#309

Post by wiseowise »

tomrod wrote: post #211
Tainnor wrote: post #142

Earlier quoted context omitted.

And any decent IDE will highlight a variable that is declared but unused. We already have "artificial intelligence" in the form of IDEs, linters, compilers, etc. but some people apparently think we should just throw it all away now that we have LLMs.

No need for quotes, the best AI integrations are the ones you see as just part of the tech stack like spell check and linters.

But that’s WoOoOrK. These stupid sobs want magic that does everything automatically.

Re: I'm tired of fixing customers' AI generated code

#310

Post by HeyLaughingBoy »

emseetech wrote: post #66
Laakeri wrote: post #63

Earlier quoted context omitted.

But he made a good living out of it, so in the end it was a good idea?

It certainly puts a ceiling on a career. And I'd argue it probably gave him a pretty rough shelf life. At some point he has to understand what he's doing. Unless he's so good at selling his services he can consistently find new clients. And if that's the case, he'd probably kill it in sales.

Most people never reach the theoretical ceiling of their careers, so he probably did quite well.
Post reply on HN