Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm tired of fixing customers' AI generated code

medium.com

Post reply on HN
Print view View original

121–130 of 345 posts

Re: I'm tired of fixing customers' AI generated code

#121

Post by fragmede »

viccis wrote: post #119
TillE wrote: post #3

I'm always a little surprised at how many people out there want to develop software yet haven't put in the effort to gain even the most basic computer nerd programming chops. You see this all the time in the more newbie-friendly game engine communities. Maybe you don't want to pursue a career in software, but anyone can spend a week learning Python or JavaScript. I suspect/hope a lot of these people are just kids who…

One of my favorite intern stories was a kid who was a compsci senior, very good university, and who was assigned to write some Python code for my team. He had the very immature "Python is a baby's language" attitude. He wasn't working on my stuff so I don't really keep track of what he's doing, but a few weeks later I look at what he has written. Almost all of his Python functions have one parameter with an asterisk,…

A friend of mine says LLMs are good at producing bad pandas code because there's just so much of it out there to train off of.

Re: I'm tired of fixing customers' AI generated code

#122

Post by giraffe_lady »

viccis wrote: post #119
TillE wrote: post #3

I'm always a little surprised at how many people out there want to develop software yet haven't put in the effort to gain even the most basic computer nerd programming chops. You see this all the time in the more newbie-friendly game engine communities. Maybe you don't want to pursue a career in software, but anyone can spend a week learning Python or JavaScript. I suspect/hope a lot of these people are just kids who…

One of my favorite intern stories was a kid who was a compsci senior, very good university, and who was assigned to write some Python code for my team. He had the very immature "Python is a baby's language" attitude. He wasn't working on my stuff so I don't really keep track of what he's doing, but a few weeks later I look at what he has written. Almost all of his Python functions have one parameter with an asterisk,…

This sort of thing is a weird relic of CS programs doing double duty as "professional school for software development" and "undergrad prep for an academic math career."

You just don't know how much they actually learned about programming as a discipline in its own right and it very well could be functionally zero. I've seen recent CS grads who didn't know how to use git, didn't know how to split code across multiple files or understand why you would even want to.

I think there's a fairly sound argument for these being different degrees, that a certain kind of researcher doesn't necessarily need these skills. But since it isn't there's just a huge range in how schools reconcile it.

Re: I'm tired of fixing customers' AI generated code

#123

Post by sixfiveotwo »

creesch wrote: post #113
brigadier132 wrote: post #111

Earlier quoted context omitted.

This is a strawman, he never said it didn't make mistakes.

Oh for crying out loud, I obviously mean these specific mistakes. If you have worked in any capacity with LLMs like this you would have seen them variables or suddenly switch up the convention of how they're written. Certainly if you are in a conversation mode after a few back and forths this happens from time to time. I am just not going to spend my time digging to previous prompts of code I might not want to share…

What you guys probably want to do instead is get to a common definition of what a typo is. Personally, I understand it as a typographic error, which is a fancy way of saying a spelling mistake (a mistake on a letter), not a mistake where one use a word for another.

Maybe you meant the latter?

Re: I'm tired of fixing customers' AI generated code

#124

Post by schainks »

atomic128 wrote: post #77
wh-uws wrote: post #64

I will take all of these customers. Please forward them to me.

For real: profitable consulting businesses have been formed to help LLM programmers. BUGFIX 66, for example, and various others. They can charge substantial money to help customers cross that "last mile" and get their LLM-generated pile of code working.

I am literally doing this right now for a client with a prototype, never realized the market is as big as people are saying!

Re: I'm tired of fixing customers' AI generated code

#125

Post by arcticfox »

Glyptodon wrote: post #87
alexeiz wrote: post #55

I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.

At least for me stupid bugs like this turn out to be some of the most time wasting to debug, no AI involved. Like accidentally have something quoted somewhere, or add an 's' to a variable by accident and I may not even correctly process what the error message is reporting at first. Always feel a bit silly after.

This type of bug is trivial for GPT to fix though. It was born for this. Sometimes it does generate real footguns but this sounds like an example from an earlier generation of generative AI.

Re: I'm tired of fixing customers' AI generated code

#126

Post by FanaHOVA »

creesch wrote: post #113
brigadier132 wrote: post #111

Earlier quoted context omitted.

This is a strawman, he never said it didn't make mistakes.

Oh for crying out loud, I obviously mean these specific mistakes. If you have worked in any capacity with LLMs like this you would have seen them variables or suddenly switch up the convention of how they're written. Certainly if you are in a conversation mode after a few back and forths this happens from time to time. I am just not going to spend my time digging to previous prompts of code I might not want to share…

[flagged]

Re: I'm tired of fixing customers' AI generated code

#127

Post by nbzso »

This is only the beginning. Imagine this when AI bot chains and "agents" replace conveniently junior devs on scale. Someone will "hack" an API/Lang Chain/Insert-LLM-Framework Solution.

The next decade of support business is here. Fix my "hallucination" market. Thank you, Microsoft. You did it again.

Re: I'm tired of fixing customers' AI generated code

#128

Post by mulmen »

malfist wrote: post #37
dylan604 wrote: post #29

Earlier quoted context omitted.

I read a "Teach yourself $language in 10 days" book in a weekend, and was banging code on Monday to create the first v0.1 in a week. Of course the code was absolutely horrendous, but it worked. I still have a copy of that old database that was used, and over the years, I have turned to it as I've learned new things and have even rewritten the UI a couple of times. It has helped me stay up to date with new trends as i…

But you're building on the context of knowing a different language. Picking up the second, fifth, or nth language is easy, as long as it isn't the first one.

I strongly disagree with this. It’s more like languages hit a common wavelength. Sometimes it makes sense, sometimes it doesn’t.

Re: I'm tired of fixing customers' AI generated code

#129

Post by f6v »

frumper wrote: post #60
alexeiz wrote: post #55

I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.

I knew a guy that made a good living as a freelance web developer decades ago. He would pretty much just copy and paste code from tutorials or stack overflow and had no real idea how anything worked. Using code without understanding it is never a good idea, it doesn’t need to be from AI for that to be true.

Or maybe you’re just exaggerating. I’ve done my fair share of copy pasting and it never worked to just do it without understanding what’s going on.

I think the problem with “AI” code is that many people have almost a religions belief. There’re weirdos on internet who say that AGI is couple years away. And by extension current AI models are seen as something incapable of making a mistake when writing code.

Re: I'm tired of fixing customers' AI generated code

#130

Post by delifue »

Someone mentioned "hallucination-based API design" on twitter (I cannot find it now). It's designing API by LLM hallucination. If there is a common hallucination API call, just add that API. This will make the API more "friendly" and resemble common similar APIs.

Considering that LLM can hallucinate in different ways unpredictably, not sure whether it will work in practice.

Post reply on HN