I'm tired of fixing customers' AI generated code • Hacker News Board

Re: I'm tired of fixing customers' AI generated code

#271

Post by berniedurfee » Thu, Aug 22, 2024, 3:48 PM UTC

alexeiz wrote: post #55

I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.

I burned a couple hours debugging some generated code only to finally realize copilot was referencing a variable as ‘variableO1’.

Artificial Incompetence indeed!

Re: I'm tired of fixing customers' AI generated code

#272

Post by dpassens » Thu, Aug 22, 2024, 3:49 PM UTC

falcor84 wrote: post #252

yifanl wrote: post #248

Earlier quoted context omitted.

for line in f: is multiple mistakes in a single line.

What are the mistakes there?

One is that the variable is called openFile and not f. I don't know enough python to see something else wrong with that but would love to know too, since I've written such a line just last week.

Re: I'm tired of fixing customers' AI generated code

#273

Post by dpassens » Thu, Aug 22, 2024, 3:53 PM UTC

maxidorius wrote: post #263

pacoWebConsult wrote: post #237

Seems to me like you have an opportunity to develop a couple SDKs in your customers' favorite languages (probably python and typescript) and a simple "Get Started" template that could alleviate a lot of these requests. Show them the base case, let them figure out how to work with your api via an SDK instead of directly with an API and let the advanced users build their own SDKs in whatever language they prefer. Since…

And then the customers will open support requests for code generated by an AI that misuse that very SDK. It doesn't look like OP's issue is with the code per say, only with the lack of skills of its customers, regardless of the code they write...

I don't really know how to point this out without sounding rude and obnoxious, which is not my intention, but it's "per se" (Latin for by itself), not "per say".

Re: I'm tired of fixing customers' AI generated code

#274

Post by pacoWebConsult » Thu, Aug 22, 2024, 3:56 PM UTC

maxidorius wrote: post #263

pacoWebConsult wrote: post #237

Seems to me like you have an opportunity to develop a couple SDKs in your customers' favorite languages (probably python and typescript) and a simple "Get Started" template that could alleviate a lot of these requests. Show them the base case, let them figure out how to work with your api via an SDK instead of directly with an API and let the advanced users build their own SDKs in whatever language they prefer. Since…

And then the customers will open support requests for code generated by an AI that misuse that very SDK. It doesn't look like OP's issue is with the code per say, only with the lack of skills of its customers, regardless of the code they write...

At the very least, GitHub Copilot will have an easier time with an SDK loaded into context than an API documented only on the web. If the customers are using typescript then they'll have some red squiggles that at least some of these people will bother to read prior to asking for help. The uninformed consumer of OPs API will probably be more comfortable to work with an SDK instead of writing their own clients.

The way I see it is that OP can either complain about customers being annoying, which will happen whether or not OP does anything about his problem, or OP could proactively produce something to make their product better for the demographic they're targeting. At this point it's pretty clear that the users would rather be helped than help themselves, so meeting them where they're at is going to be more productive than trying to educate them on why they suck at both coding and asking for help (or free work).

Re: I'm tired of fixing customers' AI generated code

#275

Post by aidos » Thu, Aug 22, 2024, 3:57 PM UTC

sa-code wrote: post #250

Tainnor wrote: post #142

Earlier quoted context omitted.

And any decent IDE will highlight a variable that is declared but unused. We already have "artificial intelligence" in the form of IDEs, linters, compilers, etc. but some people apparently think we should just throw it all away now that we have LLMs.

Unless you're using Python and said variable was meant to be reassigned, but you used a different name instead. E.g. file_name = 1 filename = 2

Fairly sure the linters would catch that (unless you referenced both of them in later code).

Re: I'm tired of fixing customers' AI generated code

#276

Post by __MatrixMan__ » Thu, Aug 22, 2024, 4:00 PM UTC

sim7c00 wrote: post #153

__MatrixMan__ wrote: post #93

Earlier quoted context omitted.

I'm similarly sympathetic when I come across a buggy implementation. It's malicious design that I'm objecting to. But I suppose it's a bit silly to say that it's the computer that's treating me like garbage. It's just that someone else is in control of my computer, and they're treating me like garbage.

sadly these days, it seems a keen mind is only a machine's mind. People spend far to little time to understand what they are telling their poor computers to do. And look what happens, people start turning against them. Blaming them for their misdoing. It's like Blaming god, the government, a nation, family or a tribe. These are all made of humans. Human bad behavior is at the core of our suffering. Nothing else.

I don't disagree substantively, but I do think there are uniquely modern aspects to the question of "am I enabling bad behavior right now?". It's not just ethics, it's education.

Consider for instance the remote support features that Intel is so keen on advertising these days. Microcode level remote access is a small help for IT departments and a huge help for authoritarian regimes looking to spy on their people. But I don't think that most people are prepared to consider what they're enabling by paying Intel to continue to grow into a telescreen vendor.

Sure, we shouldn't blame the computer's soul for bad behavior. But if it's being used as a weapon, it's not helpful to remove the computer from the conversation and say "well it's actually bad people." Mitigating bad behavior via computer means hacking that computer, and that starts with blaming it for the bad behavior to some degree.

Re: I'm tired of fixing customers' AI generated code

#277

Post by Raicuparta » Thu, Aug 22, 2024, 4:06 PM UTC

delifue wrote: post #130

Someone mentioned "hallucination-based API design" on twitter (I cannot find it now). It's designing API by LLM hallucination. If there is a common hallucination API call, just add that API. This will make the API more "friendly" and resemble common similar APIs. Considering that LLM can hallucinate in different ways unpredictably, not sure whether it will work in practice.

You're going at it all wrong. You just need another LLM on the other end of the API too.

Re: I'm tired of fixing customers' AI generated code

#278

Post by yifanl » Thu, Aug 22, 2024, 4:09 PM UTC

falcor84 wrote: post #252

yifanl wrote: post #248

Earlier quoted context omitted.

for line in f: is multiple mistakes in a single line.

What are the mistakes there?

f doesn't refer to anything.

iterating over the file object at all instead of just calling self.cache = openFile.readlines() means that calling strip() the line below removes data beyond just the trailing newlines.

Re: I'm tired of fixing customers' AI generated code

#279

Post by jrochkind1 » Thu, Aug 22, 2024, 4:11 PM UTC

I am no partiuclar fan of AI (and actually haven't used copilot or similar myself ever), but the real problem here is not AI but:

> I’ve gotten a number of angry messages from customers who essentially want me to build their whole app for free.

I guess AI maybe encourages more people to think they can build something without knowing what they are doing. i can believe that for sure. There was plenty of that before AI too. Must be even worse now.

Re: I'm tired of fixing customers' AI generated code

#280

Post by lacoolj » Thu, Aug 22, 2024, 4:13 PM UTC

lol this is just like trying to help people in programming discords that are literally using AI on screen to write and rewrite the entire app as they go. then they run into an issue, ask for help, and don't understand when you say "there's a memory leak 50 lines down" or "you have to define that variable first".

AI is a great tool to help someone start an idea. When it goes past that, please don't ask us for help until you know what the code is doing that you just generated.