Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm tired of fixing customers' AI generated code

medium.com

Post reply on HN
Print view View original

161–170 of 345 posts

Re: I'm tired of fixing customers' AI generated code

#161

Post by jonplackett »

I would suggest adding a help section with advice for people using ChatGPT that sets expectations and also gives a pre-written prompt for them to use.

Something like.

Some of you may use ChatGPT/Copilot to help with your coding. Please understand this service is designed for professional programmers and we cannot provide extensive support for basic coding issues, or code your app for you.

However if you do want to use ChatGPT here is a useful starting prompt to help prevent hallucinations - though they still could happen.

Prompt:

I need you to generate code in ____ language using only the endpoints described below

[describe the endpoints and include all your docs]

Do not use any additional variables or endpoints. Only use these exactly as described. Do not create any new endpoints or assume any other variables exist. This is very important.

Then give it some examples in in curl / fetch / axios / python / etc.

Maybe also add some instructions to separate out code into multiple files / endpoints. ChatGPT loves to make one humungous file that’s really hard to debug.

ChatGPT works fairly well if you know how to use it correctly. I would defo not trust it with my crypto though, but I guess if some people wanna that’s up to them. May as well try and help them!

Re: I'm tired of fixing customers' AI generated code

#162

Post by creesch »

protocolture wrote: post #26

Yeah so every tech revolution does this right. ATMs were meant to kill banking jobs but ah theres more jobs in banking than ever. The Cloud was meant to automate away tech people, but all it did was create new tech jobs. A lot of which is cleaning up after idiots who think they can outsource everything to the cloud and get burned. LLMs are no different. The "Ideas Man" can now get from 0 to 30% without approaching a…

> ATMs were meant to kill banking jobs but ah theres more jobs in banking than ever.

US banks, who are surprisingly behind the times in as far as automation goes. Here a lot of banks used a lot of automation to reduce the amount of manual jobs needed. to the degree that many offices are now also closing as everything can be done online.

And no, there is no need to visit banks here as I get the impression it is in the US. We don't even have physical checks anymore.

Re: I'm tired of fixing customers' AI generated code

#163

Post by fenomas »

ben_w wrote: post #145
Pikamander2 wrote: post #131

> Often this takes the form of trying to access an endpoint that does not exist, or read a property off the API response that does not exist. After probing a bit more, my suspicions are usually confirmed — ChatGPT hallucinated that endpoint or property In some cases, you might be able to use this to your advantage to improve the product. When working with third-party APIs, I've often run into situations where my code…

> It's possible that ChatGPT is "thinking" the same thing here; that most APIs have an X endpoint to make a task easier, so surely yours does too? While this is possible, I would caution with an anecdote from my ongoing side project of "can I use it to make browser games?", in which 3.5 would create a reasonable Vector2D class and then get confused and try to call .mul() and .sub() instead of the .multiply() and .sub…

I feel like we'll eventually all agree that it's a mistake to ask a generalist LLM for code. I've found ChatGPT to be fine at talking about code - like describing the difference between two APIs - but for generating nontrivial chunks of working code I think it's miles behind Copilot and similar.

And I assume that's just because, y'know, ChatGPT can write sonnets and translate Korean to Swahili and whatnot. It's amazingly broad, but it's not the right tool for the comparatively narrow problem of code generation.

Re: I'm tired of fixing customers' AI generated code

#165

Post by gorbachev »

PeterStuer wrote: post #101
gorbachev wrote: post #96

I've been saying for a while now that there's an absolute gold mine waiting for people who want to specialize in fixing AI generated applications. A lot of businesses are going to either think they can have generative AI create all of their apps with the help of a cousin of the wife of the accountant, or they unknowingly contract a 10x developer from Upwork or alike who uses generative AI to create everything. Once t…

Doesn't solve the problem that the budget they has in mind for the app was $300, while an experienced dev can directly see this is going to be a $20K v.1 before change requests project.

So they'll try 5 $300 fixes, and then either give up entirely, or figure out that maybe the 100 developers they ignored who told them it's gonna cost $20K were right.

Re: I'm tired of fixing customers' AI generated code

#166

Post by gorbachev »

mrbombastic wrote: post #17
TillE wrote: post #3

I'm always a little surprised at how many people out there want to develop software yet haven't put in the effort to gain even the most basic computer nerd programming chops. You see this all the time in the more newbie-friendly game engine communities. Maybe you don't want to pursue a career in software, but anyone can spend a week learning Python or JavaScript. I suspect/hope a lot of these people are just kids who…

A week is not anywhere close to enough to learn programming in any meaningful way for someone with no experience.

The other thing is that to work on anything really meaningful takes time and effort. It takes determination to struggle through that in the beginning when you're running into one problem after another.

Re: I'm tired of fixing customers' AI generated code

#167

Post by underdeserver »

cl3misch wrote: post #159
gumby wrote: post #68

> Helping a customer solve challenges is often super rewarding, but only when I can remove roadblocks for customers who can do most of the work themselves. One thing I loved about doing technical enterprise sales is that I’d meet people doing something I knew little or nothing about and who didn’t really understand what we offered but had a problem they could explain and our offering could help with. They’d have deep…

Your second quote is the same as the first one. Did you copy the same one twice by accident?

I suspect the quote was pasted by mistake the first time.

Re: I'm tired of fixing customers' AI generated code

#169

Post by Piskvorrr »

n4r9 wrote: post #157
Piskvorrr wrote: post #154

Earlier quoted context omitted.

We aren't. LLMs may have been useful for a moment in time, before the trick "it's now MY OWN creation, no IP strings attached - when it comes through the plagiarism machine" became apparent, and before the models started eating their own tail. Now they're just spiralling down, and it will IMNSHO take something else than an iterative "a future version will surely fix this, One Day, have faith."

There are signs of a decline in people asking and answering questions on sites like stack exchange: https://meta.stackexchange.com/questions/387278/has-stack-ex... So I hope you're right, but the evidence is currently that you're wrong. Let's see how it plays out, I suppose.

- Which might be a different matter: of specifically SE declining. (A very different, and long-running, tragedy, but one that began long before the current AI boom and prompted by very different, non-technical issues.)

- That said, surely traffic will decline for Q&A sites. "How do I connect tab A into slot B" is something that people are likely to query LLMs for; the response will surely sound authoritative, and could be even correct. That's definitely a task where LLMs could help: common questions that have been asked many times (and as such, are likely to be well-answered in the human-made training data). A 20001st question of "how do I right-align a paragraph in HTML" has not been posted? Good. Rote tasks are well-suited to automation. (Which, again, brings us back to the issue "how to distinguish the response quality?")

Post reply on HN