The customer support issue with selling SaaS is not just about AI, I think. It's widespread across the industry. There are customers who have almost no idea about how their software works, and expect you to guide them through their own specific setups. One issue we regularly have is people complain that our TLS certificate keeps changing and that they want us to notify them in advance of any change. We try to tell them that two/three-month duration TLS certificates is a best practice and that they should ensure that their trust store knows about standard root certificates. Then it turns out their software only supports one root cert at a time and they want us to jump into a call and guide them through changing it in their software.
I'm tired of fixing customers' AI generated code
321–330 of 345 posts
Re: I'm tired of fixing customers' AI generated code
#322Someone mentioned "hallucination-based API design" on twitter (I cannot find it now). It's designing API by LLM hallucination. If there is a common hallucination API call, just add that API. This will make the API more "friendly" and resemble common similar APIs. Considering that LLM can hallucinate in different ways unpredictably, not sure whether it will work in practice.
Internet explorer tried that 20 years ago. Allowing all kinds of errors in the html to just slip through and attempt to do something good out of it, forcing everyone else to be bug-for-bug compatible.
Also commonly referred to as the robustness principle; “be conservative in what you do, be liberal in what you accept from others“. An approach that is now considered an anti-pattern, for such future compatibility reasons.
Re: I'm tired of fixing customers' AI generated code
#323I had a related episode at work when my coworker asked me why his seemingly trivial 10 line piece of code was misbehaving inexplicably. It turned out he had two variables `file_name` and `filename` and used one in place of another. I asked him how he ended up with such code, he said he used copilot to create it. Using code from a generative AI without understanding what it does is never a good idea.
We hired a new guy at work. In one of his first tasks he had chosen to write some bash, and it was pure nonsense. I mean it contained things like: if [ -z "${Var}+x" ] Where I can see what the author was trying to do, but the code is just wrong. I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, b…
https://stackoverflow.com/a/13864829
It is not nonsense. You use that expression if you want to check if a variable exists or not (as opposed to being set to an empty string) which is an extremely common problem.
Re: I'm tired of fixing customers' AI generated code
#324Earlier quoted context omitted.
And then the customers will open support requests for code generated by an AI that misuse that very SDK. It doesn't look like OP's issue is with the code per say, only with the lack of skills of its customers, regardless of the code they write...
I don't really know how to point this out without sounding rude and obnoxious, which is not my intention, but it's "per se" (Latin for by itself), not "per say".
No rudeness taken, don't worry! TIL, I didn't know! Thanks!
Re: I'm tired of fixing customers' AI generated code
#325Earlier quoted context omitted.
And this is why people continue to do it. And at that, with such a discrepancy, why not? Proof of concept, you know if it adds value and makes money, enough money to pay $20k.
I think the missing proof point is whether companies would fork out the extra $19,700 once they understand the actual cost.
99% of them were never in the bespoke sofware market to begin with. If the $300 did not work put, they'll cobble something together in Excel pr use some SaaS that comes close enough.
Re: I'm tired of fixing customers' AI generated code
#326Earlier quoted context omitted.
We hired a new guy at work. In one of his first tasks he had chosen to write some bash, and it was pure nonsense. I mean it contained things like: if [ -z "${Var}+x" ] Where I can see what the author was trying to do, but the code is just wrong. I dont mind people not knowing stuff, especially when it's essentially Bash trivia. But what broke my heart was when I pointed out the problem, linked to the documentation, b…
https://stackoverflow.com/a/13864829 It is not nonsense. You use that expression if you want to check if a variable exists or not (as opposed to being set to an empty string) which is an extremely common problem.
The +x needs to belong inside the {}, ie. "${var+x}" not "${var}+x"
But I agree I wouldn't extrapolate too much from one such brain fart
Re: I'm tired of fixing customers' AI generated code
#327Earlier quoted context omitted.
fair point again. i dont think this is the way to go in either case. of your api is easy to clearly document its good. if you struggle with that, a programmer trying to call it will also. regardless of what an LLM makes of it. it seems like trying to facilitate bad things. make good things instead.
> of your api is easy to clearly document its good If it breaks a lot of conventions or norms, no it's probably not good. But also the point here is *how much do you need to really explain?". If you have apis that require a lot of re-explanation to llms to force them to do the right thing "no getUsers doesn't exist use users(filter) where... YOU MUST USE users(filter)!" That's a sign your API might be going counter t…
make good things, shouldn't be based on a sole person's opinion i suppose. I didn't want to elaborate on what that is, but as you pointed out, use standards and conventions, well documented and tested libraries. Don't stray from the common thing if it's not needed. Reason about data first, then code. - these are all common advices too, easy to find if you look on any search engine how to build something 'good' or 'properly'. or 'how to program professionally'. - i'd hope someone uses the resources and knowledge available to them that others have spent a lot of time documenting and building.
That being said, its not always the case and you are right, 'make something good' is a bad advice, essentially it's not advice at all.
I would say as advice - don't do something alone if you want others to use it. Try to get feedback as early as possible from people you trust, and from people you'd expect to be your consumers/users. And always listen to that advice, don't throw it into the wind. I think that's the best advice if you want to make something 'successful', because the label of successful ultimately will be granted to your product by your users, not yourself.
Sorry to put a 'bad advice' if it was taken as advice :-). I don't mean to make bad things even in advice ^^.
Re: I'm tired of fixing customers' AI generated code
#328Earlier quoted context omitted.
programs call apis.
This is the silliest argument I write API calls. I am a user of apis. If you want to be more pedantic than this I simply do not care and you should evaluate what benefit you are bringing to this conversation.
Silliness can also be beneficial :D. You are a user who 'writes code', and that code 'calls'. or are you using your vocal chords to magically retreive data from an API -> it could happen if you use voice-to-text and have an LLM make your code :'D.
- sorry if the comment was too silly. i'll try to keep it for myself.
Re: I'm tired of fixing customers' AI generated code
#329Earlier quoted context omitted.
Until I decided to start "reinventing the wheel" and just not using abstractions from popular libraries and frameworks I really struggled to actually understand what is happening. I feel like a week isn't anywhere near close enough but depending on what you want to do it gets you to start tinkering. Ironically I do wish that I had started working on embedded with microcontrollers than starting with web purely because…
Ironically I learned programming by playing with microcontrollers, which I got into through learning about electronics. So I had a really true "ground up" learning experience, starting with embedded C (not machine code, I wasn't that hard). I did a number of projects on AVR's and got decent at writing programs. When moved on to writing PC programs, I struggled so much because everything is so heavily abstracted and l…
Reading your experience maybe somewhere in the middle is in fact good...
Re: I'm tired of fixing customers' AI generated code
#330> The worst is when a request starts out simple — I help them fix one hallucination — but then that customer wants to build more complex logic, and somehow I’ve set the expectation that I will provide unlimited free support forever. I’ve gotten a number of angry messages from customers who essentially want me to build their whole app for free.
Annoying, but a good way to weed out bad customers. Life’s too short to deal with people who feel this entitled.