You guys realise this is AI slop on AI slop, right?
This is reality now, what do you want?
An AI Vibe Coding Horror Story
151–160 of 224 posts
Re: An AI Vibe Coding Horror Story
#152Earlier quoted context omitted.
> On my last visit i actually casually discussed their IT system with a doctor. Oh right, cool. Did it have a public-facing web-portal that you were able to "investigate" and that "Thirty minutes in, I had full read and write access to all patient data". The level of credulity in these comments is immense.
No, they were complaining about using expensive, overly complicated third-party system that they need like only basic features like keeping text records about visits, and prescriptions and sending invoices to health insurers. And in some practices you get direct access to your data as a patient. I mean the story might be fake obviously, but is definitely plausible.
Yeah sure, as a matter of rule, every time I visit any health provider I am always discussing with the medical receptionist: the software they use, the challenges the business as a whole faces, the tensions between insurers and third parties.
Things that absolutely 100% happen everytime I - a tech guy - experiences when I go to the doctor/phyiso-therapist etc... etc... These are discussions that are happening.
Re: An AI Vibe Coding Horror Story
#153Earlier quoted context omitted.
We don’t blame companies selling 3D Design software or 3D printers or mortar and cement, or graph paper and pencils. When people abuse those tools and build huts or houses or bridges that fall down, we usually blame the user for not having appropriate professional qualifications, accreditation, and experience. (Very occasionally we blame bugs in simulation software tools). AI is a tool. It’s not intelligent, and it w…
I largely agree, but if a company sold cement explicitly claiming that they will replace every job in the entire construction industry, that the cement is able to plan, verify, and build on its own, without supervision, and that any layperson can now create PhD level bridges with that cement without any input from or verification by professionals, some liability would definitely fall on the company selling that cemen…
> We should blame them for misleading marketing.
Re: An AI Vibe Coding Horror Story
#154This reads like internet fiction to me. Very vague and short.
Given the subject matter, it would be highly unethical to reveal the name of the company before verifying it was indeed fixed. I'd be wary of getting sued.
The first time I stumbled onto a big security vulnerability (exposed stripe/aws/play store keys. I was poking around an API a web app was using, and instead of hitting /api/v1, if you just hit /api it served them. I wasn't trying to do anything malicious), the very first thing I did was contacted a security researcher friend to ask about covering my ass while performing responsible disclosure.
You hear too much about people being persecuted for trying to point out security vulnerabilities. (Guess they haven't heard about "don't shoot the messenger").
(It turned out fine after finally managing to speak with someone. Had to ring up customer service and say "look, here are the last digits of your stripe private key. Please speak with an engineer". Figuring out how to talk with someone was the difficult thing)
Re: An AI Vibe Coding Horror Story
#155I saw something very similar a few months ago. It was a web app vibe coded by a surgeon. It worked, but they did not have an index .html file in the root web directory and they would routinely zip up all of the source code which contained all the database connection strings, API credentials, AWS credentials, etc.) and place the backup in the root web directory. They would also dump the database to that folder (for ba…
interesting, so the ai got the hard stuff right. password hashing, schema design, fine. it fumbled the stuff that isn't really "coding" knowledge, feels more like an operational intuition? backup folder sitting in web root isn't a security question, it's a "have you ever been burned before" question, and surgeon hadn't. so they didn't ask and the model didn't cover it, imo that's the actual pattern. the model secures…
Maybe this is what's missing in the prompt? We've learned years ago to tell the AI they're the expert principal 100x software developer ninja, but maybe we should also honestly disclose our own level of expertise in the task.
A simple "I'm a professional surgeon, but sadly know nothing about making software" would definitely make the conversation play out differently. How? Needs to be seen. But in an idealized scenario (which could easily become real if models are trained for it), the model would coach the (self-stated) non-expert users on the topics it would ordinarily assume the (implicitly self-stated) expert already knows.
Re: An AI Vibe Coding Horror Story
#156Earlier quoted context omitted.
A similar thing happened to me back in the day when Wi-Fi was still new. I joined an open network and it turned out to be a law firm. All their computers were on a Samba network with full C: drives shared. I wrote README.txt files on their drives telling them about the issue, but after some time it was still the same. Then I went directly to the place to talk to them and also with the idea I could land my first job f…
At one point I worked as a customer support agent outsourced to Apple via the company. Apple forced us to us some very outdated browser UIs, basically for filling in forms, across maybe 4-5 different services in some cases. The machines we were given by this outsourcing company of course where Apple computers, fairly locked down. But one thing they hadn't locked doll wn, was installing extensions in Safari, and given…
Hope you are doing better now
Re: An AI Vibe Coding Horror Story
#157Software engineering is looking more and more like it needs a professional body in each country, and accreditation and standards. Ie it needs to grow up and become like every other strand of engineering. Gone should be the days of “I taught myself so now I can [design software in a professional setting / design a bridge in a professional setting].” I’m not advocating gatekeeping - if you want to build a small bridge…
>> Software engineering is looking more and more like it needs a professional body in each country, and accreditation and standards.
Doesn't help much, accounting needs accreditation and standards, but that doesn't prevent competition level of some 100 accountants per job. Only way you prevent that is by limiting numbers, like lawyers do, case when connections and nepotism matter, you basically get a hereditary aristocratic caste.
I guess we better get used to going back being peasants working shit jobs barely above starvation since that's what the future of capitalism seems to bring: https://realityraiders.com/fringewalker/irreverent-humor/mon...
Re: An AI Vibe Coding Horror Story
#158Software engineering is looking more and more like it needs a professional body in each country, and accreditation and standards. Ie it needs to grow up and become like every other strand of engineering. Gone should be the days of “I taught myself so now I can [design software in a professional setting / design a bridge in a professional setting].” I’m not advocating gatekeeping - if you want to build a small bridge…
I think the problem is that the person described had no idea what they were doing even in their own professional capacity. They needed to know about patient data management, but they didn't.
The way I see it, if they didn't even realize that they are doing something they shouldn't, they wouldn't have even known they need accreditation, even if that was required. Unless we restricted access to gazillions of tools without it of course.
I think it'll work itself out over time as what AI is/isn't and what data privacy means is discussed more. I'd leave accreditation entirely out of it, because we cannot even agree on what are the actual best practices or if they matter.
Re: An AI Vibe Coding Horror Story
#159Earlier quoted context omitted.
There’s a reason why many professions have professional bodies and consolidated standards - from medicine to accountancy, actuarial work, civil engineering, aerospace, electronic and electrical engineering, law, surveying, and so many more. In most of those professions, it is a crime or a civil violation to offer services without the proper qualifications, experience and accreditation from one of the appropriate prof…
> There’s a reason why many professions have professional bodies and consolidated standards - from medicine to accountancy, actuarial work, civil engineering, aerospace, electronic and electrical engineering, law, surveying, and so many more. Professional bodies = gatekeeping. The existence of the body means that the thing its surrounding will be barred from others to enter. It means financial barriers & "X years of…
Professional bodies are a necessary form of gatekeeping for practicing the craft of software engineering professionally.
You are then bringing a whole host of other issues that are related in nature but not in practice: * Locking down of Android ecosystem * Openness of education * Remote teaching * Remote or online examination etc.
Professional bodies don't wall off the ability to learn nor to tinker at home, nor even to prototype or experiment (depending on scale and industry).
You can't confuse all these issues into one thing and say "we don't want this". It's a disingenuous way to argue the matter.
Re: An AI Vibe Coding Horror Story
#160What would a responsible on-boarding flow for all of these tools look like? > Welcome to VibeToolX. > By pressing Confirm you accept all responsibility for user data stewardship as regulated in every country where your users reside. Would that be scary enough to nudge some risk analysis on the user's part? I am sure that would drop adoption by a lot, so I don't see it happening voluntarily.
Would it? Feels a bit like when you use Facebook and handover all your data.
Yeah, fair. I am just thinking out loud here. What is a decent solution to this problem? Is there one?