Live data from Hacker News

An AI Vibe Coding Horror Story

tobru.ch

71–80 of 224 posts

Re: An AI Vibe Coding Horror Story

#71
post #32

This reads like internet fiction to me. Very vague and short.

fwiw i know tobias and its very very unlikely he made this up. my guess its intentionally vague to not leak any information about the culprit which i guess is fair.

heh, I know that username. I came to the same conclusion. (I hope all is well with you, Yannick)

Re: An AI Vibe Coding Horror Story

#72

What would a responsible on-boarding flow for all of these tools look like? > Welcome to VibeToolX. > By pressing Confirm you accept all responsibility for user data stewardship as regulated in every country where your users reside. Would that be scary enough to nudge some risk analysis on the user's part? I am sure that would drop adoption by a lot, so I don't see it happening voluntarily.

We require someone with a professional engineering designation from an accredited engineering body to sign off and approve before a building can be built. If it is found to have structural issues later, that person can be directly liable and can lose their license to operate. Why this is not the case with health software I cannot explain. Every time I propose this the only argument I recieve against it is people who…

Totally agree - not just medical software either. See replies to my other comment threads. Software engineers really don’t like the idea that they might have to show they can perform at a certain standard to be able to work as a software engineer.

Typically arguments come up:

“that’s gatekeeping” - yes, for good reason!

“Laws already exist” - yeah, and that’s not the same as professional accreditation, standards and codes of practice! Different thing, different purpose. Also the laws are a mishmash and not fit for purpose in most sectors.

Re: An AI Vibe Coding Horror Story

#73
Is there anybody making some framework where you declare the security intentions as code (for each CRUD action) and which agents can correctly do and unit test? I have seen a Lovable competitor's system prompt have 24 lines of "please consider security when generating select statements, please consider security when generating update statements..." since it expects to dump queries here and there.

Re: An AI Vibe Coding Horror Story

#74
post #32

This reads like internet fiction to me. Very vague and short.

fwiw i know tobias and its very very unlikely he made this up. my guess its intentionally vague to not leak any information about the culprit which i guess is fair.

Okay. If it's real I apologize.

But in any case it's so lacking in detail and so brief as to make it so uninteresting that it might as well be fake.

> Somebody "vibecodes" medical app/system. The app was insecure. Personal info leaked.

Okay cool.

Re: An AI Vibe Coding Horror Story

#75
post #3

Do you think if the agency hired a consultant to build this , a consultant couldn’t have made the same mistakes? Lack of security theater is a good thing for most businesses

It is physically possible for a consultant to write bad code. But you'd hope that a consultant could understand that medical data is extremely important to keep secure, and actually write it to have some level of security

Sure, but you'd hope that the LLM could understand that too.

Re: An AI Vibe Coding Horror Story

#76

I think the issue here is less about AI misbehaving and more about people doing things they should not be doing without thinking too hard about the consequences. There are going to be a lot of accidents like this because it's just really easy to do. And some people are inevitably going to do silly things. But it's not that different from people doing stupid things with Visual Basic back in the day. Or responding to f…

Particularly ironic for a doctor to have done this, given all the complaints about patients using Google (even pre-AI)!

Re: An AI Vibe Coding Horror Story

#77
post #32

Earlier quoted context omitted.

fwiw i know tobias and its very very unlikely he made this up. my guess its intentionally vague to not leak any information about the culprit which i guess is fair.

It’s unlikely any LLM tasked with a prompt involving medical records did not automatically address separation of concerns. The type of data involved is worst case scenario. One JS file is also worst case scenario. This is why it may feel manufactured. If it is true, they truly deserve to be put on blast.

I can 100% imagine prompts that would even feel natural that would never hint at any medical background of the data being processed. Could be as simple as using customer instead of patient.

Re: An AI Vibe Coding Horror Story

#78

This reads like internet fiction to me. Very vague and short.

yeah keeping it vague makes sense to protect the place if it's still online but the whole thing doesn't really make sense? The timelines mentioned are weird - he spoke to them before they built it? Or after? It's not that clear, he mentions they mentioned watching a video. > The entire application was a single HTML file with all JavaScript, CSS, and structure written inline. This is not my experience of how agents te…

> The timelines mentioned are weird - he spoke to them before they built it? Or after? It's not that clear, he mentions they mentioned watching a video.

Yeah although I didn't comment I found this weird as well. Chronology was vague and ill-defined. He went to a doctors office and the receptionist mentioned vibe coding their patient records system unprompted?

> A few days later, I started poking around the application.

What!? How... was there even a web-facing component to this system? Did the medical practice grant you access for some reason?

Yeah I'm back to calling bullshit. What a load of crap. Whole post probably written by an LLM.

Re: An AI Vibe Coding Horror Story

#79
post #61
post #47

I know, through personal acquaintance, of at least one boutique accounting firm that is currently vibe-building their own CRM with Lovable. They have no technical staff. I can't begin to comprehend the disasters that are in store.

Generally why build your own CRM? ERP and other resource planning systems I get becouse you can tailor made those to your back office. But for CRM you need mostly reliability.

Because CRMs are very expensive, and they get much more expensive if you need custom development (which you usually need)
Post reply on HN