Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI didn't delete your database, you did

idiallo.com

Post reply on HN
Print view View original

321–329 of 329 posts

Re: AI didn't delete your database, you did

#321

Post by bilbo0s »

account42 wrote: post #311
bilbo0s wrote: post #90

Earlier quoted context omitted.

Well just to be clear from a legal perspective, in the case of AI, as long as AI is "property", the owners, developers, and/or users will be held liable for things like the hypothetical fatal car accident that Sussman posits. Currently, from a legal perspective, AI is considered a "tool" without legal persona. So you sue the developer, the owner, or the user of the AI. (Just kidding, any lawyer worth his/her salt wil…

This doesn't seem to be how it works in practice. "AI" or not, complex systems are a pretty good shield from accountability in practice today.

PSA. Do not listen to advice like this.

>complex systems are a pretty good shield from accountability in practice today.

Maybe complex legal systems are, but complex software systems offer you no such protection.

My field for the past few decades has been diagnostic medical software. In that field, the 501K you got is kind of entering you into an ironclad agreement with the government. There's almost no way out of it. 501K certs significantly simplify, (for the government), holding you accountable. You have made attestations to suitability directly to the federal government. And the way our chief counsel explained it to us, literally each signature you sent to the government, for each feature that failed, is actually a single count of lying to the federal government.

Please, please, please people, don't listen to comments like the one above. Everything should be run by your qualified legal expert. Getting things right up front is so much easier than trying to fix things when the inevitable happens.

Alternatively, stick to fields free from regulation. That's also a viable strategy. But to just trust that the legal system is complicated and the technology you're deploying is complicated, so the feds will never get me? That's the start of a lot of really bad stories.

Re: AI didn't delete your database, you did

#322

Post by rmunn »

QuantumNomad_ wrote: post #314
rmunn wrote: post #294

Earlier quoted context omitted.

If healthcare is free then you aren't paying your doctors. What you mean is "when healthcare is paid for by other people ", and in that case the cost of the healthcare is still calculable.

In socialist countries we have understood that it is better for the individual that everyone pays a share of what it costs to maintain a functioning healthcare system, and making it available to everyone for free.

In capitalist countries, we do the same thing, except we allow people to choose whether to participate or not. (In theory, at least. The health insurance system in America is currently broken, for many reasons. A deep discussion of those reasons would almost immediately dive into ideological dispute, so I don't intend to do so.)

Actually, I do want to mention one of those reasons, which I hope won't trigger any arguments. (Though if they do, I don't intend to engage). I mention this because I think it's interesting.

A friend of mine is an emergency room doctor in a major US state. He mentioned to me once what he pays in malpractice insurance, and it was more than my annual salary as a programmer at the time (it was around 2010, and I've gotten a few raises since then). A LOT of the cost of healthcare in America is disappearing into the pockets of lawyers, more than most people realize.

Re: AI didn't delete your database, you did

#323

Post by aakresearch »

duxup wrote: post #292
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Yeah I'd like to know in a solid way WHY Claude kept changing a file that I explicitly told it not to. The .mds, Claude's plan all said not to touch that file, and Claude just kept at it. I've had it happen repeatedly lately. Really basic failures. The idea being that as frustrating as it is, if I knew why I might be able to do something about it. But no, we have the black box, where sometimes what comes out just is…

I am by no means an expert, but I'd like to offer my mental model - up to you to decide if it is solid or not, but it works for me.

I think the core intuition is that, like with any other "rasterized" system with finite memory that cannot encode an absence of anything - relation, concept, entity, LLM cannot encode an absence of something through its internal weights. Say, you can have "Product" or "Order" tables in you database, but you cannot have "NotAProduct" or "NotAnOrder" tables - for obvious reasons of such relations being infinite and uncountable. So, to establish an absence of Product or Order your application must execute a "search" operation through the relevant tables. But in LLM-space "search" operation does not exist. It is mathematically undefined. LLM arrives at output (or "what to do") through a sequence of projections of input token vector through its "latent space". It "moves toward" high-probability clusters, fundamentally unable to "move away". So, the success of any "negation" in the prompt ("don't touch this file", "draw me a ballot box without a flag on it") depends on how heavily such scenario represented in the training data/model space. And again, the absence-of-something may be hard-to-impossible to usefully encode, especially if "something" is not fixed. Therefore, to expect "don't touch this file" sentence to result in, well, not touching the file is pure gambling. Sometimes it may look like working, albeit for wrong reasons, and some other times LLM may do exactly the opposite - because its weight matrix statistically pushes it towards "touch this file", completely ignoring (nonexistent in its latent space) "don't".

There is no way to reliably know what will work, and no "skill" or "art" in this. Well, no more than in dice rolling or horoscope casting.

I'd like to add that for the above reason I find "agentic development" usefulness on par with avian remains reading. But when I explored it two practical advises seemed to be helpful in nudging LLM around negation problem:

- Omit the "don't" prompt completely, thus not creating a false "attractor" for LLM; and

- Provide an alternate positive directive ("what to DO", not "what to NOT DO") to act as "escape hatch" when LLM might "want" to touch the sacred file or drop the production DB.

While it looked like somewhat working, I think it is trivially obvious that trying to predict all the nonsense LLM might want to perform and coming up with possible "escape hatches" for everything very quickly becomes utterly impractical.

Re: AI didn't delete your database, you did

#324

Post by rglover »

techaqua wrote: post #298
rglover wrote: post #234

Earlier quoted context omitted.

Give it a name, but something non-human like "thingbot" or "tacosplosion." "Hey tacosplosion, generate me an exploding taco image."

You already failed. We don't say hey to machines.

Alexa has an existential crisis.

Re: AI didn't delete your database, you did

#326

Post by patcon »

pjc50 wrote: post #146
patcon wrote: post #114

Earlier quoted context omitted.

I wish you could have what you want, but I worry you won't get this, because life doesn't give you that, and these systems are tending away from machine precision, and more toward life-like trade-offs. I am almost certain that even if you did get what you want, something that isn't what you want will run circles around you and eat your lunch EDIT: I suspect this will be an unpopular take on Hacker News. And so I am s…

> something that isn't what you want will run circles around you and eat your lunch Yes, exactly. Spoken like a true biologist. It's not really surprising that there's a massive backlash against AI, introducing an unnatural predator into the ecosystem of humans. People don't want to be lunch.

Thanks for engaging the idea! But it feels a bit differently, from my perspective.

The lunch-eaters in my imagining are people working in messy collectives. I work in collective intelligence, and build tools for that, for collective introspection. I'm not talking about some abstract AI maximalism, and am certainly not rooting for that

Re: AI didn't delete your database, you did

#327

Post by jlaternman »

duxup wrote: post #292
paroneayea wrote: post #36

I think the perspective here is completely wrong. The problem is that people are now building our world around tooling that eschews accountability . Over a decade ago now, I had a conversation with Gerald Sussman which had enormous influence on me: https://dustycloud.org/blog/sussman-on-ai/ > At some point Sussman expressed how he thought AI was on the wrong track. He explained that he thought most AI directions were…

Yeah I'd like to know in a solid way WHY Claude kept changing a file that I explicitly told it not to. The .mds, Claude's plan all said not to touch that file, and Claude just kept at it. I've had it happen repeatedly lately. Really basic failures. The idea being that as frustrating as it is, if I knew why I might be able to do something about it. But no, we have the black box, where sometimes what comes out just is…

I think it's simply a context thing, and LLMs can go blind to any part of the instructions at any time, possibly when exploring complex micro tasks that create their own layers of context within them. That's how the pattern feels to me. Parallel to a limit on the number of things a human can hold in its head at the same time. The more complex the thinking involved becomes the bigger the self generated context becomes, too, it doesn't seem like an easily fixed problem to me other than to have an extremely small "mission critical instructions" context that are surfaced in a more impossible-to-ignore way.

Re: AI didn't delete your database, you did

#328

Post by array_key_first »

QuantumNomad_ wrote: post #221
Exoristos wrote: post #132

Earlier quoted context omitted.

McDonald's revenue in 1992 was almost $5,000,000,000.[0] 0. https://www.nytimes.com/1992/04/24/business/mcdonald-s-net-u...

And yet even the $20,000 she initially asked for to cover health expenses was apparently too much according to McD execs.

I think it's about sending a message, and maybe also about ego to a lesser extent. If you pay the 20,000 you're admitting you did something wrong. Of course they did do something wrong. They did something wrong thousands, probably millions of times. And they got very lucky to only be sued once. Which is why they paid millions.

Re: AI didn't delete your database, you did

#329

Post by BadBadJellyBean »

locknitpicker wrote: post #308
BadBadJellyBean wrote: post #156

Earlier quoted context omitted.

> if you install a mobile app from the app store, are you responsible and accountable for every single thing the app does in your system? Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms. Of course everyone is differently qualified to do this but my argument is more about professionals. Managers…

> Yes. I can try to vet the app to the best of your abilities and beyond that it's a tradeoff between how likely is it to cause harm and do the benefits outweigh these harms. No, you don't. If you install malware you are not suddenly held responsible for what has been done to you. Even EULAs you are forced to accept don't shift the responsibility away from bad actors.

I am talking about myself. I have to be careful with what I do. No EULA or any other legal framework protects me from my data be stolen. I have to be careful myself and not just blindly install crapware.
Post reply on HN