Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New California bill might block the "AI did it" defense in civil cases

veeto.app

Post reply on HN
Print view View original

61–70 of 75 posts

Re: New California bill might block the "AI did it" defense in civil cases

#61

Post by Muromec »

Retric wrote: post #22
Muromec wrote: post #15

Earlier quoted context omitted.

You can blame it on a car even if the car is not self-driving

It’s your responsibility to ensure that your breaks etc work. Presumably it’s also your responsibility to pick and maintain a working self driving system.

>Presumably it’s also your responsibility to pick and maintain a working self driving system.

I guess it's reasonable to say I should apply updates and get the car serviced and make sure the sensors are not obstructed, yes. Then the difference between "self driving" and "assisting" technology would be a matter of guarantee the manufacturer advertises.

Would it (hypothetically) be reasonable for me to expect that the thing will suddenly break on a highway in normal conditions without anything obstructing the way forward? No, I don't think so. Can I do something to prevent it or foresee it, except not using the self-driving technology at all? Can I choose to have a different self-driving tech installed in my car and retrain the model or control it's behavior in any way at all?

There are different answers to that and I guess the answers will also change over time.

Re: New California bill might block the "AI did it" defense in civil cases

#62

Post by doctorpangloss »

grepfru_it wrote: post #59
doctorpangloss wrote: post #27

Earlier quoted context omitted.

Prepare to be pissed off when you punch this stuff into Google and study this issue.

Google provides me a list of California law firms willing to defend against a pedal confusion liability claim. Maybe I’m missing something?

You're supposed to add "reddit" to Google searches to make them work well.

The punchline is that you can kill somebody in California using a car, claim pedal confusion, and all else being equal, you will face no criminal consequences. You will almost never be financially ruined either. The point of this comparison is that California assembly members write a lot of bills, but they rarely seem to find bandwidth for addressing any of its numerous crises of direct responsibility. It's always some fashionable bullshit.

Re: New California bill might block the "AI did it" defense in civil cases

#63

Post by roughly »

TZubiri wrote: post #57
otterley wrote: post #50

Earlier quoted context omitted.

> It's very rare for the people writing the code to have any say in what it does. That's no defense. Employment is voluntary, and you cannot break the law while doing your job, even if your employer commands you to.

It's also a separate issue. One thing is whether we are responsible for our work,and another is whether employers or employees are responsible. In the second case responsibility doesn't vanish, it is transferred.

If your argument is that employees are not responsible for the results of their work - not legally accountable, but morally responsible - I strongly disagree with this. You are always morally responsible for the results of your actions. Always. There may be mitigating circumstances - you need health care for your child’s chronic medical condition, you were faced with an impossible choice and chose the lesser evil, someone literally held a gun to your head - in which case the full moral picture may absolve you, but you still performed the actions that led to the outcomes in question, and you are still responsible for your actions and their consequences.

Re: New California bill might block the "AI did it" defense in civil cases

#64

Post by IX-103 »

deadbabe wrote: post #5

This bill sucks, a developer should never be held accountable for software they work on or produce.

Professional engineers are held liable for mistakes in their work. If a house falls or a dam breaks the responsibility falls on them to show it wasn't built as designed or there was some other factor involved that a professional engineer would not be expected to have considered in their design.

And, yes, as others have mentioned they get insurance, but there's more to it than that. The level of verification that those designs go through before being released is on a completely different level than what is usually applied to software.

Re: New California bill might block the "AI did it" defense in civil cases

#65

Post by bean-weevil »

gweinberg wrote: post #39

Has "the AI did it" ever been a valid defense? I can see why the legislature might want to make it explicit that it is not, but surely it can't be the case that harm caused by an AI is treated as if it were an act of God.

I agree. If you hire someone, and they aggrieve someone in the course of their duties, your contractor is your agent and you're responsible for what they did. It makes no sense to treat AI any differently.

Re: New California bill might block the "AI did it" defense in civil cases

#66

Post by codr7 »

otterley wrote: post #50
codr7 wrote: post #40

Earlier quoted context omitted.

Hm, once we draw that line a shit ton of people will have to start asking difficult questions about their work. Not saying it's a bad idea, but it has consequences that I suspect you have not considered. It's very rare for the people writing the code to have any say in what it does.

> It's very rare for the people writing the code to have any say in what it does. That's no defense. Employment is voluntary, and you cannot break the law while doing your job, even if your employer commands you to.

So if I write say the ORM code for a system that causes some kind of (completely unknown to me) harm, I'm guilty.

Sorry, that doesn't compute for me.

Re: New California bill might block the "AI did it" defense in civil cases

#67

Post by TZubiri »

roughly wrote: post #63
TZubiri wrote: post #57

Earlier quoted context omitted.

It's also a separate issue. One thing is whether we are responsible for our work,and another is whether employers or employees are responsible. In the second case responsibility doesn't vanish, it is transferred.

If your argument is that employees are not responsible for the results of their work - not legally accountable, but morally responsible - I strongly disagree with this. You are always morally responsible for the results of your actions. Always. There may be mitigating circumstances - you need health care for your child’s chronic medical condition, you were faced with an impossible choice and chose the lesser evil, so…

That is yet a third separate question. Let's recap:

1- Are software developers (whether a company as a whole, or an indepent developer) responsible for the software they develop?

2- When a company develops software, is the responsibility on the company owners, the managers, or the employees or all in shared manner?

3- What is the moral responsibility and what is the legal responsibility in what way are they similar in what way are they different?

At this point I'd forego the hopes of getting an answer and just focus on trying to make clear questions.

Re: New California bill might block the "AI did it" defense in civil cases

#68

Post by otterley »

codr7 wrote: post #66
otterley wrote: post #50

Earlier quoted context omitted.

> It's very rare for the people writing the code to have any say in what it does. That's no defense. Employment is voluntary, and you cannot break the law while doing your job, even if your employer commands you to.

So if I write say the ORM code for a system that causes some kind of (completely unknown to me) harm, I'm guilty. Sorry, that doesn't compute for me.

No, but if your employer asks you to write software specifically designed to defraud a financial institution, yes, you are breaking the law.

Re: New California bill might block the "AI did it" defense in civil cases

#69

Post by roughly »

TZubiri wrote: post #67
roughly wrote: post #63

Earlier quoted context omitted.

If your argument is that employees are not responsible for the results of their work - not legally accountable, but morally responsible - I strongly disagree with this. You are always morally responsible for the results of your actions. Always. There may be mitigating circumstances - you need health care for your child’s chronic medical condition, you were faced with an impossible choice and chose the lesser evil, so…

That is yet a third separate question. Let's recap: 1- Are software developers (whether a company as a whole, or an indepent developer) responsible for the software they develop? 2- When a company develops software, is the responsibility on the company owners, the managers, or the employees or all in shared manner? 3- What is the moral responsibility and what is the legal responsibility in what way are they similar i…

You’re putting in a lot of effort to avoid actually answering any of these questions. I’m sorry if your work leads you to not want those answers.

You bear moral responsibility for the outcomes of your actions, whether you do them on your own time or while acting as the agent of a company. You do not get to “transfer” moral responsibility. Soldiers are responsible for their conduct, employees for theirs.

And, the overlap between legal and moral responsibility is that legal responsibility sometimes aligns with moral responsibility, but that’s irrelevant for the discussion of the moral.

And to cut off the obvious question of “whose morality” - yours. You are accountable to whatever moral system you prefer to live by, but you are always accountable for your actions and their consequences, even when that’s uncomfortable or you’d prefer not to be. We don’t get days off from being human.

Re: New California bill might block the "AI did it" defense in civil cases

#70

Post by kjs3 »

gweinberg wrote: post #39

Has "the AI did it" ever been a valid defense? I can see why the legislature might want to make it explicit that it is not, but surely it can't be the case that harm caused by an AI is treated as if it were an act of God.

While it's not been wrung out in the courts yet, the insurance industry has been denying coverage based on "the AI said so" for a while, coupled tightly with "we can't tell you why because the algorithm is a trade secret". Given the current governance and oversight climate, I don't have much hope that more transparency is in the offing.
Post reply on HN