[deleted]
India's top court angry after junior judge cites fake AI-generated orders
121–130 of 212 posts
Re: India's top court angry after junior judge cites fake AI-generated orders
#122Earlier quoted context omitted.
I think this is an even clearer case than usual. With software engineers and office work you don’t have legal limitations on who can perform the work, but they exist for lawyers and doctors for example. So if this is a tool, the fault lies fully in the user, and if this is treated as “another persons work” then the user knowingly passed the work onto someone not authorized to do it. Both end up in the user being guil…
> With software engineers and office work you don’t have legal limitations on who can perform the work Technically true, but if you want the IP to be covered by copyright you better make sure they're not using AI or you'll find out that there are some serious legal limitations in your future when you aim to either pick up investment or sell your IP.
Correct. The US has already ruled that there are no IP protections for AI generated content
Re: India's top court angry after junior judge cites fake AI-generated orders
#123Earlier quoted context omitted.
It doesn't matter, because any process that seems right most of the time but occasionally is wrong in subtle, hard to spot ways is basically a machine to lull people into not checking, so stuff will always slip through. It's just like the cars driving themselves but you need to be able to jump in if there is a mistake, humans are not going to react as fast as if they were driving, because they aren't going to be enga…
> won't solve the core problem. what's the core problem tho? Because if the core problem is "using ai", then it's an inevitable outcome - ai will be used, and there are always incentive to cut costs maximally. So realistically, the solution is to punish mistakes. We do this for bridges that collapse, for driver mistakes on roads, etc. The "easy" fix is to make punishment harsher for mistakes - whether it's LLM or not…
The core problem is that the tool provides output that looks right and is right a lot of the time, but also slips in incorrect stuff in a hard to notice way.
Punishment isn't a problem because it doesn't work. If you create a system that lulls people into a sense of security, no punishment will stop them because they aren't doing it thinking "it's worth the risk", it's that they don't see the risk. There are so many examples of this, it's weird people still think this actually works.
Furthermore, it becomes a liability-washing tool: companies will tell employees they have to take the time to check things, but then not give them the time required to actually check everything, and then blame employees when they do the only thing they can: let stuff slip.
If you want to use LLMs for this kind of thing, you need to create systems around them that make it hard to make the mistakes. As an example (obviously not a complete solution, just one part): if they cite a source, there should be a mandated automatic check that goes to that source, validates it exists, and that the cited text is actually there, not using LLMs. Exact solutions will vary based on the specific use case.
An example from outside LLMs: we told users they should check the URL bar as a solution to phishing. In theory a user could always make sure they were on the right page and stop attacks. In practice people were always going to slip up. The correct solution was automated tooling that validates the URL (e.g: password managers, passkeys).
Re: India's top court angry after junior judge cites fake AI-generated orders
#124Earlier quoted context omitted.
They cannot even claim they weren't aware of the danger. LLM hallucinations have been a discussed topic, not some obscure failure mode. Almost every article on problems with AI mentions this. So the judge was lazy, incompetent, or both.
I do think that for this particular situation we need to step outside of our tech bubble a little bit. I am still having regular conversations with people that either don't know about hallucinations or think they are not a big problem. There is a ton of money in these companies pushing that their tools are reliable and its working for the average user. I mean there are people that legitimately think these tools are c…
I find it hard to believe the people who use AI haven't read a single article about AI. That would also disqualify this judge, if it were true.
This exceeds the tech bubble.
My local newspaper, completely clueless about tech, runs an article about AI trouble, hallucinations and whatnot every other week. Completely missing most of the nuances, of course, but my point is that this has entered the public discourse.
Re: India's top court angry after junior judge cites fake AI-generated orders
#125It's sad that it took the highest court in the country to point out lack of professionalism and misconduct.
The judge took no personal responsibility.
> She told the court that this was her first time using an AI tool and she had believed the citations to be "genuine". She had no intention to misquote or misrepresent the rulings and that "the mistake occurred solely due to the reliance on an automatic source", the high court wrote.
She had one job. And that was to read the citations. Instead of owning up to the mistake of being lazy all she wanted to talk about "intentions".
The high court also took no responsibility.
> In its order, the high court said that "the citations may be non-existent, but if the learned trial court has considered the correct principles of law and its application to the facts of the case is also correct,
This line of reasoning is questionable and attempt to gaslight everyone. Judges cite other cases in their judgement. But if the junior judge had no clue that the references were fake what correct principles was she applying?
End of the day maybe the judgement is correct but this overall bullshit.
Given that this is happening all over the world people seem to have a convenient excuse - The AI made me do it.
Re: India's top court angry after junior judge cites fake AI-generated orders
#126I feel like this points out a very general problem with the law: it generates a lot of boilerplate text. Lawyers don't really read it; they skim it for the relevant bits. Obviously lawyers should not be cheating with AI, especially when they don't even check it. But it does sound to me as if this is an opportunity to re-factor the process. We're carrying forward some ideas originally implemented in Latin, and which c…
> problem with the law: it generates a lot of boilerplate text I think the problem fundamentally is that matters of law require thorough, precise language, and unambiguous context. If you remove "the boilerplate" then you introduce a vast gray area left to interpretation. Usually attempts (by humans or computers) to "summarize" or frame things in "plain language" will apply a bias since it intentionally omits all the…
This is only the case when you care more about the letter of the law than the spirit of the law, which is, I suppose, most of the world. It doesn't have to be this way, it's a choice that society has made.
When I bought my house, in an alternate universe the paperwork could have been one sheet of paper that said "[My name] purchases home at [address] from [Seller's name] for [price]." and we'd all rely on our shared understanding of what it means to buy something and shared cultural expectations around home ownership and commerce. But our society did not make that choice, we don't live in that universe, so I had to sign a 300 page stack of papers 30 times.
Re: India's top court angry after junior judge cites fake AI-generated orders
#127Earlier quoted context omitted.
> won't solve the core problem. what's the core problem tho? Because if the core problem is "using ai", then it's an inevitable outcome - ai will be used, and there are always incentive to cut costs maximally. So realistically, the solution is to punish mistakes. We do this for bridges that collapse, for driver mistakes on roads, etc. The "easy" fix is to make punishment harsher for mistakes - whether it's LLM or not…
The human is responsible. That's the fix. I don't care if you got the results from an LLM or from reading cracks in the sidewalk; you are responsible for what you say, and especially for what you say professionally. I mean, that's almost the definition of a professional. And if you can't play by those rules, then maybe you aren't a professional, even if you happened to sneak your way into a job where professionalism…
This doesn't solve the problem, because companies will force people to use these tools and demand they work faster, eventually resulting in people slipping.
People will have to choose between being fired for being "too slow", or taking the risk they end up liable. Most people can't afford to just lose their job, and will end up being pressured into taking the risk, then the companies will liability-wash by giving them the responsibility.
You need regulation that ensures companies can't just push the risk onto employees who can be rotated out to take the blame for mistakes.
Re: India's top court angry after junior judge cites fake AI-generated orders
#128Earlier quoted context omitted.
> With software engineers and office work you don’t have legal limitations on who can perform the work Technically true, but if you want the IP to be covered by copyright you better make sure they're not using AI or you'll find out that there are some serious legal limitations in your future when you aim to either pick up investment or sell your IP.
Correct. The US has already ruled that there are no IP protections for AI generated content
An unqualified statement, the user has copyright over the elements they provide. In an image if they make manual edits for example, those are protected. In a modern agentic codebase the code itself is least valuable, what counts more are the specs and tests.
Re: India's top court angry after junior judge cites fake AI-generated orders
#129There will be many more things like this and it’s an elephant in the room for the supposed mass replacement of people with AI. Some human still has to be accountable. Someone has to get fired / go to jail when something screws up. You can make humans more productive but for the foreseeable future you can’t take the human out of the loop to have an AI implementation that’s not a disaster/lawsuit waiting to happen. Tha…
Or, AI is going to be like when land lines became unnecessary when cellphones showed up in India. India may get to skip an entire intellectual generation due to the ability of a cheap model to educate (in any language). The narrative that an entire population are “worth” less, paid less , know less, live less … Fuck this less shit, embrace the paradigm shift. God is finally providing the remedial support through the…
Or you are proven wrong entirely, again and again.
And it turns out, that the legacy, unimportant and derelict of the past - culture is all decisive It turns out that only some cultures can generate high trust societies, capable to form institutions. And you prolonged suffering, by declaring that all cultures are created equal. History may write you down a monster.
Re: India's top court angry after junior judge cites fake AI-generated orders
#130There will be many more things like this and it’s an elephant in the room for the supposed mass replacement of people with AI. Some human still has to be accountable. Someone has to get fired / go to jail when something screws up. You can make humans more productive but for the foreseeable future you can’t take the human out of the loop to have an AI implementation that’s not a disaster/lawsuit waiting to happen. Tha…
>why so many companies are now saying they see zero ROI from AI efforts. I strongly suspect this is because workers are pocketing the gains for themselves. Report XYZ usually takes a week to write. It now takes a day. The other 4 days are spent looking busy. The MIT report that found all these companies were getting nowhere with AI, also found that almost every worker was using AI almost daily. But using their person…
Quiet the contrary, companies layoff all roles (frontend, backend, qa, devops, even ui/ux) and handle a project to one competent dev. And asks them to deliver it in 1/3rd the time it would have took with a proper team. It's happening at places I know. This thread on reddit is 100% same: https://www.reddit.com/r/developersIndia/s/EIksvB15tm
I can't even imagine the stress from context switching, and since people don't realize this is still work, they do this late into the night as well.