Live data from Hacker News

India's top court angry after junior judge cites fake AI-generated orders

bbc.com

61–70 of 212 posts

Re: India's top court angry after junior judge cites fake AI-generated orders

#61
post #9

How many of these cases do we have to have before lawyers realise that they need to check that the things an LLM tells them are actually true?

It doesn't matter, because any process that seems right most of the time but occasionally is wrong in subtle, hard to spot ways is basically a machine to lull people into not checking, so stuff will always slip through. It's just like the cars driving themselves but you need to be able to jump in if there is a mistake, humans are not going to react as fast as if they were driving, because they aren't going to be enga…

As someone who has done QA on white collar work it's tiring looking for little errors in work reports. Most people are not cut out for it.

Re: India's top court angry after junior judge cites fake AI-generated orders

#62

> She had no intention to misquote or misrepresent the rulings and that "the mistake occurred solely due to the reliance on an automatic source", the high court wrote I don't think the intention matters here. Its the same deal with every profession using llm to "automate" their work. The onus in on the professional, not the llm. Arstechnica case could have been justified by same manner otherwise. Not knowing the law…

Using an LLM to automate is simply the newer cheaper outsourcing with much of the same entertainment, but less food poisoning and air travel. Over the last 20 years a lot of engineering (proper eng, not software) work in the west has been outsourced to cheaper places, with the certified engineers simply signing off on the work done elsewhere. This results in a cycle of doing things ever faster/more cheaply and safegu…

LLMs also solve the timezone and language challenges. Sadly one problem that remains is that they too tell you they have understood something even if they haven't.

Re: India's top court angry after junior judge cites fake AI-generated orders

#63
post #14

There will be many more things like this and it’s an elephant in the room for the supposed mass replacement of people with AI. Some human still has to be accountable. Someone has to get fired / go to jail when something screws up. You can make humans more productive but for the foreseeable future you can’t take the human out of the loop to have an AI implementation that’s not a disaster/lawsuit waiting to happen. Tha…

Or, AI is going to be like when land lines became unnecessary when cellphones showed up in India. India may get to skip an entire intellectual generation due to the ability of a cheap model to educate (in any language).

The narrative that an entire population are “worth” less, paid less , know less, live less …

Fuck this less shit, embrace the paradigm shift. God is finally providing the remedial support through the miracle of AI.

Re: India's top court angry after junior judge cites fake AI-generated orders

#64
post #53

Earlier quoted context omitted.

This whole thing is silly, LLMs can automate reference validation. If someone is a lawyer, accountant, doctor, teacher, surgeon, engineer etc, and is regurgitating answers that were pumped out with with GPT-5-extra-low or whatever mediocre throttled model they are using, they should just be fired and de-credentialed. Right now this is easy. The real problem is ahead: 99.999% of future content that exists will be made…

> Right now this is easy. Easy? In the US you need house impeachment to fire a judge. In some countries judges are completely immune unless they are sentenced for crimes.

Do you need impeachment to fire a lawyer, accountant, doctor, teacher, surgeon or engineer?

Re: India's top court angry after junior judge cites fake AI-generated orders

#65
post #59

I feel like this points out a very general problem with the law: it generates a lot of boilerplate text. Lawyers don't really read it; they skim it for the relevant bits. Obviously lawyers should not be cheating with AI, especially when they don't even check it. But it does sound to me as if this is an opportunity to re-factor the process. We're carrying forward some ideas originally implemented in Latin, and which c…

law texts feel like a layering problem, like just decoration around decoration to avoid breaking existing 'code' without ever simplifying it

Re: India's top court angry after junior judge cites fake AI-generated orders

#66
post #14

There will be many more things like this and it’s an elephant in the room for the supposed mass replacement of people with AI. Some human still has to be accountable. Someone has to get fired / go to jail when something screws up. You can make humans more productive but for the foreseeable future you can’t take the human out of the loop to have an AI implementation that’s not a disaster/lawsuit waiting to happen. Tha…

We should have more hygiene when it comes to AI.

Text coming out of an LLM should be in a special codeblock of Unicode, so we can see it is generated by AI.

Failing to do so (or tampering with it) should be considered bad hygiene, and should be treated like a doctor who doesn't wash their hands before surgery.

Re: India's top court angry after junior judge cites fake AI-generated orders

#67
post #58

Earlier quoted context omitted.

Using an LLM to automate is simply the newer cheaper outsourcing with much of the same entertainment, but less food poisoning and air travel. Over the last 20 years a lot of engineering (proper eng, not software) work in the west has been outsourced to cheaper places, with the certified engineers simply signing off on the work done elsewhere. This results in a cycle of doing things ever faster/more cheaply and safegu…

At least that's the story LLM labs leaders wanna tell everyone, just happen to be a very good story if you wanna hype your valuation before investment rounds. Working with LLM on a daily basis I would say that's not happening, not as they're trying to sell it. You can get rid of a 5 vendor headcount that execute a manual process that should have been automated 10 years ago, you're not automating the processes involvi…

Aero software people are not highly paid. It's a travesty.

Re: India's top court angry after junior judge cites fake AI-generated orders

#69

Earlier quoted context omitted.

> Right now this is easy. Easy? In the US you need house impeachment to fire a judge. In some countries judges are completely immune unless they are sentenced for crimes.

Do you need impeachment to fire a lawyer, accountant, doctor, teacher, surgeon or engineer?

Nope, and the article is about a judge. What's the point to incentive lawyers to carefully verify their references when they know the judge has no incentive to read them and can just make shit up anyway?

Re: India's top court angry after junior judge cites fake AI-generated orders

#70
post #14

There will be many more things like this and it’s an elephant in the room for the supposed mass replacement of people with AI. Some human still has to be accountable. Someone has to get fired / go to jail when something screws up. You can make humans more productive but for the foreseeable future you can’t take the human out of the loop to have an AI implementation that’s not a disaster/lawsuit waiting to happen. Tha…

Or, AI is going to be like when land lines became unnecessary when cellphones showed up in India. India may get to skip an entire intellectual generation due to the ability of a cheap model to educate (in any language). The narrative that an entire population are “worth” less, paid less , know less, live less … Fuck this less shit, embrace the paradigm shift. God is finally providing the remedial support through the…

We've had YouTube for two decades now. Cheap education was already available for those who wanted it.
Post reply on HN