Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Answering Legal Questions with LLMs

hugodutka.com

Post reply on HN
Print view View original

141–150 of 160 posts

Re: Answering Legal Questions with LLMs

#141

Post by _akhe »

cogman10 wrote: post #116
_akhe wrote: post #25

I saw a RAG demo from a startup that allows you to upload patient's medical docs, then the doctor can ask it questions like: > what's the patient's bp? even questions about drugs, histories, interactions, etc. The AI keeps in mind the patient's age and condition in its responses, when recommending things, etc. It reminded me of a time I was at the ER for a rib injury and could see my doctor Wikipedia'ing stuff - coul…

> I used to think AI would replace doctors before nurses, and lawyers before court clerks - now I think it's the other way around. Nurses don't read numbers from charts. Part of their duties might be grabbing a doc when numbers are bad but a lot of the work of nursing is physical. Administering drugs, running tests, setting up and maintaining equipment for measurements. Suggesting a nurse would be replaced by AI is a…

True, and there are CNAs, LVNs, and RNs, which all have different responsibilities - to your point both the CNA and RN seem safe for now, it's really the patient intake and information piece.

Some mechanic positions will be replaced by AI - probably similar to medical where those operating machinery and those making important judgments are fine for now, but asking about parts/comparisons, giving/getting info about my car, etc. will be an LLM - maybe even self-serve with a friendly UI. I can see a lot of front-of-house - everything from fast food to oil changes, being just AI.

Automotive engineers at automakers will also use LLMs though, but more like software developers, probably text-to-CAD type generation to automate work or come up with ideas, so in this analogy the modern-day drafter is replaced by AI.

Re: Answering Legal Questions with LLMs

#142

Post by _akhe »

barrenko wrote: post #72
_akhe wrote: post #25

I saw a RAG demo from a startup that allows you to upload patient's medical docs, then the doctor can ask it questions like: > what's the patient's bp? even questions about drugs, histories, interactions, etc. The AI keeps in mind the patient's age and condition in its responses, when recommending things, etc. It reminded me of a time I was at the ER for a rib injury and could see my doctor Wikipedia'ing stuff - coul…

We have a kind of popular legal forum in my country and I'm convinced if I managed to scrape it properly and format QA pairs for fine-tuning it would make a kick-ass legal assistant (paralegal?). Supply it with some actual laws and codification via RAG and voila. Just need to figure out how to take no liability.

Make it a joke brand like "Johnnie Cochran" so you can't be taken seriously but lowkey it's very good

Re: Answering Legal Questions with LLMs

#143

Post by hugodutka »

balphi wrote: post #70

How are you using regex to end the while loop? Are you detecting a specific substring or is it something more complex?

It detects if a message contains the ”Final Answer” substring preceded by a specific emoji. The emoji is there to make the substring relatively unique.

Re: Answering Legal Questions with LLMs

#144

Post by freejazz »

_akhe wrote: post #138
freejazz wrote: post #137

Earlier quoted context omitted.

>Could the same LLM technology be used except on a body of specific case documents to surface information through a convenient language interface to a legal expert? Yes. It's as safe as SQL. I see no reason at all to believe this at all.

RAG is the indexing and querying of info inside documents. It puts it in a vector database, for example, pgvector - an extension of SQL to allow you to store data in numerical form - then you can query it using natural language (via the LLM). There's a possibility for errors in regular SQL querying too, like a user-facing search input. I'm not saying language interfaces are foolproof, but it's not generally wrong whe…

There's a reason attorneys don't put the facts from cases into SQL databases to query, I think you are missing the point completely.

Re: Answering Legal Questions with LLMs

#145

Post by _akhe »

freejazz wrote: post #144
_akhe wrote: post #138

Earlier quoted context omitted.

RAG is the indexing and querying of info inside documents. It puts it in a vector database, for example, pgvector - an extension of SQL to allow you to store data in numerical form - then you can query it using natural language (via the LLM). There's a possibility for errors in regular SQL querying too, like a user-facing search input. I'm not saying language interfaces are foolproof, but it's not generally wrong whe…

There's a reason attorneys don't put the facts from cases into SQL databases to query, I think you are missing the point completely.

Not true. How would people look up cases online if that was the case?

I built Checkr's background check ETA in Ruby/React, and had to get background check certified to work there. Part of onboarding was going down to the courthouse to show us how it was done before APIs. While it's true some records are still offline in some courthouses, almost all of it is online, some is even sold to 3rd parties in some states like mugshot websites, background check sites, etc. While others are on-prem servers the state/county runs. But they definitely use databases and computers lol.

I think you're missing the point - you act like I'm suggesting AI replace the entire legal system when I'm talking about a tool people would use instead of older tech like a SQL database and UI.

For courthouses that run their SQL on-prem for security reasons, could do the same with models - they don't even need access to the internet. So if you wanted to be inaccessible to the public you could (though some states/counties require they make it public).

Nothing will satisify the neo-luddite take, just watch from the sidelines I guess!

Re: Answering Legal Questions with LLMs

#146

Post by freejazz »

_akhe wrote: post #145
freejazz wrote: post #144

Earlier quoted context omitted.

There's a reason attorneys don't put the facts from cases into SQL databases to query, I think you are missing the point completely.

Not true. How would people look up cases online if that was the case? I built Checkr's background check ETA in Ruby/React, and had to get background check certified to work there. Part of onboarding was going down to the courthouse to show us how it was done before APIs. While it's true some records are still offline in some courthouses, almost all of it is online, some is even sold to 3rd parties in some states like…

>Not true. How would people look up cases online if that was the case?

Have you ever used LexisNexis or WestLaw? It's not an SQL database of facts from a case. It's literally just string searching. Do you have any experience with the legal industry at all as you repeatedly make statements about what lawyers would/should/could do?

>While others are on-prem servers the state/county runs. But they definitely use databases and computers lol.

The assertion wasn't that lawyers don't use technology, the assertion was that lawyers do not abstract the facts from a legal case into a database for querying. That you suddenly do not distinguish that from the general use of databases at all is asinine and not conducive to conversation because it's such a ridiculous stretch of what anyone could have meant, let alone what was actually written.

>I think you're missing the point - you act like I'm suggesting AI replace the entire legal system when I'm talking about a tool people would use instead of older tech like a SQL database and UI.

I'm not suggesting that at all. I'm suggesting that the limited utility you think is there, isn't.

>Nothing will satisify the neo-luddite take, just watch from the sidelines I guess!

Rude and unnecessary.

Re: Answering Legal Questions with LLMs

#147

Post by cess11 »

_akhe wrote: post #139
cess11 wrote: post #136

Earlier quoted context omitted.

Not sure why you keep bringing that up instead of answering my question. Lawyers and doctors get it wrong all the time.

> Lawyer and doctor are often listed as comparable examples because both involve sensitive info Doesn't it answer it? > Lawyers and doctors get it wrong all the time This is a tool that helps them get it right

No, it does not.

Why do you think technical documentation and medical diagnosis is similar to what judges in courts are doing?

OK, so why do lawyers and doctors get it wrong all the time then, if it does?

Re: Answering Legal Questions with LLMs

#148

Post by _akhe »

freejazz wrote: post #146
_akhe wrote: post #145

Earlier quoted context omitted.

Not true. How would people look up cases online if that was the case? I built Checkr's background check ETA in Ruby/React, and had to get background check certified to work there. Part of onboarding was going down to the courthouse to show us how it was done before APIs. While it's true some records are still offline in some courthouses, almost all of it is online, some is even sold to 3rd parties in some states like…

>Not true. How would people look up cases online if that was the case? Have you ever used LexisNexis or WestLaw? It's not an SQL database of facts from a case. It's literally just string searching. Do you have any experience with the legal industry at all as you repeatedly make statements about what lawyers would/should/could do? >While others are on-prem servers the state/county runs. But they definitely use databas…

> no reason at all to believe this at all

> Do you have any experience with the legal industry at all

> Rude

Your repeated use of "at all" also comes across as slightly rude FYI :)

As stated, yes I built background check software for a major background check company (they're yc, now worth billions) - in particular I developed their background check ETA and built their React app which is used millions of times per year by Uber, DoorDash, and others, for background checks. I'm familiar with the space and had to become a background investigator to work there. What you say just isn't true.

> they do not abstract facts from a legal case into a database for querying

Again wrong - yes they do. How would courts operate if they didn't, think about it for 2 seconds.

Post reply on HN