Legal professional here. This is NOT a replacement for proper legal AI assistants (e.g. Westlaw, in my jurisdiction). As far as I can tell, this is just a wrapper around regular LLMs i.e. nothing that you couldn't achieve yourself with the right prompting. What legal professionals actually pay for, and that is virtually impossible to replicate unfortunately, is to give the AI access to a legal database of case law. W…
Mike: open-source legal AI
111–120 of 123 posts
Re: Mike: open-source legal AI
#112Earlier quoted context omitted.
Related question, then - what do judges use when they have to write opinions in the first place? Do they have to follow the same process and use Thomson Reuters? It's obviously even more important for judges (compared to lawyers) to be able to easily search all of the relevant case law to see which cases are controlling and would have precedence. Seems bizarre to me that this critical function would be gated behind a…
They have staff that do it for them :-)
And those staff mostly use Westlaw :)
Re: Mike: open-source legal AI
#113Legal professional here. This is NOT a replacement for proper legal AI assistants (e.g. Westlaw, in my jurisdiction). As far as I can tell, this is just a wrapper around regular LLMs i.e. nothing that you couldn't achieve yourself with the right prompting. What legal professionals actually pay for, and that is virtually impossible to replicate unfortunately, is to give the AI access to a legal database of case law. W…
In Canada there is a database of a lot of case law here: https://www.canlii.org/ Theoretically speaking if someone scraped all of it and added it to something like this open source Mike project would that then be a much better tool for lawyers?
We have bailii.org but it doesn't have everything.
Re: Mike: open-source legal AI
#114Earlier quoted context omitted.
Yes, it sounds crazy and against the principle of open justice, but unfortunately this is the reality. Certainly in the UK which is my jurisdiction - and I believe in the US too although I don't know for sure. In theory, any member of the public can obtain a judgment by applying for one at the court and paying a fee. That's fine if you just need a one-off judgment, don't mind paying the fee, and you're not in a hurry…
Related question, then - what do judges use when they have to write opinions in the first place? Do they have to follow the same process and use Thomson Reuters? It's obviously even more important for judges (compared to lawyers) to be able to easily search all of the relevant case law to see which cases are controlling and would have precedence. Seems bizarre to me that this critical function would be gated behind a…
The other answer (staff who use Westlaw) is right, but critically this is the point of adversarial litigation. The justice system doesn't hang on that judge finding the precedent; it assumes that the highly motivated lawyers on either side will find the relevant precedent that helps their case and highlight it for the judge.
Re: Mike: open-source legal AI
#115Legal professional here. This is NOT a replacement for proper legal AI assistants (e.g. Westlaw, in my jurisdiction). As far as I can tell, this is just a wrapper around regular LLMs i.e. nothing that you couldn't achieve yourself with the right prompting. What legal professionals actually pay for, and that is virtually impossible to replicate unfortunately, is to give the AI access to a legal database of case law. W…
In Canada there is a database of a lot of case law here: https://www.canlii.org/ Theoretically speaking if someone scraped all of it and added it to something like this open source Mike project would that then be a much better tool for lawyers?
> would that then be a much better tool
Better than before, yes. Good for general legal work that doesn't require robust legal research, yes. Sufficient for full legal research, no.
The problem is that "a lot of case law" isn't enough case law. You need close to everything. Otherwise this can happen: Canlii case X -> Legal principle Y. Westlaw case Z not on Canlii -> X overriden, Y no longer good law. Or you might simply not find a case which cogently supports your argument, when one does in fact exist. Or, conversely, you are unaware of a detrimental case which your opponent knows about because they have Westlaw.
Re: Mike: open-source legal AI
#116Earlier quoted context omitted.
Yes, it sounds crazy and against the principle of open justice, but unfortunately this is the reality. Certainly in the UK which is my jurisdiction - and I believe in the US too although I don't know for sure. In theory, any member of the public can obtain a judgment by applying for one at the court and paying a fee. That's fine if you just need a one-off judgment, don't mind paying the fee, and you're not in a hurry…
This all needs to be publicly accessible for free. Gonna see how blatently inconsistent laws and interpretations are
To be fair, there's sufficient of it publicly available (e.g. on https://www.bailii.org/) that you can easily disprove the conspiracy theory that laws and interpretations are "blatently inconsistent". In fact most judgments are very well thought through and carefully written and cited, because those that aren't tend to be appealed if they are establishing an important principle and getting it wrong. There just isn't enough publicly available to base a full legal AI assistant on.
Re: Mike: open-source legal AI
#117Legal professional here. This is NOT a replacement for proper legal AI assistants (e.g. Westlaw, in my jurisdiction). As far as I can tell, this is just a wrapper around regular LLMs i.e. nothing that you couldn't achieve yourself with the right prompting. What legal professionals actually pay for, and that is virtually impossible to replicate unfortunately, is to give the AI access to a legal database of case law. W…
To be clear, the value of services like Legora and Harvey was not as a replacement for legal-specific services like Westlaw. LLMs are absolutely abysmal at legal research and analysis, and every week we are seeing reports of some lawyer being called out publicly for submitting a brief to a court with hallucinated case citations. This has happened to lawyers at all levels, including lawyers at big law firms. If you ask an LLM questions about very basic principles of law, it will do a decent job, but anything requiring nuance or specialization, they will spit out things that sound plausible, but are not correct.
And this is not likely to change in the future, as the legal market is so small and niche that the leading makers of LLMs have put legal analysis near the bottom of their list of priorities in terms of improving model performance. There is very little if any effort by the major LLM companies to curate sources of additional high-quality legal training data or fine-tune their models to improve performance on legal tasks. Law is also a field with very low tolerance for error, where tiny mistakes can have big consequences, and getting the models to perform well under these constraints would require a lot of investment without a sufficient payoff.
The true reason big firms are buying Harvey and Legora subscriptions is simply to use an LLM for LLM-type tasks, like document review, spotting issues in user-provided documents, and other things that LLMs do well. True, services like Harvey and Legora have lots of cool templates and features for legal work, but you will find that most of the people who use these services in these firms use them much the same way they'd use ChatGPT, Claude, or any other AI chatbot.
The reason law firms can't just use ChatGPT or Claude is that they can't allow confidential or privileged client data (such as documents provided through the prompts) to be stored and hosted on a third party service like ChatGPT or Claude, as these companies may have to turn over client data in response to subpoenas, and depending on the type of LLM account you have, these companies could use your user prompts to train future models thus risking leakage of client data to third parties and potential privilege waiver.
Services like Harvey and Legora solve this problem by accessing the LLMs through APIs, and all client data, prompts/responses, etc., are stored encrypted on Harvey or Legora servers and protected by keys held by the customer. For many law firms, this is 95% of what they're paying for.
The big challenge "Mike" presents to services like Harvey and Legora is that it exposes how little additional value they offer over ChatGPT or Claude, for the vast majority of law firms. A system like "Mike" can provide the same security benefit at basically $0 cost, and can be hosted on the law firm's own internal servers. This is going to put a lot of pricing pressure on services like Harvey and Legora; law firms are notoriously cheap when it comes to IT and software spend and will switch quickly if cheaper alternatives arise. This confirms that Harvey and Legora are going to have to sell their services based on the value they add to lawyer productivity, and not just on being a protected wrapper around GPT or Claude.
Re: Mike: open-source legal AI
#118Presumably this is an issue for the commercial competitors too, but in light of the recent court ruling in United States v. Heppner that AI chatbots can break attorney-client privilege and/or work product doctrine, what kinds of things can this be safely used for? (I would assume you want to avoid sending anything with client-confidential information in it to a service provider like OpenAI or Anthropic.) Potentially…
It’s not different from googling. If a non-lawyer googles legal advice (”how to give yourself an alibi after murdering someone”) it will not be protected by attorney-client privilege. Same if you ask OpenAI.
There is a big difference between using something like Google search and an AI chatbot, in terms of the risk of privilege waiver.
In Google you're generally entering fairly generic and short search queries. The example you provided ("how to give yourself an alibi after murdering someone"), is generic and could apply to anyone or could have been entered for other purposes such as writing a crime novel.
With ChatGPT and Claude, the risk is much higher, because you're basically uploading entire documents with potentially privileged material to a third party, as part of your prompt. To use your analogy, instead of entering a generic "how to give yourself an alibi" query, you'd be providing privileged interview notes and other attorney work product as part of your prompt to the LLM. In the Heppner case (which actually involved a client and not a lawyer), detailed reports and discussions of potential strategies were uploaded.
Re: Mike: open-source legal AI
#119I believe this is the direction enterprise software is generally going. An open-source base with a very permissive license that then each company can adapt (with claude, codex, etc.) for it's own needs. It's either running it on it's own infrastructure or in hosted environment by the author. I've built a similarly extensible codebase for an ERP: https://github.com/lambdadevelopment/lambda-erp
This looks great. The demo is very fast. Is it static generated or is it reading the sql db?
Thanks. The demo itself is prerecorded - so no LLM calls. It's just replaying from the sql db. In an actual request it isn't much slower though in finding what tools to use, etc. You can test it btw. I've allocated an hourly budget - anything that you query after the demo goes to an actual LLM.
Re: Mike: open-source legal AI
#120Earlier quoted context omitted.
> There's a reason why companies like Thomson Reuters have an oligopoly on these types of products, and can get away with charging thousands a year. They are the only ones with access to a comprehensive set of case law, and they've entrenched their position by having exclusive contracts with the law reporting companies. I'm not in the legal field, but can someone explain that further? I would have expected that all c…
Do not know about the US, but some countries publish some type of high profile cases but _only_ after anonimization for obvious privacy reasons. To access the DB through the modern archive (well modern as new rules) you'd have to be an accredited professional passing through a few legal hardles and digital chancellor's office for each copy. It's like going to a bureaucracy^bureaucracy office. Some early companies giv…
> Do not know about the US, but some countries publish some type of high profile cases but _only_ after anonimization for obvious privacy reasons.
It doesn't work that way in the US. Legal judgements are documents of public record, and they are normally published in full - it's not uncommon to search for someone's name and see legal cases pop up that they have been involved in.
There are specific instances where a judge can seal a judicial record (and records for minors are sealed automatically), someone may petition for their own records to be expunged, and parties may ask for some information to be redacted, but these are normally (except in the case of minors) not done automatically. As I understand it, the US has much more lax rules around the publicity of legal proceedings than other jurisdictions. For example, even though someone is deemed "innocent until proven guilty", arrest records and mugshots are reported all the time in the media even though this would be illegal in many other areas of the world.