Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Law Firm Rankings Are Useless

blog.litimetrics.com

Post reply on HN
Print view View original

11–20 of 64 posts

Re: Why Law Firm Rankings Are Useless

#11

Post by wikiwawa »

libertymcateer wrote: post #4

> By applying machine learning algorithms to data mined from court judgments... And, with that sentence, this post becomes bullshit. The overwhelming majority of disputes do not reach litigation. The overwhelming majority of litigation does not reach judgment. And then a gigantic proportion of those judgments are appealed, and a non-trivial percentage are overturned. Even at that, only on the order of a single digit-…

A large part of statistics is devoted to making valid inference from noisy samples, largely by making sensible assumptions. Even in your example, it is common practice to assess a surgeon on outcomes of surgery, even though a lot of a surgeon's work does not lead to surgery.

We aren't claiming to assess the wide practice of law, just litigation. We do Litigation Analytics.

Re: Why Law Firm Rankings Are Useless

#12

Post by School-Cotton »

wikiwawa wrote: post #11
libertymcateer wrote: post #4

> By applying machine learning algorithms to data mined from court judgments... And, with that sentence, this post becomes bullshit. The overwhelming majority of disputes do not reach litigation. The overwhelming majority of litigation does not reach judgment. And then a gigantic proportion of those judgments are appealed, and a non-trivial percentage are overturned. Even at that, only on the order of a single digit-…

A large part of statistics is devoted to making valid inference from noisy samples, largely by making sensible assumptions. Even in your example, it is common practice to assess a surgeon on outcomes of surgery, even though a lot of a surgeon's work does not lead to surgery. We aren't claiming to assess the wide practice of law, just litigation. We do Litigation Analytics.

do you understand the difference between "noisy" and "biased"?

Re: Why Law Firm Rankings Are Useless

#13

Post by harryh »

x1798DE wrote: post #9
gohrt wrote: post #5

Earlier quoted context omitted.

> Thus, there are fantastic lawyers working at Sierra Club or NRDC, but they're not going to have the same record of wins as similarly-good lawyers at a big New York firm. Not a problem for the analysis of law firms . Sierra Club is a less effective law firm than the big New York firm.

> Not a problem for the analysis of law firms. Sierra Club is a less effective law firm than the big New York firm. Why is this not a problem? The parent comment was saying that the Sierra Club takes harder cases and is often the plaintiff, not the defendant, so they may be more effective as a law firm but have worse numbers because the win/loss numbers don't mean the same things in different contexts.

It's also worth nothing that the Sierra Club isn't a law firm at all. It's an environmental organization that, among many other things, sometimes uses the legal system to advance its goals.

Re: Why Law Firm Rankings Are Useless

#14

Post by wikiwawa »

School-Cotton wrote: post #12
wikiwawa wrote: post #11

Earlier quoted context omitted.

A large part of statistics is devoted to making valid inference from noisy samples, largely by making sensible assumptions. Even in your example, it is common practice to assess a surgeon on outcomes of surgery, even though a lot of a surgeon's work does not lead to surgery. We aren't claiming to assess the wide practice of law, just litigation. We do Litigation Analytics.

do you understand the difference between "noisy" and "biased"?

I understand sarcasm, that's for sure.

As a matter develops, parties tend to drop out if they think that their chances have declined beyond some threshold. But if they continue, or more likely, their legal representation suggests they continue, it implies that they think they have a good chance of success.

Of course, you can get irrational litigants but most of the time, if it goes to judgement, both parties think they have a high chance of success. Otherwise they would have bailed.

Re: Why Law Firm Rankings Are Useless

#16

Post by hellcow »

libertymcateer wrote: post #4

> By applying machine learning algorithms to data mined from court judgments... And, with that sentence, this post becomes bullshit. The overwhelming majority of disputes do not reach litigation. The overwhelming majority of litigation does not reach judgment. And then a gigantic proportion of those judgments are appealed, and a non-trivial percentage are overturned. Even at that, only on the order of a single digit-…

Your comment is filled with unhelpful hyperbole that makes civil, constructive discussion much more difficult.

It seems reasonable to me that a firm with a great track record of judgments is more likely to succeed in other significant areas than a firm without a similarly great track record of judgments. That is, I would expect a strong, positive correlation between winning judgments and other desirable outcomes in disputes, like winning settlements.

We could even measure that correlation through a representative sample of firms to prove or disprove that hypothesis (and the OP probably should), but dismissing it outright with hyperbole isn't an effective response.

Re: Why Law Firm Rankings Are Useless

#19

Post by andrewfong »

rayiner wrote: post #2

As they acknowledge, it is essential to account for case difficulty in comparing success rates. Having been on both sides of the "v." I can say it is easier on the defense side (at least on the civil side). Defense counsel representing Big Cos. will have a better record not only because those companies are often the target of weaker lawsuits, but because various rules intended to filter out those weaker lawsuits stac…

If I were interested in litigation funding, I wouldn't evaluate the ability of law firms to litigate but rather the ability of lawyers to accurate predict the outcome of cases (regardless of whether they're actually representing the client) -- e.g. associates at law firms frequently write memos to assess the probability of success before a case makes it way to trial. Seems like there's value in actually correlating memos with actual outcomes.

Re: Why Law Firm Rankings Are Useless

#20

Post by ska »

hellcow wrote: post #16
libertymcateer wrote: post #4

> By applying machine learning algorithms to data mined from court judgments... And, with that sentence, this post becomes bullshit. The overwhelming majority of disputes do not reach litigation. The overwhelming majority of litigation does not reach judgment. And then a gigantic proportion of those judgments are appealed, and a non-trivial percentage are overturned. Even at that, only on the order of a single digit-…

Your comment is filled with unhelpful hyperbole that makes civil, constructive discussion much more difficult. It seems reasonable to me that a firm with a great track record of judgments is more likely to succeed in other significant areas than a firm without a similarly great track record of judgments. That is, I would expect a strong, positive correlation between winning judgments and other desirable outcomes in d…

   It seems reasonable to me that a firm with a great track record of judgments is more likely to succeed in other significant areas than a firm without a similarly great track record of judgments.
That seems like as strong enough assumption to require quite careful analysis and support. I know very little about the practice of law, but I am certain many other fields of endeavor just don't work like this.
Post reply on HN