Hundreds of extreme self-citing scientists revealed in new database • Hacker News Board

Re: Hundreds of extreme self-citing scientists revealed in new database

#71

Post by gdhbcc » Tue, Aug 20, 2019, 10:18 PM UTC

PeterisP wrote: post #57

hannob wrote: post #21

The core problem here is that universities think that citation statistics are a useful metric to evaluate the quality of the work of a scientist. There's plenty of evidence that this is not the case or that even the reverse may be the case [1], but this idea refuses to die. [1]

It sucks as a metric but it does have some rough correlation in most cases, and I'm not aware of any better easily measurable metric - if you have one in mind, it'd be great to hear. The alternative of having a bureaucrat "simply judge quality" IMHO is even worse, even less objective, and even more prone to being gamed. The main problem is that there is an objective need (or desire?) by various stakeholders to have s…

Why not simply use replication as a measure? Have your studies been replicated? How many other studies have you replicated?

Would both help solve the replication crisis, and resolve this problem.

Of course then you might have 10 000 studies replicating the same easy to do study... which is why the "score" should be reduced based on how many other times that study has been replicated.

Re: Hundreds of extreme self-citing scientists revealed in new database

#72

Post by asimjalis » Tue, Aug 20, 2019, 10:18 PM UTC

hannob wrote: post #21

The core problem here is that universities think that citation statistics are a useful metric to evaluate the quality of the work of a scientist. There's plenty of evidence that this is not the case or that even the reverse may be the case [1], but this idea refuses to die. [1]

PageRank might be better way to evaluate quality. It too can be gamed. Maybe not as easily, though.

Re: Hundreds of extreme self-citing scientists revealed in new database

#73

Post by your-nanny » Tue, Aug 20, 2019, 10:20 PM UTC

duskwuff wrote: post #18

jacquesm wrote: post #7

Earlier quoted context omitted.

Sure, but there is no 'pruning of the tree' in case a dead end is reached, so the citations stay allowing the quack to pretend they have more credibility than they do. In fact, the whole idea of these citations is to build credibility where there is none.

Right. The purpose of citation metrics is to measure the intellectual influence of a person's ideas. If an academic's works are never being cited by anyone else, the correct conclusion would be that their ideas have little influence -- regardless of how many self-citing papers they've published.

influence is not the same thing as credibility.

Re: Hundreds of extreme self-citing scientists revealed in new database

#74

Post by ci5er » Tue, Aug 20, 2019, 10:23 PM UTC

jedberg wrote: post #20

std_throwaway wrote: post #9

Earlier quoted context omitted.

If you work in a very narrow field of science you're basically on your own and have to cite yourself because there's nobody else to cite.

In which case you have to ask yourself, are you so brilliant that you’ve found an important topic that no one has considered yet, or have all the brilliant people already figured out that topic isn’t worthy of study? It’s the same with the startup world. If you’re the only one doing a thing, are you brilliant or foolish?

This is actually a very interesting question that might take one down the rabbit hole.

Acquiring knowledge (I should say 'beliefs about valid knowledge') and brainstorming (and certainly collaboration and getting an advisor to adopt you) appear to be social activities, as much as purely logical and analytic activities.

Social activities like this, for social or herd creatures, are subject to flock or swarming patterns.

Maybe all the brilliant people are swarming around a locus of interest? It's certainly a good way to have the population explore the ins and outs of a, well, locus of interest. It's also a good way to have a loner get shunned by wandering off and poking at an uninteresting pile of dung.

I guess my point is: why not both? (Mathematically, statistically, egotistically, I know the idea that I am the foolish one is almost certainly more likely to be the case)

Re: Hundreds of extreme self-citing scientists revealed in new database

#75

Post by jeffwass » Tue, Aug 20, 2019, 10:25 PM UTC

My undergrad college physics professor (Fay Ajzenberg-Selove) introduced this metric back in the 50’s. She faced major sexism and bullshit claims against her productivity, and had to use this metric to prove her detractors wrong, that she was as good as or better than most of her male colleagues in terms of performing useful and interesting research, to earn herself a faculty position.

https://en.m.wikipedia.org/wiki/Fay_Ajzenberg-Selove

Re: Hundreds of extreme self-citing scientists revealed in new database

#76

Post by apathy » Tue, Aug 20, 2019, 10:31 PM UTC

asimjalis wrote: post #72

hannob wrote: post #21

The core problem here is that universities think that citation statistics are a useful metric to evaluate the quality of the work of a scientist. There's plenty of evidence that this is not the case or that even the reverse may be the case [1], but this idea refuses to die. [1]

PageRank might be better way to evaluate quality. It too can be gamed. Maybe not as easily, though.

Aka eigenfactor

Re: Hundreds of extreme self-citing scientists revealed in new database

#77

Post by ChuckMcM » Tue, Aug 20, 2019, 10:32 PM UTC

Perhaps it would be useful for reviewers to point out which citations do not contribute to the paper? It really is a tough problem. If someone is toiling along in some niche they have carved out, they and their colleagues may be the only one working in that space. That leads to a lot of cross citation and self citation.

That said, if you publish paper A, and then cite it in paper B which builds on that work, then in paper C you really only need to cite paper B if you're building on the work, not B and A. It might make for in interesting data set to plot out those sorts of relationships.

Re: Hundreds of extreme self-citing scientists revealed in new database

#78

Post by SI_Rob » Tue, Aug 20, 2019, 10:34 PM UTC

austinjp wrote: post #31

tomtomistaken wrote: post #16

Earlier quoted context omitted.

More like Goodhart‘s law: "When a measure becomes a target, it ceases to be a good measure."

"What gets measured gets treasured."

and the corollary, "the uncountable is of no account"

Re: Hundreds of extreme self-citing scientists revealed in new database

#79

Post by your-nanny » Tue, Aug 20, 2019, 10:35 PM UTC

guntars wrote: post #5

your-nanny wrote: post #3

People who are working out their ideas far outside the mainstream have no one to cite but them selves. Some are quacks, to be sure, but sometimes a field is just not ready for their work because the utility of the idea is not easily apparent, or because it's perceived to be too risky. A field needs a healthy mix of the the curmudgeonly stubborn thinkers going their way no matter the cost, and those making steady prog…

Do you have any good examples, historical or otherwise?

possibly Sutton and Barto's work on reinforcement learning throughout the 80s.

Re: Hundreds of extreme self-citing scientists revealed in new database

#80

Post by opo » Tue, Aug 20, 2019, 10:36 PM UTC

SketchySeaBeast wrote: post #62

kromem wrote: post #51

Earlier quoted context omitted.

Two economists are walking down the street. One sees a $20 bill and starts to bend over to pick it up. The other economist says "Don't bother - if it were really worth $20, someone else would have picked it up." There's countless numbers of scientists that made great strides working on topics others deemed rediculous. Heck, many of the Nobel prize winners were ridiculed by their colleagues as borderline wack-jobs at…

> Heck, many of the Nobel prize winners were ridiculed by their colleagues as borderline wack-jobs at the time they were working on their research. Even after winning the prize, some still were with their later work (Crick's search for consciousness comes to mind, and why it would be so worthless a search does not). Do you have any good examples of being considered wack-jobs before their winning?

It isn't too hard to find examples of scientists who were ridiculed for their ideas and eventually win the Noble prize:

>...Stanley B. Prusiner, a maverick American scientist who endured derision from his peers for two decades as he tried to prove that bizarre infectious proteins could cause brain diseases like “mad cow disease” in people and animals, has been awarded the ultimate in scientific vindication: the Nobel Prize in medicine or physiology.

>...Prusiner said the only time he was hurt by the decades of skepticism “was when it became personal.” After publication of an especially ridiculing article in Discover magazine 10 years ago, for example - which Prusiner Monday called the “crown jewel” of all the derogatory articles ever written about him - he stopped talking to the press. The self-imposed media exile became increasingly frustrating to science journalists over the past decade as his theories gained scientific credibility.

https://www.spokesman.com/stories/1997/oct/07/nobel-prize-vi...

>....The recent 2011 Nobel Prize in Chemistry, Daniel Schechtman, experienced a situation even more vexing. When in 1982, thirty years ago, he made his discovery of quasicrystals, the research institution that hosted him fired him because he « threw discredit on the University with his false science ».

>...He was the subject of fierce resistance from one of the greatest scientists of the 20th century, Linius Pauling, Nobel Laureate in Chemistry and Peace Nobel Laureate. In 1985, he wrote: Daniel Schechtman tells non-sence. There are no quasi-crystals, there are only quasi-scientists!

https://www.davidovits.info/publish-or-perish-2-2011-nobel-p...

An example that is pretty well known is Barry Marshall

>...In 1984, 33-year-old Barry Marshall, frustrated by responses to his work, ingested Helicobacter pylori, and soon developed stomach pain, nausea, and vomiting -- all signs of the gastritis he had intended to induce.

>...Marshall wrote in his Nobel Prize autobiography, "I was met with constant criticism that my conclusions were premature and not well supported. When the work was presented, my results were disputed and disbelieved, not on the basis of science but because they simply could not be true."

https://abcnews.go.com/Health/PainManagement/25-years-peptic...

It was Max Plank who said "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." - so this isn't a new issue and things are probably better now than they were in the past.