A few months ago a couple of physics postdocs set up a website (vanityindex dot com) and proposed a few "vanity metrics". Can be fun to check out your own vanity index and that of your colleagues.
Hundreds of extreme self-citing scientists revealed in new database
141–150 of 215 posts
Re: Hundreds of extreme self-citing scientists revealed in new database
#142Perhaps it would be useful for reviewers to point out which citations do not contribute to the paper? It really is a tough problem. If someone is toiling along in some niche they have carved out, they and their colleagues may be the only one working in that space. That leads to a lot of cross citation and self citation. That said, if you publish paper A, and then cite it in paper B which builds on that work, then in…
As a reader I personally prefer if they do a more complete set of citations, instead of making me follow up a multi-step chain to dig them up, as if I'm a compiler resolving transitive dependencies. I like little history-map sentences like: "This technique was introduced by Foo (1988) and recast in the modern computational formalism by Bar (2009); the present work uses an optimized variant (Bar 2012)." You could just…
That kind of sentence is gold.
Often half or (much) more of the value of a paper is in the references, and that's not a bad thing. Sometimes it is the first thing I read.
There's no ink shortage, no link limit on the Internet, and every paper has an abstract for quick filtering. As a curious person I want everything that serves to establish the argument cited so I can be guided to papers of interest and get a better idea of where an idea fits in the broader field.
Re: Hundreds of extreme self-citing scientists revealed in new database
#143Went into the data and took the top 1000 individuals with self cite percentages over 40%, then sorted by institution. Nearly every major institution had individuals in this group: Johns Hopkins (4), Cal Tech (4), Georgia Tech (2), MIT (5), each of the Max Planck Institute campuses (3-7), Moscow State (7), Penn State (6), Stanford (1), Utrecht (2), University of Zurich (4), ETH Zurich (1), DLR (3), Imperial College London (3), University of Tokyo (2), Princeton (5), Kyoto University (4)...
I feel like if this problem were very concerning we'd see the distribution concentrated at certain institutions but I'm not sure there's one with over 10 researchers at them. We hear a lot about questionable Chinese journals, but the highest institution in this list is the Chinese Academy of Sciences with 3 individuals.
I think the more likely case is there are a few bad apples, some bad practices we can't ever fully get rid of, and that some research lends itself more to self-citation.
Re: Hundreds of extreme self-citing scientists revealed in new database
#144Earlier quoted context omitted.
Err, isn't PageRank almost the same as "citations"?
No, because it counts citations from influential papers with a higher weight.
That's the same with academic scores. Citations in the "Self-published journal of amateur chiropractors" don't buy you much academic credit...
In fact PageRank was inspired by academic rankings in that aspect:
"PageRank was influenced by citation analysis, early developed by Eugene Garfield in the 1950s at the University of Pennsylvania, and by Hyper Search, developed by Massimo Marchiori at the University of Padua. In the same year PageRank was introduced (1998), Jon Kleinberg published his work on HITS. Google's founders cite Garfield, Marchiori, and Kleinberg in their original papers."
Re: Hundreds of extreme self-citing scientists revealed in new database
#145Earlier quoted context omitted.
I think a problem here is Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." [1] And it seems like there's an element of the streetlight effect [2], too; sometimes a bad metric really is worse than no metric. Also, I really question your notion that people outside a field should be able to evaluate the quality of someone's work, especially in academia, where the whole point is to be we…
Okay, I'll try to clarify what exactly I mean by "people outside a field should be able to evaluate the quality of someone's work" - especially because, as I said regarding peer review, we generally consider that it's impossible to do so directly . It's about the question of resource allocation. Pretty much every subfield of academia is a net consumer of resources, i.e. someone outside of that subfield is funneling r…
Require all academic scientists to be self funded...no universities, no gov't agencies, no foundations, no philanthropists...problem solved.
Re: Hundreds of extreme self-citing scientists revealed in new database
#146The core problem here is that universities think that citation statistics are a useful metric to evaluate the quality of the work of a scientist. There's plenty of evidence that this is not the case or that even the reverse may be the case [1], but this idea refuses to die. [1]
PageRank might be better way to evaluate quality. It too can be gamed. Maybe not as easily, though.
I think this is a neat idea. Basically, you'd get more "credit" if you're cited by a good paper than if you're cited by a bad paper.
Re: Hundreds of extreme self-citing scientists revealed in new database
#147Earlier quoted context omitted.
> PageRank might be better way to evaluate quality And suddenly Google is the authoritative source on literally everything in the world. I hope you like their political views, because they would become "the one".
Pagerank is referring to the graph algorithm known as pagerank, not anything provided specifically by Google.
"PageRank" is a Google trademark, and also the subject of a patent belonging to Stanford University and licensed exclusively to Google.
Re: Hundreds of extreme self-citing scientists revealed in new database
#148Earlier quoted context omitted.
I think a problem here is Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." [1] And it seems like there's an element of the streetlight effect [2], too; sometimes a bad metric really is worse than no metric. Also, I really question your notion that people outside a field should be able to evaluate the quality of someone's work, especially in academia, where the whole point is to be we…
> the whole point is to be well ahead of what most people can understand That’s not the case at all. Being at the leading edge of research should mean that you are creating new knowledge. That doesn’t imply that people cannot understand it. This expectation that laypeople cannot possibly understand science is one of the reasons so many papers are written so densely and obtusely. “They” can’t understand it anyway, rig…
I think a lot of cutting-edge work is also done at the edge of understanding, and that's fine. It can be hard enough to explain groundbreaking work to experts with deep context; it's reasonable to me that it takes more time and work to find the explanations that make sense to the average person.
I do agree that researchers should be able to give decent "here's what I do" explanations to the general public. But that's very different than a member of the general public understanding the context well enough that they can judge the value of the work to the field.
Re: Hundreds of extreme self-citing scientists revealed in new database
#149The fundemental problem with science today is that it needs funding. Funding means vested interests.
If a scientist is trying to game the system, they are doing it to secure future funding. Either they are gaming the system to make the funder look good, or to appeal to a future funder.
Re: Hundreds of extreme self-citing scientists revealed in new database
#150Earlier quoted context omitted.
I think a problem here is Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." [1] And it seems like there's an element of the streetlight effect [2], too; sometimes a bad metric really is worse than no metric. Also, I really question your notion that people outside a field should be able to evaluate the quality of someone's work, especially in academia, where the whole point is to be we…
Okay, I'll try to clarify what exactly I mean by "people outside a field should be able to evaluate the quality of someone's work" - especially because, as I said regarding peer review, we generally consider that it's impossible to do so directly . It's about the question of resource allocation. Pretty much every subfield of academia is a net consumer of resources, i.e. someone outside of that subfield is funneling r…
Just to be clear, I understood what you were saying before. I just disagree. I think the right approach is to select trusted experts and let them make the decisions about their fields.
Again, since this is a tech community, let me use that for an analogy. It's a classic problem for non-technical founders to evaluate their technical hires. They aren't qualified.
The right solution is not to find some gameable metric of tech-ness, like LoC/day or Github stars. Instead one uses either direct experience-based trust or some sort of indirect trust, like where you have a technical expert you trust and have that person interview your first tech hires.
Yes, having expert humans make the decisions is imperfect. But it's not like a managerialist approach is either. And the advantage of using expert humans, rather than a gameable metric and managerial control, is that we have centuries of experience in how people go wrong and many good approaches for countering it.