Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Too many new papers may mean novel ideas rarely rack up citations

blog.pnas.org

Post reply on HN
Print view View original

221–230 of 241 posts

Re: Too many new papers may mean novel ideas rarely rack up citations

#221

Post by native_samples »

germandiago wrote: post #39
LurkingPenguin wrote: post #32

Earlier quoted context omitted.

> Hell, the only thing that's waning is public perception of science and that seems to be a deliberate political attack. No doubt there are political attacks on science, but it's also true that science is often politicized and also corporatized, both by forces outside of science and those who practice it.

How come a person that is interested in doing science politizes it? I think the first party interested in doing it is people who live from politics, not from science. At least not people who honestly live from science.

Well because they see science as a means to an end (political control), not an end in and of itself.

For example, epidemiology has this problem. Epidemiologists routinely publish supposedly scientific papers that are actually policy papers in disguise. Often these papers violate the scientific method in some major way. Nobody seems to care. They get published regardless. Here's an example:

https://www.nature.com/articles/s41598-021-95699-9#Sec7

It unashamedly cherry-picks the UK, Denmark and Sweden to try and argue that the UK/Sweden should have adopted Danish policy. Why not study all countries for which data is available? They easily could have done, the data is there. But they don't because if you do that you end up with a null result (no policy makes reliably makes any difference). So they cherry pick in order to be "informative" (as they put it) with respect to policy.

Re: Too many new papers may mean novel ideas rarely rack up citations

#222

Post by native_samples »

iamcurious wrote: post #139
findalex wrote: post #18

Earlier quoted context omitted.

> Disagreement, the possibility to be stupid and wrong, and the ability to take random choices based on intuition are eliminated when the majority of a field acts like scientific managers. How do we democratize physical sciences in the same way as CS? My bet would be on the combination of the two (simulation) and providing the high-level tools to the masses.

By focusing on Small Science. Big Science needs big money, big equations, big machines, big careers, big meetings, etc, so big science should be last resort. Make experiments cheap, cheap enough that it becomes embarrassing not to double check. Cheap enough that half of the comments in a science article are about people doing the experiment themselves right there.

That doesn't work. The scientific literature is already flooded with cheap theory papers and a clear absence of real heavy lifting. Such papers are routinely filled with basic errors or flawed methodologies. There's no embarrassment associated with not double checking, failed replications happen all the time and there are usually no consequences.

Re: Too many new papers may mean novel ideas rarely rack up citations

#223

Post by native_samples »

echelon wrote: post #17
goalieca wrote: post #7

Was academia terribly relevant in my field when I was a grad student? Nope. In general, the work was poorly supervised, not-reproducible, and the peer review process was completely broken. Students just wanted to graduate and professors just wanted tenure and funding points.

Something has changed. I can't imagine Einstein's generation functioning like that. Have pioneers of innovation moved to private corporations now that they have capital that rivals academia? Private companies can reward innovators with more than just credentials. Bell Labs, Xerox Parc, Google Brain, OpenAI, Tesla, SpaceX, ... Granted, this isn't even across all fields as they are not all economic drivers.

Well but Einstein wasn't an academic when he was doing his best work. After he became famous he became an academic in the USA and (I read somewhere) spent much of the rest of his life being quite depressed because he never again reached the epic highs he achieved in his 20s.

Re: Too many new papers may mean novel ideas rarely rack up citations

#224

Post by carbocation »

prionassembly wrote: post #178
carbocation wrote: post #158

Earlier quoted context omitted.

It's painful because (IMHO) the H index is just a much worse approximation of something that we could actually achieve with PageRank for academic citations. In that case, a bunch of middling papers would be rewarded, but so too would one critical paper that lays a foundation for a field.

Wouldn't betwenness centrality be better? Think of an army randomly moving through your citation graph; the more particularly nodes are trampled over, the more pagerank it has. Now: if this army is informed about the shortest routes and instead moves about optimally, the most-trampled over places have higher betweenness centrality. I'd like my simulated citing scientist to be smart.

I've tried applying PageRank to pubmed with reasonable results, but betweenness does sound compelling. Thanks!

Re: Too many new papers may mean novel ideas rarely rack up citations

#225

Post by tpoacher »

blt wrote: post #144
tpoacher wrote: post #90

My view is that, like many other fields (including notably software engineering) before it, Academia has fallen victim to Goodhart/Campbell's law. Goodhart's law is an adage named after economist Charles Goodhart: "When a measure becomes a target, it ceases to be a good measure." This follows from individuals trying to anticipate the effect of a policy and then taking actions which alter its outcome. Campbell's law (…

What is a REF exercise?

Ah it's a UK thing. Not sure what kind of equivalents other countries have, but I'm sure they do in similar forms.

It stands for Research Excellence Framework, and effectively it's a metric used by government and research councils to allocate funding to more 'deserving' universities / groups, by rewarding departments with "high quality paper outputs".

This has led to effectively departments launching tedious bureaucratic exercises and workshops where all members of staff need to submit summaries for "star-ratings" for departmental sifting, submissions of only certain kinds of papers to particular journals, eschewing both high-risk and low-hanging-fruit research in favour of controversial or 'popular' topics which are more likely to gather citations, "buying" researchers for their paper co-authorships within the currently valid REF-cycle rather than based on their research/teaching skills or interests, etc.

Re: Too many new papers may mean novel ideas rarely rack up citations

#226

Post by _Wintermute »

native_samples wrote: post #220
xmprt wrote: post #140

Earlier quoted context omitted.

Biology is one field that seems to be making massive breakthroughs recently. On the other hand, it seems like a lot of theoretical sciences like Physics haven't have similar breakthroughs in decades.

Is it? Biology is one of the fields most often cited as stagnant. Yes there's lots of papers, and some progress in understanding at the micro level, but progress against the most important diseases is nearly zero (see: Alzheimers, obesity, heart disease). Also, the field is full of low standards and fraud. That's one reason why there's so much arguing over COVID, and why tech VCs stay away from it. Way too many impre…

> In biology you get things like Theranos, which is the sort of long term scam that's much harder to pull off in the software world.

It's funny that in the biomedical circles, it's been remarked that Theranos is what happens when the software world (Silicon Valley) try their hand at biology. You can't fake it to you make it here, if the science doesn't work it doesn't work, no matter how much VC funding and employees you throw at the problem.

Re: Too many new papers may mean novel ideas rarely rack up citations

#228

Post by mrjangles »

Ar-Curunir wrote: post #215
mrjangles wrote: post #164

Earlier quoted context omitted.

As a general rule, if you are hearing about some scientific endeavor in the popular press, it is because that science isn't very important, and they need publicity to get funding. What a lot of people don't understand is that there is actually a lot of real science going on in physics. There are two branches of physics, what you call condensed matter / atom optics. And then there is Cosmology / High energy physics. c…

We know less about the oceans on our planet than the surface of the moon. Does that make lunar scientists “crackpots”?

haha.. funny because it is literally exactly the same thing. No we do not know more about the surface of the moon than our oceans. When academics can't be proven wrong, they come up with all sorts of theories that they are certain of.

We know so little about the surface of the moon that we don't even know if it is possible to land a rocket on it or not.

Re: Too many new papers may mean novel ideas rarely rack up citations

#229

Post by prionassembly »

carbocation wrote: post #224
prionassembly wrote: post #178

Earlier quoted context omitted.

Wouldn't betwenness centrality be better? Think of an army randomly moving through your citation graph; the more particularly nodes are trampled over, the more pagerank it has. Now: if this army is informed about the shortest routes and instead moves about optimally, the most-trampled over places have higher betweenness centrality. I'd like my simulated citing scientist to be smart.

I've tried applying PageRank to pubmed with reasonable results, but betweenness does sound compelling. Thanks!

I'll give you a bonus: the Louvain algorithm for community detection. Whatever ships with Networkx, Gephi, etc. doesn't work for my (correlation-derived, pruned with graphical lasso) networks, but the Louvain method (a greedy approximation to modularity maximization; the real math magic is in the concept of modularity and the configuration model) is awesome.

Re: Too many new papers may mean novel ideas rarely rack up citations

#230

Post by native_samples »

_Wintermute wrote: post #226
native_samples wrote: post #220

Earlier quoted context omitted.

Is it? Biology is one of the fields most often cited as stagnant. Yes there's lots of papers, and some progress in understanding at the micro level, but progress against the most important diseases is nearly zero (see: Alzheimers, obesity, heart disease). Also, the field is full of low standards and fraud. That's one reason why there's so much arguing over COVID, and why tech VCs stay away from it. Way too many impre…

> In biology you get things like Theranos, which is the sort of long term scam that's much harder to pull off in the software world. It's funny that in the biomedical circles, it's been remarked that Theranos is what happens when the software world (Silicon Valley) try their hand at biology. You can't fake it to you make it here, if the science doesn't work it doesn't work, no matter how much VC funding and employees…

Wow. I really hope that's not really the takeaway biotech people are getting from Theranos.

Firstly, Therano's investors were mostly not tech VCs - I can only find one in the list actually, unless you count Larry Ellison in a private capacity. Theranos got a lot of money from non-tech investors like Rupert Murdoch, Walgreens, private equity etc. One reason they had to do this is exactly because tech VCs know to stay away from anything biotech related - the people who work in that space should be wondering why.

Secondly, Holmes did fake it to make it. That's the point. Faking it was sufficiently easy that they went years before the scam fell apart, they even got their machines deployed!

What's the software equivalent of such a large scale, 100% fraudulent operation? I can't think of any offhand, because it's normally pretty clear early on whether software is working or not.

Post reply on HN