Live data from Hacker News

Seaborn bug responsible for finding of declining disruptiveness in science

arxiv.org

41–50 of 77 posts

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#41

Bizarre. How do people make such big, splashy findings that can mess with people’s sense of optimism about science and innovation, without doing the simplest types of checks on their data and methodology.

No, the question is: how did peer review not catch it? I have the impression that reviewers don't have the time or incentive to give papers more that a cursory review. Independent of this case, a great many papers are published where the only "proof" is a user study or survey with an extremely low number of participants, but it still gets published. Many papers don't publish their datasets and don't contain enough detail to try and replicate their results.

There should be a real incentive/compensation for reviewing properly and real consequences if a paper gets retracted for reasons that should have been caught in review.

In this case it's fortunate that it did get found out in the end.

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#42
post #17
post #13

Seaborn is a visualization library. No statistical tests should have been done with seaborn as an intermediate processing step. I guess they used some of the convenience functions as part of the data analysis. Seaborn is a final step tool, not a data analysis tool. That's an embarrassing lesson to learn post-publication.

Take a look at the linked chart in my other comment. Visualization is absolutely a driver during research, it isn’t just an embarrassing revelation. Charts killed the Challenger crew.

[deleted]

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#43
post #17

Earlier quoted context omitted.

Take a look at the linked chart in my other comment. Visualization is absolutely a driver during research, it isn’t just an embarrassing revelation. Charts killed the Challenger crew.

> Charts killed the Challenger crew. https://www.tiktok.com/t/ZT8oG7ym6/ This is one of my favorite TikToks of all time, and you’ll see why. It goes into detail about how charts killed the Challenger crew. But the storytelling is second to none.

I composed this comment concurrently with yours, so I'm moving it here as a response...

This is going off-topic, but Tufte's attempt to cast the problem as fundamentally one of poor data presentation is rather self-servingly tendentious, IMHO, in a way that unfairly attributes a degree of culpability to the engineers who tried to stop the launch.

The excellent video you link to, taken as a whole, supports this view, I believe.

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#44

Earlier quoted context omitted.

No, bad analysis caused it. Graphs are at best secondary tools to interpret findings. You don't use graphs to draw conclusions.

But we do use plots to help identify problems in datasets, like all the time. Statistics 101

That's exploratory analysis. You don't use exploratory analysis as-is to draw conclusions. It guides you through your research but you do support it with other tools (statistical, logical). Science 101.

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#45

I saw this headline and my first thought was that someone was claiming that a mind impacting virus that evolved in the ocean was causing scientists to do research with less ambition. Which is of course ridiculous lol. But a bug in a visualization library impacting science is also ridiculous.

Oh man, I’ve been tricked so many times by the names of packages and frameworks… and here I did it to others. Sorry!

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#46

I saw this headline and my first thought was that someone was claiming that a mind impacting virus that evolved in the ocean was causing scientists to do research with less ambition. Which is of course ridiculous lol. But a bug in a visualization library impacting science is also ridiculous.

Have you ever seen the “BrainDead” series? This idea is not unheard of.

https://www.imdb.com/title/tt4877736

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#47

I saw this headline and my first thought was that someone was claiming that a mind impacting virus that evolved in the ocean was causing scientists to do research with less ambition. Which is of course ridiculous lol. But a bug in a visualization library impacting science is also ridiculous.

Have you ever seen the “BrainDead” series? This idea is not unheard of. https://www.imdb.com/title/tt4877736

Thanks for reminding me of this oldie https://m.youtube.com/watch?v=GVvL2ca65DA

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#48
Of course, it has nothing to do with rampant fraud, unreproducible results, incentive structures which reward the number of papers over the quality of papers, having researchers spend their prime scientific years writing grant proposals instead of actual research...

...nor does it have anything to do with tech companies hoarding cash by the trillions of dollars oversees instead of spending it on R&D, and even what R&D they internally produce they have no incentive to publish or productize, because virtually no new business will be more profitable than the monopoly business they already have...

Re: Seaborn bug responsible for finding of declining disruptiveness in science

#50
post #26
post #24

Earlier quoted context omitted.

it is science that discovers errors in science :)

And here we are, discovering errors!

> Now these points of data make a beautiful line. And we're out of beta. We're releasing on time.
Post reply on HN