Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to (seriously) read a scientific paper (2016)

science.org

Post reply on HN
Print view View original

31–40 of 66 posts

Re: How to (seriously) read a scientific paper (2016)

#31

Post by dekhn »

I see a lot of people who basically skip to the figures, and take them at face value, and read nothing more about the paper.

When I read a paper I start with the abstract and go to the methods. then I inspect the figures carefully for the common tricks scientists play- axes tricks, overfitting, etc, looking to see whether I'm c onvinced the figures actually came from running an experiment, or just from copying and pasting other gels (this is extremely common).

I used to do: read the whole paper in order, recursively visiting each citation (often 2-3 levels deep); within a few hours, I was convinced the entirety of science is build on somewhat shoddy foundations.

Re: How to (seriously) read a scientific paper (2016)

#32

Post by passwordoops »

dekhn wrote: post #31

I see a lot of people who basically skip to the figures, and take them at face value, and read nothing more about the paper. When I read a paper I start with the abstract and go to the methods. then I inspect the figures carefully for the common tricks scientists play- axes tricks, overfitting, etc, looking to see whether I'm c onvinced the figures actually came from running an experiment, or just from copying and pa…

The methods section is my major judgement call. That's where you set off they have a clue what they're doing and care enough to worry about ensuring the work can be received. Which, after all, should be the main reason a to publish a scientific paper based on experiments

Re: How to (seriously) read a scientific paper (2016)

#34

Post by ants_everywhere »

timr wrote: post #29
ants_everywhere wrote: post #24

Earlier quoted context omitted.

Yeah it depends heavily on the field and sometimes on the subfield. Also the approach I use depends on how much I know about the field. If I know less, I weight the introduction more. If I know a lot, I skip almost all the sales and stage setting content. E.g. if it's a psych paper, often I jump directly to the results and only read other content if the results look reasonable. And so on. So, I guess my only real poi…

It's certainly true that it depends on the subfield and somewhat on your level of experience in that field. But the approach I outlined works pretty well, generally, and only serves to underscore how bad many papers are. For example, I'm not going to be able to pick up a paper from CERN and do most of the steps I outlined, but if I did do that, and found an obvious statistical anomaly in the first table, well...now I…

oh yeah, I agree. I was agreeing and expanding on your comment, not disagreeing with it.

Re: How to (seriously) read a scientific paper (2016)

#35

Post by mturmon »

sxcurry wrote: post #25
timr wrote: post #17

> As editor-in-chief of Science....Generally, I start with the corresponding editors' summaries, which are meant for someone like me: a science generalist who is interested in everything but dives deeply only into one field. Next, I check to see if someone wrote a News article on the paper. Third, I check to see if there is a Perspective by another scientist. This says so much about the volume of absolute crap that g…

I guess you didn’t read the article very closely. The Science editor was talking about Science News (note the capitalization) which is a well respected information resource.

Alas, I believe you two are not communicating!

Marcia McNutt, the (now former) editor-in-chief of Science, was certainly talking about the News section of Science magazine, the magazine of the AAAS.

That is, McNutt is talking about: https://www.science.org/news

This is not the same as "Science News" -- https://www.sciencenews.org/sn-magazine

The reason I think this is that the next sentence refers to a "Perspective" (also with initial capital, just like she used earlier with "News"):

> Third, I check to see if there is a Perspective by another scientist.

If you read Science regularly, you know there is a News section that has some "hot topic" stuff, like "Curiosity rover discovered ancient lakebeds on Mars", or "Arecibo telescope collapsed." It's right up front.

And following the News section of Science magazine, there is a Perspectives section that has contextual overviews of less "front-page" technical stuff that is still important, but more niche. ("A major advance in modeling of superconductors that could yield new materials in upcoming years" type thing.)

Perspectives are tied to a technical article, and News is sometimes, but not always.

I read the print version of Science during these years. It was outstanding. My recollection is that they didn't usually do both a "News" and a "Perspective" on the subject of a technical article. It was typically one or the other.

*

My personal opinion is that this order (I look at News and then at Perspectives) is perfectly fine. In particular (@timr), Marcia McNutt is not talking about reading "the news", as in "the newspaper."

Slagging Marcia McNutt as some kind of lightweight is all kinds of wrong. She's the president of US National Academies of Science, and that's just the top line (https://www.nasonline.org/member-directory/members/52683.htm...).

This kind of callow dismissal is one of the least attractive aspects of HN.

Re: How to (seriously) read a scientific paper (2016)

#36

Post by jamilton »

timr wrote: post #28
sxcurry wrote: post #27

Earlier quoted context omitted.

Well, I don’t agree with your opinion of Science News, but that’s OK. What I was pointing out is that she was indeed referring to Science News (and also Science Perspectives) as sources for her reading and understanding of a paper.

Maybe that was implied by the capitalization of "News", but regardless, it's the same argument. Reporters are not scientsts. The whole thing is like a hilarious public announcement that the editor in chief of Science has Gell-Mann Amnesia.

If I'm interpreting it right, no, it's not the same argument. It sounds like she said she basically checks if it's been talked about in Science besides as an existing paper. Seems reasonably for a publication to consider it's own standards of publishing as good enough.

Science reporters are probably, hopefully, scientifically literate.

Re: How to (seriously) read a scientific paper (2016)

#37

Post by jamilton »

paulpauper wrote: post #12
amelius wrote: post #2

Still waiting for an open discussion forum for scientific papers. Like StackOverflow, but every post is a paper. And there are experts of various gradations that help with questions.

It would be quickly overwhelmed, especially for complicated math papers

Why is that?

Re: How to (seriously) read a scientific paper (2016)

#38

Post by etrautmann »

dekhn wrote: post #31

I see a lot of people who basically skip to the figures, and take them at face value, and read nothing more about the paper. When I read a paper I start with the abstract and go to the methods. then I inspect the figures carefully for the common tricks scientists play- axes tricks, overfitting, etc, looking to see whether I'm c onvinced the figures actually came from running an experiment, or just from copying and pa…

What do you mean copy pasting? Like straight up fraud and fabricating results of a blot?

Re: How to (seriously) read a scientific paper (2016)

#39

Post by mncharity »

mturmon wrote: post #35
sxcurry wrote: post #25

Earlier quoted context omitted.

I guess you didn’t read the article very closely. The Science editor was talking about Science News (note the capitalization) which is a well respected information resource.

Alas, I believe you two are not communicating! Marcia McNutt, the (now former) editor-in-chief of Science , was certainly talking about the News section of Science magazine, the magazine of the AAAS. That is, McNutt is talking about: https://www.science.org/news This is not the same as "Science News" -- https://www.sciencenews.org/sn-magazine The reason I think this is that the next sentence refers to a "Perspective"…

> It was outstanding.

Fond memories. I've wondered if it, and Nature's equivalent, might be used for llm training? Hoping to get correct stories, let alone insightful and integrated ones, from textbooks, reddit, and wikipedia... seems unlikely to end well. Perhaps also survey papers. And parts of doorstop-tome-on-topic's. Maybe the intro and related work sections of research papers? Pity research talk intros and Q&As are rarely captured.

Re: How to (seriously) read a scientific paper (2016)

#40

Post by dekhn »

etrautmann wrote: post #38
dekhn wrote: post #31

I see a lot of people who basically skip to the figures, and take them at face value, and read nothing more about the paper. When I read a paper I start with the abstract and go to the methods. then I inspect the figures carefully for the common tricks scientists play- axes tricks, overfitting, etc, looking to see whether I'm c onvinced the figures actually came from running an experiment, or just from copying and pa…

What do you mean copy pasting? Like straight up fraud and fabricating results of a blot?

Yes, apparently copying gel images is fairly common. And the folks who do it don't give much thought to how easy it is to detect, by skilled human eye or by computer. https://www.nytimes.com/interactive/2022/10/29/opinion/scien... https://www.nytimes.com/2024/01/22/health/dana-farber-cancer... https://www.nytimes.com/2023/07/19/us/stanford-president-res...

I should say, when I learned about Elisabeth Bik it was a revelation because I never thought other people saw copy patterns in papers, although I always assumed it just happened accidentally (I've run gels before and the whole process is terribly manual and prone to mistakes).

Post reply on HN