Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scite: Smart Citations for Better Research

scite.ai

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Scite: Smart Citations for Better Research

#11

Post by sleepingsoul »

rmbyrro wrote: post #9
sleepingsoul wrote: post #7

Earlier quoted context omitted.

Hi there, Well, that's a tricky question to answer! :) At the moment we use DOIs to link papers / citations, and get our information on DOIs from crossref ( https://www.crossref.org/ ). So every DOI there is also in our system (and any citation coming from a source without a DOI currently won't be picked up). From crossref we also have metadata on which publications a given paper references, so we can always show inf…

I think you have an issue in your UI, or I don't understand how it's supposed to be used When I have the results from an article in my screen and use the top search to find another, it only updates the upper part of the page with the paper title, not the stats and summary below it I had searched for an article before that had no citations. That's why I continued to see "no citations" when I searched for the DOI I sha…

Hi,

Thanks for that description, I managed to reproduce this and it is our fault not yours. You are using it correctly.

We'll get it fixed ASAP!

EDIT: Fixed now :)

Re: Scite: Smart Citations for Better Research

#12

Post by rsa4046 »

uniqueuid wrote: post #2

On the tech side, this looks nice. But one thing people don't seem to take seriously enough is: If you're doing science and you don't know the literature, then you're doing a bad job. Understanding a paper won't work by reading AI summaries. Finding surprising new papers in your field should be very rare even without tech. [edit] Let me re-phrase the very negative statement above. There are a couple of important task…

This. As a recently retired academic research scientist (i.e., still ‘working’, but no longer in the lab, and no longer paid LOL), this can't be emphasized enough. Most scientists are highly specialized: experts in their local area of interest, the command of which must be fairly deep. They are aware of which papers are about to appear in press, either because they were either a referee at some point in the review process, or received a 'heads up' preprint.

Your statement on ‘finding cumulative truth, i.e. figuring out the overall state of research’ is a key point.To become newly acquainted with a research area (on one's own) is a daunting task, requires plowing through dozens and dozens of past research articles, review summaries, abstracts, texts, even conversing with authors of these works. But rapid progress is oft realized when people in two or more separate areas serendipitously interact in new projects, thus bringing expertise — heretofore missing or weak — to bear on a common problem. This is the intrinsically collaborative aspect of science that survives despite intense intramural competition.

Re: Scite: Smart Citations for Better Research

#13

Post by glup »

I find it interesting how this is premised on the "one big claim per paper" model, which for better or worse seems increasingly common. In my experience a lot of 60's-90's research is of the form "the results of our first study were consistent with [some specific paper], and our second experiment was not." This nuance is pretty hard for people to keep in mind, and often seems to get lost over time (as a postdoc in cognitive science at MIT, I've noticed a big difference between what papers actually say, vs. what people say those papers say 30 years on). I'm excited to see how well Scite actually works as I test it over the next week!

Re: Scite: Smart Citations for Better Research

#14

Post by uniqueuid »

sleepingsoul wrote: post #5
uniqueuid wrote: post #2

On the tech side, this looks nice. But one thing people don't seem to take seriously enough is: If you're doing science and you don't know the literature, then you're doing a bad job. Understanding a paper won't work by reading AI summaries. Finding surprising new papers in your field should be very rare even without tech. [edit] Let me re-phrase the very negative statement above. There are a couple of important task…

Hi, Thanks for the thoughtful response. Disclaimer: I work at scite. I want to clarify one point in your comment because I know a lot of people read HN comments before clicking through: we do NOT offer AI summaries! scite is fundamentally built on the idea that not all citations are equal. We offer disambiguation into citations by extracting the in-text contexts when citations happen between two papers. With that, yo…

Thanks for taking the time to reply.

I agree that those use cases are definitely valid. And I would also say that research is a technologically underdeveloped space. So any innovation here is great and will definitely help people!

The only worry I have (from past experience) is that easy tools have a differential impact: That they facilitate bad research more than they facilitate sound research. My perception as a scientist is that we primarily need to address actively harmful science (as in: spurious results, p-hacking). Fostering more science, on the other hand, is definitely a second or even third priority.

Re: Scite: Smart Citations for Better Research

#15

Post by sleepingsoul »

uniqueuid wrote: post #14
sleepingsoul wrote: post #5

Earlier quoted context omitted.

Hi, Thanks for the thoughtful response. Disclaimer: I work at scite. I want to clarify one point in your comment because I know a lot of people read HN comments before clicking through: we do NOT offer AI summaries! scite is fundamentally built on the idea that not all citations are equal. We offer disambiguation into citations by extracting the in-text contexts when citations happen between two papers. With that, yo…

Thanks for taking the time to reply. I agree that those use cases are definitely valid. And I would also say that research is a technologically underdeveloped space. So any innovation here is great and will definitely help people! The only worry I have (from past experience) is that easy tools have a differential impact: That they facilitate bad research more than they facilitate sound research. My perception as a sc…

Yep, I agree. I would like to point out though that one of our aims with this is to buffer against that bad research you mention.

Along with our extraction of citation statements, we have a deep learning model that classifies each statement as "supporting", "contrasting", or "mentioning" of the claims made in the original paper.

Indeed there is a lot of nuance here, and we've tried to address this in a number of ways (including a manual flag process for review where someone believes it is incorrectly labeled), and we're constantly improving in this area. In general we would expect that research that is not sound will receive more contrasting citations rather than supporting citations because their findings won't be as well supported by future research.

I should say explicitly that a "contrasting" citation is not a bad thing per se, especially given the nature of scientific discovery. But a study that was published through p-hacking whose findings are not reproducible should be easier to discern through subsequent contrasting citations. In today's world, all we might know is that it was cited a bunch, with the nuance buried underneath.

- Ashish

Re: Scite: Smart Citations for Better Research

#18

Post by darosati »

glup wrote: post #13

I find it interesting how this is premised on the "one big claim per paper" model, which for better or worse seems increasingly common. In my experience a lot of 60's-90's research is of the form "the results of our first study were consistent with [some specific paper], and our second experiment was not." This nuance is pretty hard for people to keep in mind, and often seems to get lost over time (as a postdoc in co…

Disclaimer: I work at scite.

We, at scite, actually designed scite to help folks see beyond the "one big claim per paper" model. Specifically, we designed scite to present show how specific results were either supported or contrasted by other papers, rather than showing that the publication itself was disputed or reproduced as a whole. If there was any feedback you could give on how we could present things so that folks don't get that impression we'd love it - feel free to DM me or let us know at hi@scite.ai as you test it out.

In an ideal world we would present the claims, results, and argumentation as structured information to the user but we are not quite at that stage yet!

Post reply on HN