Retire This Idea: Scientific Knowledge Structured as “Literature” • Hacker News Board

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#31

Post by nopinsight » Mon, May 04, 2015, 9:05 PM UTC

One possibility is to keep the current publishing model but add an additional "meta" layer in a central repository to indicate the relations between the contributions of each paper.

The repository should be independent of an individual journal or academic field so that multidisciplinary collaboration would be easier and more visible. If designed well, cross-pollination of ideas between fields may multiply.

Each node in the dependency graph should not represent an entire paper, but a chunk of contribution (knowledge, information, or experimental data), with an option to delve deeper to see its source code/raw data. A hierarchy of compositional structures can represent broader units of knowledge. Citation links should allow for labels to indicate the reason(s) for citation: prior work, refutation, tool, methodology, etc.

This is a more practical transition than revamping the whole system and should still add much valuable information for researchers and consumers of research.

If realized, an automated system may be able to analyze the graph for additional insights into our knowledge structure and point to interesting research directions as well.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#32

Post by CHY872 » Mon, May 04, 2015, 9:09 PM UTC

nopinsight wrote: post #24

dalke wrote: post #15

Earlier quoted context omitted.

While it's certainly possible, in an abstract sense, what is a "chunk"? Who gets to decide? How do we resolve disagreements? There is a vast set of reasons for why something is cited. "This argument fails if reference X is wrong" is different from "I cite X because my professor/grant reviewer wrote the paper" vs. "I don't want to repeat something I described in a previous paper" vs. "I used the same training set desc…

> Who gets to decide? The authors under a community guideline. Editors should get to edit as appropriate. > There is a vast set of reasons for why something is cited... Adding reasons as a label to the citation link in the dependency graph would be very useful.

You seem to be describing a high-maintenance search engine, if I'm not mistaken.

The thing is, to my knowledge most academics keep up to date on most everything that is happening in their field; they don't need this high level overview because they don't communicate solely through journals.

It's only if you don't have the view that it's useful; which is obviously a problem for PhDs, but I'd question whether it's worth the necessary annotation investment (hours per paper for good classification, I'd imagine).

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#33

Post by ThomPete » Mon, May 04, 2015, 9:10 PM UTC

Retric wrote: post #29

ThomPete wrote: post #21

Earlier quoted context omitted.

I think it's in fact very true and there is a very good reason what that is the case. Most expert know more and more about less and less because in order to understand something you need to dig ever deeper to understand the specifics. What is needs is expert generalists that are able to understand several fields well enough to see where knowledge from one area can lead to understanding in others and vice versa.

You don't really understand a field without digging deep. A great example is how much students are willing to trust surveys before and after they do a large one. Another is mice running mazes, the numbers may look nice in theory, but they can hide a lot of problems.

I totally agree but the question is what constitutes a field.

I once read about a whole series of areas where solutions to issues in one field happened from understanding from understanding another field. Unfortunately ver few people span over multiple fields and can call themselves experts.

So the questions is whether we could spread out the expertise to more in between studies and letting go of literature as the only way to collect knowledge might be a great first step.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#34

Post by dalke » Mon, May 04, 2015, 9:10 PM UTC

nopinsight wrote: post #24

dalke wrote: post #15

Earlier quoted context omitted.

While it's certainly possible, in an abstract sense, what is a "chunk"? Who gets to decide? How do we resolve disagreements? There is a vast set of reasons for why something is cited. "This argument fails if reference X is wrong" is different from "I cite X because my professor/grant reviewer wrote the paper" vs. "I don't want to repeat something I described in a previous paper" vs. "I used the same training set desc…

> Who gets to decide? The authors under a community guideline. Editors should get to edit as appropriate. > There is a vast set of reasons for why something is cited... Adding reasons as a label to the citation link in the dependency graph would be very useful.

You are being entirely too abstract.

Once you try to embody these abstract ideas into concrete form, you'll discover that it does not work. Or rather, the gains are limited compared to the cost of creating, curating, and maintaining the extra metadata, and the error rate will be high.

"a label to the citation link in the dependency graph ..."

To start with, here's the Citation Typing Ontology: http://www.essepuntato.it/lode/http://purl.org/spar/cito .

I then write a paper which says that approach X is generally three times faster than Y, and http://purl.org/spar/cito/citesAsEvidence paper ABC.

Now, paper ABC is the one I mentioned earlier, which omitted a data point because of a transcription error by the authors. This is not a serious error, but it is a mistake. So you write a paper which http://purl.org/spar/cito/corrects paper ABC.

How does this automated system of scientific dependencies supposed to figure out that my paper is not invalidated by your correction to paper ABC?

"... would be very useful"

It may well be useful, but it can't be used to give what the OP wants. My comments concern the infeasibility of the proposed system, not the feasibility or usability of other projects which are in the same general area.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#35

Post by justbecause » Mon, May 04, 2015, 9:13 PM UTC

I like the concept, however I'm afraid "Going Agile" is not a solution for science, it's just more jargon.

Would be nice to see science testing several different methods though. Also, it feels like some areas of discipline might work better with different workflows. For instance theoretical physics may need a different process than structural engineering or chemistry. I think it's also weird that we currently use very similar processes across disciplines when there are surely better tools that could be made specifically for each discipline with interoperability between disciples handled on another level.

The right tools for the job would hopefully lead to more science faster.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#36

Post by tormeh » Mon, May 04, 2015, 9:14 PM UTC

return0 wrote: post #28

xiaq wrote: post #12

Earlier quoted context omitted.

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

can you give an example of any problem in science (not engineering) that is definitely solved?

All that is known in the entire field of mathematics. But yeah, mostly just mathematics. I think sorting algorithms is a solved problem, but you can say that's just math as well.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#37

Post by tormeh » Mon, May 04, 2015, 9:15 PM UTC

nopinsight wrote: post #31

One possibility is to keep the current publishing model but add an additional "meta" layer in a central repository to indicate the relations between the contributions of each paper. The repository should be independent of an individual journal or academic field so that multidisciplinary collaboration would be easier and more visible. If designed well, cross-pollination of ideas between fields may multiply. Each node…

Sciencepedia, with hyperlinks.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#38

Post by jordigh » Mon, May 04, 2015, 9:18 PM UTC

dalke wrote: post #26

jordigh wrote: post #4

Earlier quoted context omitted.

> How does his hypothetical dependency graph capture the multitude of disprovable points in the average paper? How does it capture what the "pillars" are? How about bug trackers? Papers that are continuously revised as they're published? Scientific prestige based on how often your papers are cited and how active their bug trackers are? That's how we do it in programming. I think the analogies could be stretched to sc…

Could you be a bit more concrete as I am having a difficult time trying to understand how this would work? To start with, if there is no activity in the bug tracker, doesn't that indicate success in the paper, because it contained no problems? Or should I put in a few silly errors (misplaced commas, typos in the citation list, etc.) in order to increase the bug activity? You suggest that programming prestige is based…

Every paper has problems. Yes, even typos should be mentioned. Why not? They might trip up someone.

Perhaps instead of structuring the whole thing around papers, it should be something like Wikipedia and editing articles in it.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#39

Post by entee » Mon, May 04, 2015, 9:19 PM UTC

Speaking from the perspective of life science, I think the idea of the "article" and "review" as two separate complimentary parts of constructing knowledge actually makes a lot of sense.

An individual article is an argument, with evidence for how some process works. A review looks at all the papers, getting the more global view of "knowledge" the OP seems to want.

Perhaps in the spirit of the article we should have reviews that are ongoing, sort of like wiki pages for various research areas. These might capture a living, complete state of the field better than current review articles.

It's really unclear how this would be done though because:

1.) How fields and areas interact is constantly changing, how do you manage the merging and splitting of review articles? You end up with a confusing view of "what we know" once again. 2.) As others have noted, who gets to write these things? The cutting edge literature is full of big personalities in conflict with one another. It takes years to figure out who is actually right.

That said, at a minimum retractions should be much easier to track. I'll bet we'd save a whole bunch of wasted effort.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#40

Post by dluan » Mon, May 04, 2015, 9:20 PM UTC

ThomPete wrote: post #21

dluan wrote: post #16

Earlier quoted context omitted.

That's not necessarily true, and I'd be incredibly worried if it were true. I think it's more an artifact of the design of this version of 'science' as an institution. Freeman Dyson was a brilliant drop out mathematician who happened to meet Richard Feynman and demonstrated a proof for Feynman's work (which led him to winning a Nobel). He says, "I think it's almost true without exception if you want to win a Nobel Pr…

I think it's in fact very true and there is a very good reason what that is the case. Most expert know more and more about less and less because in order to understand something you need to dig ever deeper to understand the specifics. What is needs is expert generalists that are able to understand several fields well enough to see where knowledge from one area can lead to understanding in others and vice versa.

Expert generalists are precisely the thing that's missing, especially with how younger scientists are trained. It's harder to fund an expert generalist.

Freeman Dyson is a great example because at 89 he published the solution to the prisoner's dilemma, decades later than the original theoretical physics work.