Retire This Idea: Scientific Knowledge Structured as “Literature” • Hacker News Board

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#11

Post by return0 » Mon, May 04, 2015, 7:54 PM UTC

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final

I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disproved or even to figure out the average of some quantity. Try making a concrete model of something and you will find yourself searching for experimentally measured values in tomes among the blabber of introductions and discussions, like monks did in the middle ages... Databases sometimes exist, but they are not mandatory so you never know if they are to be trusted. We need a structured way to catalog scientific results , and journals are not it (esp. with politics which lead scientists to publish in marginally related journals ).

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#12

Post by xiaq » Mon, May 04, 2015, 7:55 PM UTC

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Exactly.

We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?).

Once you solve a problem in science, you solve it for good. But in technology there can always be "better" ways to solve it, and things keep evolving.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#13

Post by dluan » Mon, May 04, 2015, 7:55 PM UTC

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

I think that is true, which is why what this author should be suggesting is changing the incentives of what the 'output' of a scientist should be.

Literature is great because it measures how far you're progressing the measuring stake of knowledge in a field. Old distribution models like journals just help disseminate that "hey everyone, here's the new mile marker". You then get paid based on how far you manage to advance it.

I think while that should remain true, it can definitely use some rebalancing. It's incredibly lopsided and inefficient, and too many nodes in the process are hoarded and centralized. In this case, I think science should more follow art.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#14

Post by mhuffman » Mon, May 04, 2015, 7:59 PM UTC

There has also been a big push to only count "recent" scientific literature as valid for reference in some scientific fields. I know that this helps keep any new papers "fresh", but I have always had a bitter aftertaste about the way that still-valid, but older, research is looked down upon.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#15

Post by dalke » Mon, May 04, 2015, 7:59 PM UTC

nopinsight wrote: post #7

dalke wrote: post #2

While not essential to the argument, it uses software development as an analogy for scientific publications. Unfortunately, that analogy isn't correct. > Just as the software industry has moved from a "waterfall" process to an "agile" process—from monolithic releases shipped from warehouses of mass-produced disks to over-the-air differential updates—so must academic publishing move from its current read-only model an…

Isn't it possible to consider each little 'chunk' of knowledge, data, information as a node in the dependency graph, as opposed to one whole paper? We could also have a hierarchy to represent layers and composition of knowledge. Not to say it is easy, but it looks quite possible.

While it's certainly possible, in an abstract sense, what is a "chunk"? Who gets to decide? How do we resolve disagreements?

There is a vast set of reasons for why something is cited. "This argument fails if reference X is wrong" is different from "I cite X because my professor/grant reviewer wrote the paper" vs. "I don't want to repeat something I described in a previous paper" vs. "I used the same training set described in X but for different purposes" vs ...

Or, suppose I write a paper which analyzes the evolution of how the term "gene" is used over the 1900s. It's unlikely that every detail that might be analyzed will be dissected into its individual components. And if it were, that's an awful lot of work, with little chance automation support (especially if some of the reasons, like promoting one's earlier work, are gauche to state explicitly).

These the concrete problems that make it really hard to convert the "possible to consider" into something practical.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#16

Post by dluan » Mon, May 04, 2015, 8:01 PM UTC

return0 wrote: post #11

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disp…

That's not necessarily true, and I'd be incredibly worried if it were true. I think it's more an artifact of the design of this version of 'science' as an institution.

Freeman Dyson was a brilliant drop out mathematician who happened to meet Richard Feynman and demonstrated a proof for Feynman's work (which led him to winning a Nobel). He says,

"I think it's almost true without exception if you want to win a Nobel Prize, you should have a long attention span, get hold of some deep and important problem and stay with it for ten years. That wasn't my style."

Here's a list of Dyson's awards: Heineman Prize (1965), Lorentz Medal (1966), Hughes Medal (1968), Harvey Prize (1977), Wolf Prize (1981), Andrew Gemant Award (1988), Matteucci Medal (1989), Oersted Medal (1991), Fermi Award (1993), Templeton Prize (2000), Pomeranchuk Prize (2003), Poincaré Prize (2012).

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#17

Post by dalke » Mon, May 04, 2015, 8:04 PM UTC

return0 wrote: post #11

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disp…

What was Einstein's single idea that he worked on? The photoelectric effect, for which he won the Nobel prize, or special and general relativity? Or statistical mechanics like Bose-Einstein statistics? Because it seems to me like he worked on many ideas during his life.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#18

Post by [deleted] » Mon, May 04, 2015, 8:06 PM UTC

xiaq wrote: post #12

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

[deleted]

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#19

Post by dalke » Mon, May 04, 2015, 8:19 PM UTC

xiaq wrote: post #12

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

When Darwin "solved" the question of evolution, he did not solve it for good. There has been a lot of work to make it a better, deeper, and more powerful mechanism for understanding biology.

This is true for just about all research.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#20

Post by dalke » Mon, May 04, 2015, 8:26 PM UTC

xiaq wrote: post #8

Cool ideas. We will probably end up with a better world if someone implements them, but I would argue that these "agile" things are much less important in scientific publications than in software development. Science and technology are different. Software artifacts are huge and uncontrollable. Today it is very rare for a software developer to know all the ins and outs of a software he/she develops, along with all its…

What about scientific papers that depend on software artifacts?

Either scientists somehow understands all the "ins and outs of a software he/she develops, along with all its dependencies" in order to publish a paper, where a programmer doesn't have the same understanding, or you are comparing two entirely different concepts as if they were the same.