Retire This Idea: Scientific Knowledge Structured as “Literature” • Hacker News Board

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#21

Post by ThomPete » Mon, May 04, 2015, 8:35 PM UTC

dluan wrote: post #16

return0 wrote: post #11

Earlier quoted context omitted.

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disp…

That's not necessarily true, and I'd be incredibly worried if it were true. I think it's more an artifact of the design of this version of 'science' as an institution. Freeman Dyson was a brilliant drop out mathematician who happened to meet Richard Feynman and demonstrated a proof for Feynman's work (which led him to winning a Nobel). He says, "I think it's almost true without exception if you want to win a Nobel Pr…

I think it's in fact very true and there is a very good reason what that is the case.

Most expert know more and more about less and less because in order to understand something you need to dig ever deeper to understand the specifics.

What is needs is expert generalists that are able to understand several fields well enough to see where knowledge from one area can lead to understanding in others and vice versa.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#22

Post by tikhonj » Mon, May 04, 2015, 8:36 PM UTC

Personally, what really gets me with the current publishing regime is how fashion-driven it is (at least in many parts of CS). Oftentimes, it feels like the most challenging aspect of producing publishable research at first is not rigor or novelty or writing a sufficiently readable complete paper but hitting something sufficiently "interesting"—largely a function of what happens to be popular at the moment.

Publishing has other issues, but they can be mitigated by releasing results in forms besides simple papers. (Again, I'm mostly talking about CS.) A lot of successful projects end up releasing blog posts, enhanced online resources, libraries or complete, living open source projects.

At least in CS, this is happening enough to be useful even if it is by no means universal. This even comes up in more theoretical fields—somebody recently pointed me to a website[1] on tree edit distance, which is a great example. (I really wish it had existed a few years ago when I needed the algorithm originally!)

[1]: http://www.inf.unibz.it/dis/projects/tree-edit-distance/

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#23

Post by gnaritas » Mon, May 04, 2015, 8:44 PM UTC

xiaq wrote: post #12

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

Sure, that's called engineering, and applies to far more than just technology. Engineering is the practical application of science.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#24

Post by nopinsight » Mon, May 04, 2015, 8:45 PM UTC

dalke wrote: post #15

nopinsight wrote: post #7

Earlier quoted context omitted.

Isn't it possible to consider each little 'chunk' of knowledge, data, information as a node in the dependency graph, as opposed to one whole paper? We could also have a hierarchy to represent layers and composition of knowledge. Not to say it is easy, but it looks quite possible.

While it's certainly possible, in an abstract sense, what is a "chunk"? Who gets to decide? How do we resolve disagreements? There is a vast set of reasons for why something is cited. "This argument fails if reference X is wrong" is different from "I cite X because my professor/grant reviewer wrote the paper" vs. "I don't want to repeat something I described in a previous paper" vs. "I used the same training set desc…

> Who gets to decide?

The authors under a community guideline. Editors should get to edit as appropriate.

> There is a vast set of reasons for why something is cited...

Adding reasons as a label to the citation link in the dependency graph would be very useful.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#25

Post by gnaritas » Mon, May 04, 2015, 8:46 PM UTC

dalke wrote: post #19

xiaq wrote: post #12

Earlier quoted context omitted.

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

When Darwin "solved" the question of evolution, he did not solve it for good. There has been a lot of work to make it a better, deeper, and more powerful mechanism for understanding biology. This is true for just about all research.

Science doesn't solve problems in that sense, Darwin didn't solve the evolution question, he proposed a successful theory to explain it and that core idea remains true though modern biologists have gone much further than Darwin ever could as his time was before DNA. To use the word solved just wouldn't be correct in any sense of the word.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#26

Post by dalke » Mon, May 04, 2015, 8:51 PM UTC

jordigh wrote: post #4

dalke wrote: post #2

While not essential to the argument, it uses software development as an analogy for scientific publications. Unfortunately, that analogy isn't correct. > Just as the software industry has moved from a "waterfall" process to an "agile" process—from monolithic releases shipped from warehouses of mass-produced disks to over-the-air differential updates—so must academic publishing move from its current read-only model an…

> How does his hypothetical dependency graph capture the multitude of disprovable points in the average paper? How does it capture what the "pillars" are? How about bug trackers? Papers that are continuously revised as they're published? Scientific prestige based on how often your papers are cited and how active their bug trackers are? That's how we do it in programming. I think the analogies could be stretched to sc…

Could you be a bit more concrete as I am having a difficult time trying to understand how this would work?

To start with, if there is no activity in the bug tracker, doesn't that indicate success in the paper, because it contained no problems? Or should I put in a few silly errors (misplaced commas, typos in the citation list, etc.) in order to increase the bug activity?

You suggest that programming prestige is based on the activity of the bug tracker. Can you give some examples? As an admittedly extreme case, TeX has has very few bugs, and has no bug tracker, but is a well known project.

Bug trackers only work for active projects. If the main author writes a paper in solid state physics, is awarded a PhD, and gets a job working for Seagate on high density magnetic compounds for storage tapes, then why should the author care about maintaining the paper's bug tracker?

But getting back to the topic, the dependency graph for programming is extremely coarse grained. For example, here is a recent bug for Python - "integer overflow in itertools.permutations" - http://bugs.python.org/issue23363 .

Given your software dependency graph, can you tell me which programs that depend on Python are affected by that bug?

Because that's the sort of thing that Brian Christian (the author of this piece) wants for scientific publications:

> A dependency graph would tell us, at a click, which of the pillars of scientific theory are truly load-bearing. And it would tell us, at a click, which other ideas are likely to get swept away with the rubble of a particular theory. An academic publisher worth their salt would, for instance, not only be able to flag articles that have been retracted—that this is not currently standard practice is, again, inexcusable—but would be able to flag articles that depend in some meaningful way on the results of retracted work.

How do I indicate "in some meaningful" that my program depends on itertools.permutations() not having an integer overflow?

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#27

Post by return0 » Mon, May 04, 2015, 8:52 PM UTC

dalke wrote: post #17

return0 wrote: post #11

Earlier quoted context omitted.

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disp…

What was Einstein's single idea that he worked on? The photoelectric effect, for which he won the Nobel prize, or special and general relativity? Or statistical mechanics like Bose-Einstein statistics? Because it seems to me like he worked on many ideas during his life.

the nature of spacetime and gravitation seem to have followed him to his death.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#28

Post by return0 » Mon, May 04, 2015, 8:53 PM UTC

xiaq wrote: post #12

jgmjgm wrote: post #5

I what is missing from the article is an understanding of the incentives of academics. Quite often putting a piece of research "to bed" is the goal. Why? Because any good academic has a long list of things they want to get to. Very few people want to get mired in the (inevitable) problems that arise in a paper for eternity. This is why there are review papers that summarize the state of the knowledge at a given point…

Exactly. We often have the idea of "solved problem" in science, but very few of them in software development (overgeneralizing a bit, any field of technology). This is probably because science is often about "what" (what is the fastest algorithm for matrix multiplication?), "whether" (does P = NP?), while technology is often about "how" (how to implement matrix multiplication efficiently?). Once you solve a problem i…

can you give an example of any problem in science (not engineering) that is definitely solved?

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#29

Post by Retric » Mon, May 04, 2015, 8:57 PM UTC

ThomPete wrote: post #21

dluan wrote: post #16

Earlier quoted context omitted.

That's not necessarily true, and I'd be incredibly worried if it were true. I think it's more an artifact of the design of this version of 'science' as an institution. Freeman Dyson was a brilliant drop out mathematician who happened to meet Richard Feynman and demonstrated a proof for Feynman's work (which led him to winning a Nobel). He says, "I think it's almost true without exception if you want to win a Nobel Pr…

I think it's in fact very true and there is a very good reason what that is the case. Most expert know more and more about less and less because in order to understand something you need to dig ever deeper to understand the specifics. What is needs is expert generalists that are able to understand several fields well enough to see where knowledge from one area can lead to understanding in others and vice versa.

You don't really understand a field without digging deep. A great example is how much students are willing to trust surveys before and after they do a large one. Another is mice running mazes, the numbers may look nice in theory, but they can hide a lot of problems.

Re: Retire This Idea: Scientific Knowledge Structured as “Literature”

#30

Post by return0 » Mon, May 04, 2015, 9:01 PM UTC

dluan wrote: post #16

return0 wrote: post #11

Earlier quoted context omitted.

Most of the great scientists worked on a single idea their entire life and expanded upon it, improved it, corrected it and were never really done with it (einstein comes to mind). Scientific knowledge is never final I personally have a huge beef with the way life scientists publish their results in tiny tiny bits, that makes it extremely hard to cross-check them with other studies, to find out if they were later disp…

That's not necessarily true, and I'd be incredibly worried if it were true. I think it's more an artifact of the design of this version of 'science' as an institution. Freeman Dyson was a brilliant drop out mathematician who happened to meet Richard Feynman and demonstrated a proof for Feynman's work (which led him to winning a Nobel). He says, "I think it's almost true without exception if you want to win a Nobel Pr…

Maybe it worked for him; I have found that people who tend to do the most important work in my field are dedicated and believe in their work, instead of perpetually looking for something new but unimportant to publish.