Earlier quoted context omitted.
Frequently a lot of this hoarded data is flawed or defective due to improper setup or execution of the experiment. That isn't to say the information in this "dark data" is useless, but it needs to be taken in context. The cleanest data with the best results are put forward into a paper; the chaff is not.
Shouldn't this be noted? Isn't providing the "best" data risking making data fit your hypothesis?
Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
11–20 of 62 posts
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#12Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#13Earlier quoted context omitted.
Shouldn't this be noted? Isn't providing the "best" data risking making data fit your hypothesis?
Not if your dark data is "I forgot to autoclave an instrument and contaminated my samples." In that case without an unexpected positive result it's just error and not worth reporting.
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#14Earlier quoted context omitted.
A lot of data is hoarded because scientists often have to compete for limited research funding (at least from what I've seen in the US).
next up: "let's depend funding on how much data is shared and other people use it" after that: "researchers overshare data and use each others data for no reason but to bump numbers" it's interesting how every system tends to sooner or later be gamified by its players
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#15Earlier quoted context omitted.
A lot of data is hoarded because scientists often have to compete for limited research funding (at least from what I've seen in the US).
next up: "let's depend funding on how much data is shared and other people use it" after that: "researchers overshare data and use each others data for no reason but to bump numbers" it's interesting how every system tends to sooner or later be gamified by its players
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#16Earlier quoted context omitted.
Not if your dark data is "I forgot to autoclave an instrument and contaminated my samples." In that case without an unexpected positive result it's just error and not worth reporting.
biology is not my forte but- i imagine "dark data" (with context) is always more valuable than no data.
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#17Earlier quoted context omitted.
Not if your dark data is "I forgot to autoclave an instrument and contaminated my samples." In that case without an unexpected positive result it's just error and not worth reporting.
biology is not my forte but- i imagine "dark data" (with context) is always more valuable than no data.
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#18Could someone please offer a TLDR; The article seems very clickbait and full of teases.
How this "Ayasdi" company's analysis probably works (based on "Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival" and the original "Mapper" paper "Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition"): They take point cloud data and connect each point with its neighbors (the distance metric that is used is probably domain-specific) to build a proximity graph that approximates a simplicial complex. As input to their algorithm, they also have one or more scalar functions defined on the point cloud data that contain information which is relative to the problem at hand. For example, each point could be a gene, and maybe the scalar function value at that gene could be probability of association with some disease, and the distance between two genes might be the Levenshtein distance between their genetic codes.
With data in this form, they approximate the Reeb graph of one of the scalar functions, which is a sort of "data skeleton." They can do potentially interesting/useful things with it.
The approximation of the Reeb graph reveals zero-cycles (connected components of the simplicial complex) and some one-cycles (handles/tunnels in the graph, sort of like holes in a donut). This "skeleton" of the data allows them to do a variety of things, such as segment the data into components that are (approximately) topologically "simple" (they do not contain any 1-cycles), identify local maxima/minima, find saddle points where forks in the data merge together, and locate "essential saddles" which constitute the high points and low points of handles/tunnels. They can also remove "topological noise", which helps them to separate spurious topological thingies from features that might be important.
Their technique doesn't necessarily recover "true" topological information since a lot of what they do is approximate. There are actually more accurate techniques (e.g., simplicial homology, or fast Reeb graph algorithms) for getting an exact answer, albeit with potentially higher computational cost.
Topological data analysis is a big field, and this Ayasdi company appears to mainly use this one approach (but I could be wrong). I think they are trying to lay claim to the term "topological data analysis" and get people with money excited about it.
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#19Could someone please offer a TLDR; The article seems very clickbait and full of teases.
Re: Medical Breakthrough in Spinal Cord Injuries Was Made by a Computer Program
#20Earlier quoted context omitted.
A lot of data is hoarded because scientists often have to compete for limited research funding (at least from what I've seen in the US).
next up: "let's depend funding on how much data is shared and other people use it" after that: "researchers overshare data and use each others data for no reason but to bump numbers" it's interesting how every system tends to sooner or later be gamified by its players