Live data from Hacker News

Ask HN: What can I do to accelerate scientific research?

news.ycombinator.com

161–170 of 179 posts

Re: Ask HN: What can I do to accelerate scientific research?

#161
post #126

Earlier quoted context omitted.

>develop a paper reference system that actually works well. Mendeley is the best there is and it’s (IMO) rubbish. Poorly designed. Zotero works wonderfully for me. And it's open, which is a must.

I'll preface this by clearly stating that Zotero is absolutely indispensable; I wouldn't be nearly as organized without it. It's very important to me that such tools be open, and Mendeley in particular is a complete disaster in this regard (see their history with encrypting user data). That being said, Zotero is very much a "least worst" tool in my opinion. * Overly rigid in how it goes about modeling document types…

Why don’t data versioning tools like git lfs do the job? Is it lack of awareness or is the problem more complex than that?

Re: Ask HN: What can I do to accelerate scientific research?

#162

I think there's lot of room to improve tools in bioinformatics. In practice bioinformatics pipelines tend to be bundles of loosely organised python scripts and I've heard files on the order of a few GB described as Big Data because the processing times are so slow (days for stuff that could take milliseconds) It would help to pair up with practicing scientists and explore what parts of their workflow can be improved

Definitely. A lot of work is done in R or python and could be sped up in a compiled language.

Yes. I’m not a scientist, but I think if there were good compiled python/R alternatives, the scientific world would benefit greatly, if not only for the reduced waiting times... Maybe a language with Go simplicity and speed and Python ease of use and appeal... It should have an almost real-time compile mode (with very little optimization) to enable interactive playgrounds, like Jupyter. Of course it’d need also strong optimization modes for final production code.

Re: Ask HN: What can I do to accelerate scientific research?

#163
A lot of journals (including the one I am an editor at) would probably consider leaving the fangs of elsevier and co and go full open access publishing. However, there are a few obstacles to that, such as missing open and reliable editorial and publishing platforms, contacts to libraries, etc.

In my opinion providing these things and actively encouraging journals to make the jump to open access would be a huge service to academia. Also, I wouldn’t be surprised if one could find funding for such a venture.

Re: Ask HN: What can I do to accelerate scientific research?

#164
post #16

This will sound harsh, but it isn’t meant to be. Scientists code better then you do science. This is simply a consequence Of a weeding out mechanism for those that have no coding skills. The only ppl who get away with no coding skills are important professor with grad students to do the coding. This isn’t to say that our skills are great, but a generic programmers (I.e. CS majors) science abilities are approximately…

> matplotlib And now a problem that would be great to solve is having vectored images and being able to change my fonts on size, lines are too small, I changed the paper format, whatever. I can't express how often I've had to redo plots simply because they don't look right in a paper. I also cannot express the beauty of LaTeX but the absolute horror it is to create Tikz images. They are beautiful but it is definitely…

Check out matplotlib's PGF backend: https://matplotlib.org/users/pgf.html

Re: Ask HN: What can I do to accelerate scientific research?

#165

> indexing all open research with free unlimited access, similar to arxiv-sanity.com but better This space is pretty crowded, in my opinion. I don’t know much about biology, but I can tell you that in materials, it’s all about data. The materials design problem and predicting new materials comes down to knowing properties of other materials. A lot of progress has been made by using datasets generated by quantum mecha…

https://citrine.io/ You might find this company interesting.

I actually know and work with several people there :)

Re: Ask HN: What can I do to accelerate scientific research?

#166
post #164

Earlier quoted context omitted.

> matplotlib And now a problem that would be great to solve is having vectored images and being able to change my fonts on size, lines are too small, I changed the paper format, whatever. I can't express how often I've had to redo plots simply because they don't look right in a paper. I also cannot express the beauty of LaTeX but the absolute horror it is to create Tikz images. They are beautiful but it is definitely…

Check out matplotlib's PGF backend: https://matplotlib.org/users/pgf.html

This is pretty nice and I will likely be using it from now on. Thanks.

But I do want something a little more native to latex. The major issue is that sometimes font sizes, axes, titles, even plot thickness doesn't look right in a paper. The issue is when you have a large plot and have to replot to fix these things. But vectored images will help.

Re: Ask HN: What can I do to accelerate scientific research?

#167
post #16

This will sound harsh, but it isn’t meant to be. Scientists code better then you do science. This is simply a consequence Of a weeding out mechanism for those that have no coding skills. The only ppl who get away with no coding skills are important professor with grad students to do the coding. This isn’t to say that our skills are great, but a generic programmers (I.e. CS majors) science abilities are approximately…

I actually strongly disagree with the message in the lead-in. All scientists can “code” better than OP can do science, yes, but most woefully lack any software engineering experience. Most postdoc research code would earn a failing grade from an undergraduate software engineering professor, or get themselves fired from a real world programming job. Think spaghetti code, lack of testing or continuous integration, non…

> or get themselves fired from a real world programming job. Think spaghetti code, lack of testing or continuous integration, non compliance with standards, etc.

Perhaps, but doing all those things would get them fired from the job they are currently in.

Re: Ask HN: What can I do to accelerate scientific research?

#168
post #25

Find any way to make a lot of money, gain a lot of political power or gain influence over those who have money or power and use it to fund research and make it more appealing culturally.

No, the most significant advances in science were done on a shoestring budget (or no budget at all) - just by THINKING. It doesn't require lots of money to support people who do that kind of work, and they are easy to find, especially in theoretical physics and mathematics.

Maybe it was true a hundred years ago, but it is not true today, unfortunately. Science became much more complicated.

https://news.ycombinator.com/item?id=20190468

Re: Ask HN: What can I do to accelerate scientific research?

#169
post #45
post #16

This will sound harsh, but it isn’t meant to be. Scientists code better then you do science. This is simply a consequence Of a weeding out mechanism for those that have no coding skills. The only ppl who get away with no coding skills are important professor with grad students to do the coding. This isn’t to say that our skills are great, but a generic programmers (I.e. CS majors) science abilities are approximately…

> Science needs models based on mechanistic understanding of the underlying phenomena. A model that merely predicts is useful for engineers, not scientists I'm not sure I agree. I'm aware of quite a bit of supercomputing time that is spent doing lattice QCD calculations (which apparently some scientists find useful), and though I'm no quantum physicist I'm pretty sure there is not much of a "mechanistic understanding…

I guess "Mechanistic understanding" was meant to contrast against machine learning, not quantum mechanics. To elaborate, machine learning means fitting of a bunch of data by a given model. In science (eg lattice QCD) one often tries to theoretically (or computationally) explore regimes where data is not yet available. As a (former?) theoretical physicist, I am more than happy to admit that this is not immediately useful, though it will hopefully become useful in the long run.

Re: Ask HN: What can I do to accelerate scientific research?

#170
post #161
post #126

Earlier quoted context omitted.

I'll preface this by clearly stating that Zotero is absolutely indispensable; I wouldn't be nearly as organized without it. It's very important to me that such tools be open, and Mendeley in particular is a complete disaster in this regard (see their history with encrypting user data). That being said, Zotero is very much a "least worst" tool in my opinion. * Overly rigid in how it goes about modeling document types…

Why don’t data versioning tools like git lfs do the job? Is it lack of awareness or is the problem more complex than that?

Well I could just be unaware of some functionality they have, but all those tools do is version things. There's no integration with reference managers like Zotero (that I'm aware of), and no tracking of interrelations or metadata.

In contrast, Zotero (and other reference managers) don't do any versioning at all (at least that I'm aware of). Instead, they keep track of the metadata that's necessary to put together a works cited section for an academic paper.

... or at least that's what they started out doing. These days they also try to organize your papers into some sort of category structure, facilitate tagging and notes, provide synchronization between your devices, and probably a few other things that don't come to mind right now.

Feature creep? Sure, but all that stuff is central to the research and writing process. It's also all tightly coupled, so splitting it between multiple tools doesn't work very well. And that's the current problem - how to integrate, for example, a few of your browser bookmarks with your academic literature collection. Or how to track a list of all the papers cited by a particular paper. Or link a specific paper tracked by your reference management software against a specific version of a large data set, perhaps itself tracked by Git LFS.

Generalizing a bit, what about linking experimental notes (typically pen and paper) with data collection software (typically a binary), as well as the collected data (perhaps Git LFS), as well as a specific version of some data analysis scripts you wrote (perhaps Git). Now try to track everything as you work on multiple paper revisions with collaborators, each version of which adds (and sometimes removes) citations and could use a different (likely newer) revision of the collection software, data set, or analysis scripts.

Alternatively, for a data management scenario not directly involving writing papers consider molecular cloning using plasmids. You have a dozen semi-related tubes in a cryogenic freezer that you need to track over many years (ie long term inventory management), each of which has one or more pieces of sequencing data attached to it (so a small data set), they're all interrelated (you create a new one by physically modifying an old one), and each has the typical meta-links to experimental protocols, notes, academic literature, and other things.

I'm not aware of any software solutions that comprehensively address all of this stuff, so people still use pen and paper. But pen and paper is time consuming, it's error prone, it doesn't sync between devices, it's slow and tedious to cross reference - all the typical problems that software is good at addressing.

Post reply on HN