Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Too much of the research process is now shrouded by the opaque use of computers

theconversation.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Too much of the research process is now shrouded by the opaque use of computers

#31

Post by sparkie »

davexunit wrote: post #17

GNU Guix is starting to be used for reproducible science in the bioinformatics industry. Much better than bundling opaque binary VM/container images or just having some scripts that bitrot.

It's funny how the reaction of most people in this thread is to bash on 'the other sciences' for bad coding practice, while completely ignoring how inadequate their own practices are in creating reproducible programs.

The heart of the problem is described in the article: It's point and click interfaces (and yes, this includes regurgitating out commands into the terminal), which are expected to be followed by the dot, where they could be automated by the machine if the program were ever to be completed to be reproducible.

A big problem is a non-computer scientist is probably working in a lab on some ancient machine running ancient software - he has no control over the machine, and the system-admin is so far behind because his main job is to reproduce the (unreproducible) software written by so called 'computer scientists'. He struggles so much that he has to share his work with thousands of others as a 'package maintainer', and is grateful that so many other package maintainers exists, because without eachother, they would all have absolutely zero chance of reproducing anything.

It's time to stop bashing the coding practices of other people folks, and look in the mirror. We are the friction that causes code in scientific research to be unreproducible - it's not the code itself. If we're to educate non-computer scientists in how to create reproducible research, surely the absolute minimum is that we do so ourselves.

And so far, Nix and Guix are the only two projects (afaik) which are seriously attempting to tackle this. If you call yourself a 'computer scientist', and you regularly write research (which all code is), then start living up to the name and make it reproducible. This means you should be using Nix or Guix, and packaging your software for it. Without such tool to reproduce the software, you're suffering from the same reproducibility problems this article is highlighting about the other sciences.

Re: Too much of the research process is now shrouded by the opaque use of computers

#32

Post by sklogic »

I remember some time ago somebody proposed that scientists should supply entire virtual machines with all the relevant setup and data for a better reproducibility.

It was before the dawn of Docker and alike, so now this idea is probably a bit less insane.

Re: Too much of the research process is now shrouded by the opaque use of computers

#33

Post by vtlynch »

grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

>In my experience, scientists are like extremely junior developers with PhDs.

So from your experience, you are assuming every scientist disdains coding and does it poorly, and every programmer is a master at writing clean and working code.

Perhaps a scientist can teach you about the follies of anecdotal evidence.

Re: Too much of the research process is now shrouded by the opaque use of computers

#34

Post by davexunit »

sparkie wrote: post #31
davexunit wrote: post #17

GNU Guix is starting to be used for reproducible science in the bioinformatics industry. Much better than bundling opaque binary VM/container images or just having some scripts that bitrot.

It's funny how the reaction of most people in this thread is to bash on 'the other sciences' for bad coding practice, while completely ignoring how inadequate their own practices are in creating reproducible programs. The heart of the problem is described in the article: It's point and click interfaces (and yes, this includes regurgitating out commands into the terminal), which are expected to be followed by the dot,…

Amen!

Re: Too much of the research process is now shrouded by the opaque use of computers

#36

Post by p4wnc6 »

kjdal2001 wrote: post #23
grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

I have had a similar experience working with financial engineers and traders turned programmers (I am a programmer). They don't seem to value software the same way as programmers. They seem to view their ideas as the most important aspect of their work, and writing software is only a way of expressing their ideas. The problem with this is that if you don't feel that their are non purely functional requirements worthy…

The other problem with it is that they don't appreciate that probably more than 90% of all code (including the code that touches their ideas) is for unsexy reporting purposes only. The domain expertise accounts for probably 1% to 5% of any business's code, and generally it's the easiest code to design, test, and change later on because it follows very tightly with well-worn practices in scientific computing. You might invent a great new trading strategy, but implementing it is still just the same old vector math and efficient algorithms stuff everyone's been doing for decades.

But with the logging, reporting, parameter management, data provenance, data resource management, etc., etc., the actual creative design of the code matters much more, and a failure to make it extensible is way more expensive to fix down the road than tweaking some O(n^2) scientific algorithm.

As an aside, this is one reason that I think algorithmic brain teasers are a really stupid thing for interviews or predicting positive business impact. With brainteasers, you are testing for rote memorization of a set of optimization techniques (data structure implementations) that maybe touches 1%-5% of your easiest-to-optimize-from-textbooks-or-wikipedia code, whereas the other 95% can almost always be assumed to rely on library implementations of these kinds of structures, and the high-level design of components, particularly optimizing it so that the addition or refactoring of features later is not expensive, is the dominant concern for adding actual business value.

It's just another status game like everything else. Programming labor is asserted to be a commodity item to the company, so that managers can justify lower wage, worse working conditions, etc., for programmers than for business domain experts who are supposedly less easily replaced. But the reality is that a reasonably smart programmer can probably pick up the domain expertise to a high level, even to the same level as a Ph.D. researcher, in a short time, whereas for some reason because of some kind of status-based mental block or lack of curiosity, the domain experts seem incapable of picking up legitimate programming skills.

When most of the code is for reporting, designing the reporting system is a huge priority. But when domain expertise carries political status, the importance of system design is often neglected in favor of campaigning for the supposed critical importance of the favored domain area.

Re: Too much of the research process is now shrouded by the opaque use of computers

#37

Post by jarofgreen »

Very interesting, and we were talking about the exact same process at an Open Data event this weekend - documenting the data processing work people do so others can reproduce and build on it.

One thing that is not mentioned at all in this as far as I can see is what about personal data? When the experiment handles personal data - a medical trial is the most likely candidate - the details shouldn't be published sometimes, only anonymised aggregate tables. Are there any guidelines for handling that?

Re: Too much of the research process is now shrouded by the opaque use of computers

#39

Post by ubasu »

I don't disagree that badly-written scientific code exists (both in academia and in the industry), but I want to point out a couple of things for people who have only a software engineering background.

1. Keep in mind that most scientific code for numerical analysis, being based in mathematics, follows the convention for using algebraic symbols, e.g. even in physics, we write

F=m*a

instead of the wordy version, which we overcame a couple of centuries ago. So using shortened variable names [1] comes from that background, and using longer names as in Java-world seems like a regression.

2. Writing code like [2]

    Ax_1 += Bx_1 + Cx_1 # Add Bx_1 to Cx_1
    Ax_2 += Bx_2 + Cx_2 # Same thing, but for Ax_2
    Ax_3 += Bx_3 + Cx_3 # "
    Ay_1 += By_1 + Cy_1 # "
is known as loop unrolling [3,4] and is used for optimization.

[1] https://news.ycombinator.com/item?id=10539078

[2] https://news.ycombinator.com/item?id=10540446

[3] https://en.wikipedia.org/wiki/Loop_unrolling

[4] http://stackoverflow.com/questions/2349211/when-if-ever-is-l...

Re: Too much of the research process is now shrouded by the opaque use of computers

#40

Post by chrisseaton »

ubasu wrote: post #39

I don't disagree that badly-written scientific code exists (both in academia and in the industry), but I want to point out a couple of things for people who have only a software engineering background. 1. Keep in mind that most scientific code for numerical analysis, being based in mathematics, follows the convention for using algebraic symbols, e.g. even in physics, we write F=m*a instead of the wordy version, which…

Why isn't your compiler unrolling your loops for you?
Post reply on HN