Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Too much of the research process is now shrouded by the opaque use of computers

theconversation.com

Post reply on HN
Print view View original

11–20 of 55 posts

Re: Too much of the research process is now shrouded by the opaque use of computers

#11

Post by guard-of-terra »

amelius wrote: post #7
guard-of-terra wrote: post #2

The solution is obvious - provide a github repository with all the code for the research. Provide clear ways to reproduce all the results of the research from raw data by running build. Bonus points: Hirschware as software license.

But if somebody would read the publication in, say, 20 years, then the whole platform on which the computation has to run might have become obsolete. Suddenly, you have to look for emulators. Or even for emulators running emulators.

With some effort you could make it work, perhaps make a pull request. Programming languages will likely still be around.

Using open formats and tools would likely help.

Re: Too much of the research process is now shrouded by the opaque use of computers

#12

Post by mbostleman »

woah wrote: post #10
grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

I don't think that a startup can solve their coding problems. Only scientists can solve their own coding problems by having the diligence and attention to detail that one would expect of a high school dropout with 2 weeks of a coding bootcamp under their belt.

Exactly. The root problem is that they're just not scientists.

Re: Too much of the research process is now shrouded by the opaque use of computers

#13

Post by grhmc »

woah wrote: post #10
grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

I don't think that a startup can solve their coding problems. Only scientists can solve their own coding problems by having the diligence and attention to detail that one would expect of a high school dropout with 2 weeks of a coding bootcamp under their belt.

What I mean by that is:

The startup I work at employees scientists, therefore the scientists are part of the startup, and it is our collective responsibility to create an atmosphere of good science.

Creating that atmosphere, in the face of years of habits and history, is a very difficult problem for a startup to solve.

Re: Too much of the research process is now shrouded by the opaque use of computers

#14

Post by maliker »

Amen!

I work in electric utility research. Pretty much every paper gives some simulation results. Getting the source? The only option is emailing and hoping the researchers respond, want to share, and have code that runs on something other than laptop of the student that built it. I haven't ever succeeded in getting working source code.

There are many simple improvements that could be done. Just a github repo for each paper would be a monumental step forward. Installation instructions that have been tested and proven to work would be valuable. And then good coding practices, smart reuse of existing packages and integration with other projects would be beyond awesome (and pretty hard).

Re: Too much of the research process is now shrouded by the opaque use of computers

#15

Post by nmrm2 »

mbostleman wrote: post #12
woah wrote: post #10

Earlier quoted context omitted.

I don't think that a startup can solve their coding problems. Only scientists can solve their own coding problems by having the diligence and attention to detail that one would expect of a high school dropout with 2 weeks of a coding bootcamp under their belt.

Exactly. The root problem is that they're just not scientists.

"The code is bad therefore it's not science" is a questionable and even arrogant attitude.

In more cases than not, the software is doing something that was previously done by hand by some grad student or support staff(1). If those people made a mistake, then tough luck. Hopefully you'd catch it with a sanity check on the output. Were those people who relied on manual labor for centuries before the invention of modern computers not doing science? Absurd.

IMO, what's happening is that scientists are doing the same sort of validation that they always did -- placing no trust in the process (the code), and instead looking at checkpoints in the computation process and sanity checking the final output.

None of this is to say that there isn't room for improvement -- there is. But it's possible to do great science while writing shitty code. Hell, it's possible to do great Computer Science while writing shitty code!

(1) The vast majority of scientists programming are not, in fact, doing anything revolutionary with computation. It's just a more efficient way of doing what they used to do by hand. And those that are using code in an essential way typically (not always) have higher quality code.

Re: Too much of the research process is now shrouded by the opaque use of computers

#16

Post by cryoshon »

grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

This is accurate. I am a good example of this phenomenon. Scientists don't know how to code in a way that is intelligible to someone else, and in academic shops, there's probably only going to be one person who writes/understands a program in the first place. This one person probably views coding purely as an icky means to an end, and his adviser thinks it's a waste of time if it even reaches his knowledge whatsoever. As soon as this person moves on, the software he was working on will be rendered nonfunctional.

Part of it is complete ignorance of standards and best practices; another part is a complete lack of documentation (too much time spent on non-science activity to justify making it). It's a problem because scientists can always make use of information technology, but they don't quite have the focus to prioritize it. Honestly, passing it off to non-science devs hasn't worked well in my experience, there's a knowledge gap that's really hard to get over.

Re: Too much of the research process is now shrouded by the opaque use of computers

#18

Post by gilleain »

grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

A little harsh, but yes in general scientists don't write great code or use the right tooling. The reason is simple : the purpose of science is not to write code but to write papers.

That's not _at all_ to say this state of affairs is a good thing, but the practical reality is that scientists are judeged by academic papers, not code.

So, time taken learning how to code is time taken away from learning to write papers. Time taken designing code, testing it, documenting it, is time taken away from doing the research that goes into the paper. And so on.

Furthermore, most scientific code is actually scripting built on underlying frameworks. For example, plugging together a bunch of analytic tools to process some data. Or transforming data from one format to another before running it through a machine learning framework.

There _are_ scientists who write well documented, well designed framework code that is properly stored in a VCS and has release cycles, code review, etc etc. However, it is usually done in 'free time' - such as it is - and with limited resources.

Re: Too much of the research process is now shrouded by the opaque use of computers

#19

Post by cema »

grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

Yes, it is a different culture. I really would like to learn how to help the scientists absorb the proper software culture.

Re: Too much of the research process is now shrouded by the opaque use of computers

#20

Post by xioxox »

grhmc wrote: post #5

In my experience, scientists are like extremely junior developers with PhDs. They name functions "wstok" instead of "whitespace_tokenizer" because it uses fewer keystrokes. They can't be bothered to commit code. And when you do manage to get them to commit code, it is isn't the same code. It is close to the same code, but different. They think "close enough" is close enough, even though they recognize that without th…

I think that's a bit unfair to label all scientists like that. There are many of us that care about writing good code and working in version control. You just need to browse github to see many such people. There's not much incentive to actually do this in an academic environment, however, other than improving their own ability to improve and maintain their own codes. A well documented code available for everyone is usually well down the list of most scientist's priorities, for understandable reasons when you must publish papers.

Although I have quite complex programs I have written, which do benefit from software engineering methodologies, most of what I write are single-use scripts to automate tasks or try out ideas. These definitely aren't for public consumption and shouldn't be - they're more extensions of the thinking process than products of the research.

Post reply on HN