Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are the best coding practices in research projects?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 14 posts

Ask HN: What are the best coding practices in research projects?

#1

Post by mlajszczak »

Research environment differs significantly from regular software engineering:

- researchers conduct experiments that may fail

- they often produce PoC instead of final products

- it's tempting to produce tonnes of low quality code (since it just an experiment/PoC!)

- often researchers are not software engineers so they don't really care about code quality / tests

How to find a good trade-off between high coding standards and not getting in the way of research? Is it possible to move smoothly from PoC to production solution without rewriting everything from scratch? How to share code between experiments / PoCs?

Re: Ask HN: What are the best coding practices in research projects?

#2

Post by fundamental »

In an ideal world you start off with loose highly manual code/processes when validating an idea and as you get closer to a publication components are restructured/rewritten to make it easy to replicate results.

Other researchers care about the final code which is used to generate the results. So, in my book it's ok if there's a large gap between the code that lead to the initial idea and the code which was used to show the idea in practice (i.e. the code used to generate all graphs/tables within a submitted publication).

Re: Ask HN: What are the best coding practices in research projects?

#3

Post by mipmap04 »

I've been lead on a few research projects. I found it useful to require that all work be tied to work items in our project management tool. The work item would need to describe the purpose and hypothesis of the work and capture summary results. We then created a file structure that matched to our work item IDs for all work product created while working on that task. Towards the end of the engagements, we go through and find what's relevant and include those artifacts in our report with steps for reproduceability, assumptions, and other clarifying points that may be salient.

We also made good use of tagging features in our project management toolset to make report writing easier at the end of the project.

Re: Ask HN: What are the best coding practices in research projects?

#4

Post by throwawayjava »

There is no one answer. The answer will change drastically depending on the sort of research you're doing, and your role in that research.

Are you a mathematician simulating a dynamical system? A theoretical computer scientist exploring the effects of parameters that are difficult to nail down analytically?

Are you a computer scientist working on a new sort of system? Is the point of that system to support a long-running research agenda, or to demonstrate the feasibility of a general notion/idea?

Or are you a software engineer supporting a natural scientist (e.g., in a large bio/neuro/chem/physics lab)?

Are you the PhD student, the research scientist, the supporting engineer, or the PI?

But in any case, the correct answer will start with interrogating the purpose/role of the software in your research project. And that answer could range from "hack out the MATLAB and sanity check" all the way to "lives are on the line; practice extreme rigor". And certainly not excluding "convince your funding agency/PI that it's time to hire a professional"!

Re: Ask HN: What are the best coding practices in research projects?

#5

Post by hprotagonist »

It's a free-for-all. (welcome to my world :( )

Something small but meaningful that I believe in are tools like versoneer (https://github.com/warner/python-versioneer/) which bump the version of your code on every commit.

Then, embed this version string in all output. Figures, serialized data, whatever.

It is very powerful to be able to point at a figure and say "this graph was produced by precisely this code". If you're feeling particularly anal, include the hashes of the datasets that generated it too.

Re: Ask HN: What are the best coding practices in research projects?

#6

Post by irvingprime »

I used to be a software lead in a research organization. I have a LOT of opinions on the subject but I'll give you just a few points.

- I've seen many cases where researchers refused to share their code because they knew it wasn't up to any reasonable standard. This is a red flag. If they are embarrassed by their code, I tend to discount their alleged results entirely.

- Even in research, people should be required by the organization to follow some kind of process. Use version control (like git or even svn. This basic step is still not universal), put in pull requests, get code reviewed from someone else.

- For that purpose, every research organization should have someone on staff who can do a competent review. They do not need to specialize in the researcher's field. They just need to know a code smell when they see it.

- Every researcher I have known will resist this strenuously. That is a sign of how much they need it.

- When publishing research results, code and data should always be required. Otherwise, the results cannot be judged. (A lot of people like it that way. They should not be accommodated).

I could go on but I'll be nice and stop here.

Re: Ask HN: What are the best coding practices in research projects?

#7

Post by electricslpnsld »

Used to be a research scientist at a ~3k person Bay Area company. Early versions of a project typically existed outside of the mainline code branch at the company (we usually just tossed it into the company's private gitlab). We tried to keep things reasonably organized and conforming to the company's standards, but as paper deadlines approached the code usually devolved into a mess of technical debt (just like grad school!). After we knew we had some new algorithm or technique that worked, we typically went back and stripped away the cruft, cleaned up the code so it conformed with the company's coding standards, added required tests, pushed to the company's main branch, and began the really hard work: convincing people in the company that what we did was useful for their work to drive adoption.

Addressing the specific points:

> - researchers conduct experiments that may fail

Very true! Often many, many, many experiments...

> - they often produce PoC instead of final products

This is going to vary a lot depending on where you are working. We were responsible for developing, deploying, and for some time supporting whatever we developed.

> - it's tempting to produce tonnes of low quality code (since it just an experiment/PoC!)

Yep, especially in early stages.

> - often researchers are not software engineers so they don't really care about code quality / tests

This really depends on what stage in the lifecycle of a research project we were in. We were responsible for deploying the final code, so at the end of the day it had to be of the same quality as something someone with the title of software engineer would generate.

Re: Ask HN: What are the best coding practices in research projects?

#9

Post by tripn »

researcher should NOT care anything related to "best coding practices", that is engineer's job. if you organization is very cheap, try to put both "researcher" and "software engineer" hat on same person, unless he/she wants to be both, otherwise this organization is asking for cheap research result, very simple, always true.

Re: Ask HN: What are the best coding practices in research projects?

#10

Post by quickthrower2 »

> How to find a good trade-off between high coding standards and not getting in the way of research?

I think you can have both. Unit tests and good coding practices should make you faster once you have more than "1 screen"'s worth of code and you are relying on human memory when navigating and maintaining code.

I'm not a "unit test all the things" kind of person though.

Post reply on HN