Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research software code is likely to remain a tangled mess

shape-of-code.coding-guidelines.com

Post reply on HN
Print view View original

41–50 of 171 posts

Re: Research software code is likely to remain a tangled mess

#41

Post by lmilcin »

So here is simple fact.

It does not make sense to judge any piece of code that does not meet "highest standard" to be a tangled mess.

There are valid reasons to have varying quality of code and also the idea of quality might be changing from problem to problem and project to project.

A quality of code that governs your car's ECU should be different from quality of code that some research team threw together to demonstrate an idea.

A coding project should achieve some kind of goal or set of goals as efficiently as possible and in many valid cases quality is just not high on the list and for a good reason.

Right now I am working on a PoC to verify an idea that will take a longer time to implement. We do this because we don't want to spend weeks on development just to see it doesn't work or that we want to change something. So spending 2-3 days to avoid significant part of the risk of the rest of the project is fine. It does not need to be spelled out that the code is going to be incomplete, messy and maybe buggy.

There is also something to be said for research people to be actually focusing on something else.

Professional developers focus their careers on a single problem -- how to write well (or at least they should).

But not all people do. Some people actually focus on something else (physics maybe?) and writing code is just a tool to achieve some other goals.

If you think about people working on UIs and why UI code tends to be so messy, this is also probably why. Because these guys focus on something else entirely and the code is there just to animate their graphical design.

Re: Research software code is likely to remain a tangled mess

#42

Post by stult »

Dumblydorr wrote: post #27
asdf_snar wrote: post #24

This article seems to cover research software that even can be built. I claim the majority of _code_ written to support research articles is a collection of scripts written to produce figures to put in the paper. Even when the article is about an algorithm, the script that runs this algorithm is just good enough to produce the theoretically expected results; it is never tested, reproduced, or published, never mind be…

I am a research scientist published via R, Stata, and Excel analyses. My code documents wouldn't be helpful since the data is all locked up due to HIPAA concerns. We're talking names, health conditions, scrambled SSN, this isn't reproducible because the data is locked to those without security clearance. The code itself is a ton of munging and then some basic stat functions. This information can be gleaned from the m…

In well designed software, data ingestion should be easily separable from the core logic of the application. Which is the point the parent comment is making. Some basic best practices would allow you to share your core code without implicating HIPAA. Even if it’s just basic stats, sharing the code makes it easier to reproduce your results and to check your logic.

Although I agree with your analysis that enabling competitors in science is not common, it really, really should be. That’s kinda the point of publication, at least in theory. Sharing knowledge and methods.

Re: Research software code is likely to remain a tangled mess

#43

Post by Xelbair »

svalorzen wrote: post #3

I don't really agree with the reasons given, even though my conclusions are the same. The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Moreover, you have no idea on advance where your experiments are going to take you, thus…

>To make a concrete example, imagine writing an application where requirements changed unpredictably every day, and where the scope of those changes is unbounded.

That sounds like software development, alright. It takes a while for domain experts to learn that if programmer ask "is X always true/false", they mean that there are no exceptions from that rule.

I would like for researchers to just name variables sensibly. Even that would improve code quality a lot.

Still the key problem is that there are zero incentives for researchers to even make their code readable! It does not improve any of the metrics they are judged by.

Re: Research software code is likely to remain a tangled mess

#44

Post by exdsq »

orange_tee wrote: post #32

Well as somebody who has written research software, I don't agree that research software is a "tangled mess". A couple of points, 1. often when I read read software written by profession programmers I find it very hard to read because it is too abstract, almost every time I try to figure out how something works, it turns out I need to learn a new framework and api, by contrast research code tends to be very self cont…

Point 1 is so true, I think it’s why I like Golang without generics so people can’t go crazy with abstractions.

Re: Research software code is likely to remain a tangled mess

#45

Post by sdwvit »

lmilcin wrote: post #41

So here is simple fact. It does not make sense to judge any piece of code that does not meet "highest standard" to be a tangled mess. There are valid reasons to have varying quality of code and also the idea of quality might be changing from problem to problem and project to project. A quality of code that governs your car's ECU should be different from quality of code that some research team threw together to demons…

Yeah but you spend more time debugging, that if you write it once with a good architecture and unittests let’s say

Re: Research software code is likely to remain a tangled mess

#46

Post by [deleted] »

lmm wrote: post #28
svalorzen wrote: post #3

I don't really agree with the reasons given, even though my conclusions are the same. The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Moreover, you have no idea on advance where your experiments are going to take you, thus…

> To make a concrete example, imagine writing an application where requirements changed unpredictably every day, and where the scope of those changes is unbounded. I don't have to imagine it, I'm employed in the software industry. Seriously, nothing you describe sounds any different from normal software development.

[deleted]

Re: Research software code is likely to remain a tangled mess

#47

Post by hnedeotes »

fabian2k wrote: post #13

There is no real incentive to organize and clean up the code, even if the scientists involved have the skills to write well-organized software. And organizing this kind of code that often starts in a more exploratory way is a pretty large amount of additional effort. This kind of effort is simply not appreciated, and if spending time on it means you publish fewer papers it's a net negative for your career. I'd settle…

Yeah I mean, if your study is implying the apocalypse (or even if not, but more so if that's the case) you better put the code there, because that's what the scientific method requires, how should I believe your conclusions or and cute graphs if I can't see how you arrived at it? Maybe it was drawn in Narnia for all I know, maybe it has significant errors, or it's so tailored to produce those results that it's irrelevant.

And if the tools and methods you used for arriving at them are so messy that you dare not publish them what does that tell me about: - your process; - the organisation of your ideas; - the conclusions or points made in the paper?

I don't mean it has to be idiomatic well written code, but it should be readable enough to be followed.

Re: Research software code is likely to remain a tangled mess

#48

Post by einpoklum »

One of the things which has helped derail my own research career [1] is the tendency to not write tangled-mess code, and to publish and maintain much of my research code after I was supposed to be done with it.

Annoyingly, more people now know of me due to those pieces of software than for my research agenda. :-(

[1] : Not the only thing mind you.

Re: Research software code is likely to remain a tangled mess

#49

Post by yummypaint »

xgb84j wrote: post #36
svalorzen wrote: post #3

I don't really agree with the reasons given, even though my conclusions are the same. The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Moreover, you have no idea on advance where your experiments are going to take you, thus…

I think software quality in research has nothing to do with the problems themselves. It's more like article suggests that nobody cares about your software. The only goal is to get published and be cited as many times as possible. Your coding mistakes don't matter if they cannot be found out or hurt your reputability. How many tests would be written for business software if it had only to run for one meeting and then…

There seems to be an underlying assumption in many of these posts that code has no value once papers are published. This hasn't been my experience working in a research environment at all. The big, complex pieces of code are almost always re-used in some way. For example, theory collaborators send us their code so we can generate predictions from their work without bothering them. Probably 50% or more (and usually the most important parts) of the code written to process experimental data ends up in other experiments. From the perspective of an individual experimentalist, there is tremendous value in creating quality code that can be easily repurposed for future tasks. This core code tends to follow the individual in their career. In some ways it's an extension of commonly used mental tools, and there are diverse incentives to maintain it.

Re: Research software code is likely to remain a tangled mess

#50

Post by einpoklum »

ArtWomb wrote: post #38

There's a huge digital divide forming as well. Between the hardware a junior software engineer at a well funded research institution such as DeepMind has access to. Compared to the postdoc in Theoretical Physics at Princeton. Who is expected not only to write software. But maintain hardware for a proprietary "supercomputer" that was probably cast off ages ago from a government lab or wall street. We don't expect Aero…

> We don't expect Aerospace / Mechanical engineering students to learn metalworking. They typically have access to shop technicians for that work.

You'd be surprised, but that is often not the case. Lack of sufficient funding, or technicians being dicks, or mis-management by PIs, often result in graduate students having to do the technical work of metalwork, welding, lab equipment calibration, and a bunch of other tasks. Sometimes they even have to operate heavier machinery, or lasers etc without the minimum reasonable technical staff support.

I know this from my time on the executive committee of my old university's Grad Student Organization.

Post reply on HN