Research software code is likely to remain a tangled mess • Hacker News Board

Re: Research software code is likely to remain a tangled mess

#111

Post by pessimizer » Mon, Feb 22, 2021, 4:48 PM UTC

jmcdl wrote: post #92

svalorzen wrote: post #35

Earlier quoted context omitted.

Yes... and no. It is true that after a result is obtained, one could clean up the code for publication. And it is true that coding is not seen add first class at the moment. At the same time, you need to consider that such a clean up is only realistically helpful for other people to check whether there are bugs in the original results, and not much else. Reproducing results can be done with ugly code, and future rese…

Shouldn't checking for bugs be of primary importance. How many times have impressive research results turned out to be a mirage built upon a pile of buggy code? I get the sense that is far too common already.

> How many times have impressive research results turned out to be a mirage built upon a pile of buggy code?

You're actually making bugs sound like a feature here. I'm pretty sure that if you've gotten impressive results with ugly code, the last thing you want to do is touch the code. If you find a bug, you have no paper.

Re: Research software code is likely to remain a tangled mess

#112

Post by cryptica » Mon, Feb 22, 2021, 4:51 PM UTC

Math and physics are a tangled mess so it's not surprising that mathematicians and physicists write code which looks like a tangled mess. Mathematicians and physicists are trained to handle ambiguous concepts and they can work with weird abstractions which are far detached from reality. Unlike programming languages, the language of math is full of gaps - This requires the reader to make assumptions using past knowledge and conventions. Computers, on the other hand cannot make assumptions so the code must be extremely precise and unambiguous.

Writing good code requires a different mindset; firstly, it requires acknowledging that communication is extremely ambiguous and that it takes a great deal of effort to communicate clearly and to choose the right abstractions.

A lot of the best coders I've met struggle with math and a lot of the best mathematicians I've met struggle with writing good code.

Re: Research software code is likely to remain a tangled mess

#113

Post by matthewdgreen » Mon, Feb 22, 2021, 4:56 PM UTC

Bukhmanizer wrote: post #53

svalorzen wrote: post #3

I don't really agree with the reasons given, even though my conclusions are the same. The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Moreover, you have no idea on advance where your experiments are going to take you, thus…

As someone who has been on both the research and industry software end, there’s really not that much difference. Requirements change, you build that into your plans. Frankly, a lot of best practice software development that gets totally ignored by academia (e.g. OOP) can handle this exact case, and makes things way more flexible. If the problem was only unpredictability, then projects with a clear and defined end goa…

>I know of a researcher that held a multimillion dollar informatics grant for 3 years. In that 3 years they literally did nothing except collect money.

I hate that every HN post about academia ends with an anecdote describing some rare edge-case they've heard about. Intentional academic fraud is a very small percentage of what happens in academia. Partly this is because it's so stupid: academia pays poorly compared to industry, requires years to establish a reputation, and the systems make it hard to extract funds in a way that would be beneficial to the fraudster (hell, I can barely get reimbursed for buying pizza for my students.) So you're going to do a huge amount of work qualifying to receive a grant, write a proposal, and your reward is a relatively mediocre salary for a little while before you shred your reputation. Also, where is your "collected money" going? If you hire a team, then you're paying them to do nothing and collude with you, and your own ability to extract personal wealth is limited.

A much more common situation is that a researcher burns out or just fails to deliver much. That's always a risk in the academic funding world, and it's why grant agencies rarely give out 5-10 year grants (even though sometimes they should) and why the bar for getting a grant is so high. The idea is to let researchers do actual work, rather than having teams manage them and argue about their productivity.

(Also long-term unfunded project maintenance is a big, big problem. It's basically a labor of love slash charitable contribution at that point.)

Re: Research software code is likely to remain a tangled mess

#114

Post by deklund » Mon, Feb 22, 2021, 5:05 PM UTC

acmj wrote: post #108

deklund wrote: post #98

Earlier quoted context omitted.

As someone who's worked for a large part of my career as a sort of bridge between academia and industry (working with researchers to implement algorithms in production), both you and the original author are right to an extent. On one hand, academics I've worked with absolutely undervalue good software engineering practices and the value of experience. They tend to come at professional code from the perspective of "I'…

> a lot of the smartest software engineers I've known have a terrible tendency to over-engineer things. Your definition of "smartest software engineers" is the opposite of mine. In my view, over-engineering is the symptom of dumb programmers. The best programmers simplify complex problems; they don't complicate simple problems.

I don't know that our definitions are that different. Most of the over-engineering I've seen in practice was done in the name of simplifying a complex problem, but resulted in a system that was too rigid to adapt. Our definition of "over-engineered" might be different, though.

Re: Research software code is likely to remain a tangled mess

#115

Post by civilized » Mon, Feb 22, 2021, 5:14 PM UTC

I see people here saying research is like writing software with fast-changing requirements. I can see how that could seem like an adequate analogy to a software engineer, but it's not.

Researchers use code as a tool of thought to make progress on very ambiguous, high-level problems that lack pre-existing methodology. Like, how could I detect this theoretical astrophysical phenomenon in this dataset? What would it take to predict disease transmission dynamics in a complex environment like a city? Could a neural network leveraging this bag of tricks in some way improve on the state-of-the-art?

If you have JIRA tickets like that in your queue, maybe you can compare your job to that of a researcher.

Re: Research software code is likely to remain a tangled mess

#116

Post by RangerScience » Mon, Feb 22, 2021, 5:21 PM UTC

hyperpallium2 wrote: post #109

SavageBeast wrote: post #70

I'm currently refactoring a fairly large piece of research code myself. It was written with lean startup thinking in that a little code ought to produce some value in its results. If i was able to eeek some usefulness out of this code, then Id put more energy into it. Otherwise I was perfectly happy to Fail Fast and Fail Cheap. How did it become such a mess in the first place? Simple - I didn't know my requirements w…

I like Brooks' "plan to throw one away; you will, anyhow.": This [first] system acts as a "pilot plan" that reveals techniques that will subsequently cause a complete redesign of the system. However, in practice I'm not confident enough in my understanding, and fear losing all that hard-won work, so I refactor too. A rewrite from scratch is probably more viable when the project is small enough to keep in your head at…

Good architecture is pretty much just about slicing things up so that rewrites / refactors can happen incrementally, rather than all at once. This can actually go both bottom-up (these functions are easy to re-arrange and don't need a rewrite) and top-down (these functions suck, but I don't have to rearrange anything to replace them).

"Good architecture is the one that allows you to change."

Re: Research software code is likely to remain a tangled mess

#117

Post by deeeeplearning » Mon, Feb 22, 2021, 5:33 PM UTC

Why is this surprising? Has anyone been inside a Chemistry or Bio lab? You think that what happens in those labs to get research done is industrial grade?

Re: Research software code is likely to remain a tangled mess

#118

Post by sumanthvepa » Mon, Feb 22, 2021, 5:36 PM UTC

What I find really surprising about research software, is that even people in Computer Science write poorly designed code as part of their research. I would have imagined that they would be better qualified to create good code. Just goes to show that Software Engineering != Computer Science.

Re: Research software code is likely to remain a tangled mess

#119

Post by amelius » Mon, Feb 22, 2021, 5:46 PM UTC

analog31 wrote: post #93

SavageBeast wrote: post #70

I'm currently refactoring a fairly large piece of research code myself. It was written with lean startup thinking in that a little code ought to produce some value in its results. If i was able to eeek some usefulness out of this code, then Id put more energy into it. Otherwise I was perfectly happy to Fail Fast and Fail Cheap. How did it become such a mess in the first place? Simple - I didn't know my requirements w…

I believe that of all the lessons to come from contemporary software development, constant refactoring may be the most valuable. The spaghetti monster looms large when you're in the heat of battle. But we've all got some idle time for whatever reason. I spend some time every week doing a couple of things: 1) Reading about good techniques. 2) Working through old code and cleaning it up. Because changing your code coul…

Makes you wonder why most languages don't come with good refactoring tools.

Re: Research software code is likely to remain a tangled mess

#120

Post by warkdarrior » Mon, Feb 22, 2021, 5:55 PM UTC

f6v wrote: post #99

svalorzen wrote: post #3

I don't really agree with the reasons given, even though my conclusions are the same. The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Moreover, you have no idea on advance where your experiments are going to take you, thus…

> The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Oh, boy, how many times have I heard this working at a startup. There is some truth to it, it's hard to organise code in the first weeks of a new project. But if you work o…

> I guarantee that most of the low-quality research code is a result of a lack of discipline and experience in writing maintainable software.

Bingo! Most research code is written by graduate students who never had a job before, so they do not know how to write maintainable software. You are definitely the exception, as you held a software dev job before going back to school.