Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research software code is likely to remain a tangled mess

shape-of-code.coding-guidelines.com

Post reply on HN
Print view View original

151–160 of 171 posts

Re: Research software code is likely to remain a tangled mess

#151

Post by geebee »

jonnycomputer wrote: post #91
Zababa wrote: post #52

Earlier quoted context omitted.

> At the same time, you need to consider that such a clean up is only realistically helpful for other people to check whether there are bugs in the original results, and not much else. I assumed that most code published could be directly useful as an application or a library. Considering what you're saying, this might be only a minority of the code. In that case, I agree with your conclusion about smaller gains.

Most academic code runs once, on one collection of data, on a particular file system. Academic code can be really bad. But most of the time it doesn't matter, unless they're building libraries, packages, or applications intended for others. That's when it hurts and shows. I'm a research programmer. I have a master's in CS. I take programming seriously. I think academic programmers could benefit from better practice.…

I know a lot of "research programmers" (meaning people who write code in research labs but are not themselves the researchers or investigators on a study), and they often have MS degrees in CS - though actually, highly quantitative masters degrees where very elaborate code is used to generate answers is a bit more common than CS per se (math, operations research, branches of engineering, bioinformatics, etc).

Here's the thing - in industry, this background (quant undergrad + MS, high programming ability, industry experience) is kind of the gold standard for data science jobs. In academic job ladders it's... hmm. Here's the thing - by the latest data, MS grads in these fields from top programs are starting at between 120k-160k in industry, and there are very good opportunities for growth.

I actually think that universities and research centers can compete with highly in demand workers in spite of lower salaries, but highly talented people in demand will not turn away an industry job with salary and advancement potential to remain in a dead end job.

Re: Research software code is likely to remain a tangled mess

#153

Post by titanomachy »

analog31 wrote: post #93
SavageBeast wrote: post #70

I'm currently refactoring a fairly large piece of research code myself. It was written with lean startup thinking in that a little code ought to produce some value in its results. If i was able to eeek some usefulness out of this code, then Id put more energy into it. Otherwise I was perfectly happy to Fail Fast and Fail Cheap. How did it become such a mess in the first place? Simple - I didn't know my requirements w…

I believe that of all the lessons to come from contemporary software development, constant refactoring may be the most valuable. The spaghetti monster looms large when you're in the heat of battle. But we've all got some idle time for whatever reason. I spend some time every week doing a couple of things: 1) Reading about good techniques. 2) Working through old code and cleaning it up. Because changing your code coul…

That was my exact thought reading this.

I used to write some crazy spaghetti code as an untrained student working in a lab. Coding would go really quickly at first, but as I kept adding on to accommodate new requirements it became a huge kludgy mess.

Recently (after quite a few years of software engineering experience) I helped a researcher friend to build some software. He was following along with my commits and asked why I kept changing the organization and naming of the code, pulling things out into classes, deleting stuff that he thought might be needed later, etc. He spends only a small part of his time writing code, so he's never realized how much time it actually saves to keep things organized and well-factored.

Re: Research software code is likely to remain a tangled mess

#154

Post by porker »

hntrader wrote: post #137

These are some concepts that I believe in for research code. Research code shouldn't be a monolith. Each hypothesis should be a script that follows a data pipeline pattern. If you have a big research question, think about what the most modular progression of steps would be along the path from raw data to final output, and write small scripts that perform each step (input is the output from the previous step). Glue th…

This, absolutely this.

I looked over a friend's PhD program because the results were unstable. I knew nothing about the domain which was a large disadvantage, but on the code front it was a monolith following a vague data pipeline approach. Unfortunately components wouldn't run separately and there were only a single end to end tests taking hours to run. Had each section had its own tests, diagnosing which algorithm(s) were malfunctioning would have been easier. We never did.

Re: Research software code is likely to remain a tangled mess

#155

Post by titanomachy »

hyperpallium2 wrote: post #109
SavageBeast wrote: post #70

I'm currently refactoring a fairly large piece of research code myself. It was written with lean startup thinking in that a little code ought to produce some value in its results. If i was able to eeek some usefulness out of this code, then Id put more energy into it. Otherwise I was perfectly happy to Fail Fast and Fail Cheap. How did it become such a mess in the first place? Simple - I didn't know my requirements w…

I like Brooks' "plan to throw one away; you will, anyhow.": This [first] system acts as a "pilot plan" that reveals techniques that will subsequently cause a complete redesign of the system. However, in practice I'm not confident enough in my understanding, and fear losing all that hard-won work, so I refactor too. A rewrite from scratch is probably more viable when the project is small enough to keep in your head at…

Brooks has since amended this[1] to say that he really meant it in the context of traditional "waterfall" development, where the first iteration is meticulously planned and designed as a whole system before any code is written at all.

Rapid, iterative prototyping, followed by refactoring, is a perfectly reasonable approach today. No need to create a fresh repository and rewrite all code from scratch.

David Heinemeier Hanssen, creator of Rails and a big advocate of building working code as early as possible, wasn't even born in 1975 when the mythical man-month was written. Linus Torvalds was a (presumably) plucky 6-year-old. Brooks wrote that book for an audience that would have known waterfall as the only way.

[1] https://wiki.c2.com/?PlanToThrowOneAway

Re: Research software code is likely to remain a tangled mess

#156

Post by Bukhmanizer »

lazyjeff wrote: post #148
Bukhmanizer wrote: post #128

Earlier quoted context omitted.

> I hate that every HN post about academia ends with an anecdote describing some rare edge-case they've heard about This isn’t a rare edge case, this is very common in software projects. I’ve heard of it because I was part of the team brought in to fix the situation. Intentional fraud only is rare when it’s recognized as fraud. P-hacking was incredibly widespread (and to some extent still is) because it wasn’t recogn…

Just curious, what kind of 3-year informatics grant not being completed ends up with a team brought in to fix the situation? Multi-million dollar grants don't sound big enough to be a dependency for any major customer (like defense or pharma), so I imagine if fraud was detected, they would just demand a reimbursement and ban the PI. But I think you're both right in some sense. The cases of intentional major fraud is…

I'll be a bit vague to protect my coworker's privacy, but the scientist was fired for other, unrelated violations, and my boss was brought in to replace him. I think he was leading an arm of a "U" grant, so he wasn't the only senior PI on it. Since they handled it internally, they couldn't just demand a reimbursement. On some level administration knew that the project wasn't moving forward, but once we started asking around, it was clear that there was no effort to start the project at all.

>But I totally agree that maintaining software that people are using should be funded and rewarded by the academic communities. A possible way to do this is have a supplement so that after a grant is over, people who have software generated from the grant that is used by at least 10 external parties without COI, should be funded 100K/yr for however many years they are willing to maintain and improve it. Definitions of what this means needs to be carefully constructed, of course.

I think that this is a great idea.

Re: Research software code is likely to remain a tangled mess

#157

Post by cratermoon »

grahamlee wrote: post #135
cratermoon wrote: post #122

Earlier quoted context omitted.

Safe automatic refactoring requires the ability to do static analysis of the code. Many refactorings are harder in loosely-typed languages.

Refactoring tools were invented in Smalltalk and worked just fine.

Smalltalk is dynamically typed, but I wouldn't call it loosely typed. It's closer to Ruby's "duck typing". I was thinking more of Javascript and PHP. Weak and loose.

Re: Research software code is likely to remain a tangled mess

#158

Post by titanomachy »

disabled wrote: post #110
orange_tee wrote: post #32

Well as somebody who has written research software, I don't agree that research software is a "tangled mess". A couple of points, 1. often when I read read software written by profession programmers I find it very hard to read because it is too abstract, almost every time I try to figure out how something works, it turns out I need to learn a new framework and api, by contrast research code tends to be very self cont…

I work on mathematical modeling, dealing with human physiology. Likewise, the software packages used can be esoteric, and the structure of your "code" can be very different looking, to say the least. This is certainly a lot of work, and this takes a lot of practice to perform efficiently: But no matter what, I comment every single line of code, no matter how mundane it is. I also cite my sources in the commenting its…

"Esoteric software used for mathematical models of physiology" brought back a strong memory of the xpp software we had to use as undergrads. Apparently it was the best tool available for graphing bifurcations in nonlinear systems... but damn that was some old software.

Writing long descriptions in comments works if you're the only one editing the code, or you supervise all contributions... in a fast-changing industrial codebase, those things go out of date very quickly, so comments are used more sparsely. I document the usage of any classes or functions that my package exports, and I'll write little inline comments explaining lines of code whose purpose or effect is not obvious. Mostly I just try to organize things sensibly and choose descriptive names for variables and functions.

Re: Research software code is likely to remain a tangled mess

#159

Post by bjarneh »

nirse wrote: post #67
bjarneh wrote: post #16

I agree that academia produces its fare share of spaghetti code, but I don't think all of his arguments are correct. > writing software is a low status academic activity This is just not true. People like: Stallman, Knuth, Ritchie, Kernighan, Norvig or Torvalds are not considered as people of low status in the academic world. Writing horrible spaghetti code in academia may be considered "low status"; but that's anoth…

>> writing software is a low status academic activity > This is just not true. People like: Stallman, Knuth, Ritchie, Kernighan, Norvig or Torvalds are not considered as people of low status in the academic world. I understand the meaning of 'academic software developers' to mean 'software developers that assist in building software for other, non-CS, fields of research', but you only mention people famous within CS.…

Your probably correct, I just skimmed through the first section when I saw his bullet lists.

I guess that's true for academia in general, i.e. they consider anything but their own field as a joke.

Re: Research software code is likely to remain a tangled mess

#160

Post by innerlee »

Checkout the openmmlab project [1], where messy research codes in computer vision are rewritten an reorganized into a coherent whole. If there are more researchers join this, then not only research are fully reproducible, much more accessible to everyone, but also be compared fairly. (I'm the maintainer of mmpose [2], mmaction2 [3], and mmediting [4])

[1] https://github.com/open-mmlab [2] https://github.com/open-mmlab/mmpose [3] https://github.com/open-mmlab/mmaction2 [4] https://github.com/open-mmlab/mmediting

Post reply on HN