Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research software code is likely to remain a tangled mess

shape-of-code.coding-guidelines.com

Post reply on HN
Print view View original

131–140 of 171 posts

Re: Research software code is likely to remain a tangled mess

#131

Post by f6v »

warkdarrior wrote: post #120
f6v wrote: post #99

Earlier quoted context omitted.

> The main reason why research code becomes a tangled mess is due to the intrinsic nature of research. It is highly iterative work where assumptions keep being broken and reformed depending on what you are testing and working on at any given time. Oh, boy, how many times have I heard this working at a startup. There is some truth to it, it's hard to organise code in the first weeks of a new project. But if you work o…

> I guarantee that most of the low-quality research code is a result of a lack of discipline and experience in writing maintainable software. Bingo! Most research code is written by graduate students who never had a job before, so they do not know how to write maintainable software. You are definitely the exception, as you held a software dev job before going back to school.

Some researchers from top-10 schools still publish python2 code in 2020. I don't have an explanation for that. It's not even a lack of experience, but something on another level.

Re: Research software code is likely to remain a tangled mess

#132

Post by qudat »

mattkrause wrote: post #105
mkl95 wrote: post #15

If you want to write good research software, a good way is to have professional developers implement it. I worked closely with an NLP researcher for a while on a project that had received a hefty state grant. She knew more or less what her team needed, but she needed someone to implement it cleanly and in a way that would not make users step on each others toes. The chances of that project being a buggy mess would ha…

Here's the problem with hiring a pro. The workhorse NIH grant[0] is a R01 with a $250,000/year x 5 years "modular" budget. Most labs have, at most, one. Some have two, and a very few have more than that. This covers everything involved in the research: salaries (including the prof's), supplies, publication fees, etc. Suppose you find a programmer for $75k. With benefits/fringe (~31% for us, all-in), that's nearly $10…

> In contrast, you can get a student or postdoc for far less

Yeah I totally agree on this part. The academic system relies not on monetary compensation for its labor, rather it provides them reputation by getting their names on a paper.

I worked essentially for free for a lab in my spare time for 4 years. They get to the result they want, even if its built on a shaky foundation, and for basically free (it doesn't cost anything to put a name on a paper). At the end of the 4 years the dream of getting my name on a paper didn't even pan out (lab was ramping down and was essentially a teaching research lab by the time I showed up).

Re: Research software code is likely to remain a tangled mess

#133

Post by selimthegrim »

Bukhmanizer wrote: post #101
qmmmur wrote: post #59

Earlier quoted context omitted.

I can tell you why the sites went offline, because the funding stopped. I don't know what you're research background is but its painful to even get 5 GBP a month to host a droplet on digital ocean in a pretty lucrative department with liberal internal funding.

Agreed, but all these little things are just a sign that the industry just does not give a shit about software. They could develop mechanisms to fund this stuff, pretty easily actually. But they don’t. A couple of other weird inequities that I’ve found are: 1. It’s hard to get permission to spend money on software subscription based licenses since you won’t “have anything” at the end. However, it’s much easier to get…

Louisiana state government spent a buttload of money on dedicated high speed fiber optic lines between a bunch of different universities in the state for videoconferencing, telenetworking, "grid computing" etc. 10 years later the only people who remember how to use the system are at LSU, rendering the purpose moot. Everyone else just uses Zoom or Skype.

https://www.regents.la.gov/assets/docs/Finance_and_Facilitie...

Re: Research software code is likely to remain a tangled mess

#134

Post by realusername »

amelius wrote: post #119
analog31 wrote: post #93

Earlier quoted context omitted.

I believe that of all the lessons to come from contemporary software development, constant refactoring may be the most valuable. The spaghetti monster looms large when you're in the heat of battle. But we've all got some idle time for whatever reason. I spend some time every week doing a couple of things: 1) Reading about good techniques. 2) Working through old code and cleaning it up. Because changing your code coul…

Makes you wonder why most languages don't come with good refactoring tools.

I don't think refactoring tools are that useful for refactoring, most of the time you are doing non-obvious refactoring tools can't help with anyways. Depends what we call "refactoring", tools are mostly useful for what I would call "housekeeping".

Re: Research software code is likely to remain a tangled mess

#135

Post by grahamlee »

cratermoon wrote: post #122
amelius wrote: post #119

Earlier quoted context omitted.

Makes you wonder why most languages don't come with good refactoring tools.

Safe automatic refactoring requires the ability to do static analysis of the code. Many refactorings are harder in loosely-typed languages.

Refactoring tools were invented in Smalltalk and worked just fine.

Re: Research software code is likely to remain a tangled mess

#136

Post by Symbiote »

f6v wrote: post #131
warkdarrior wrote: post #120

Earlier quoted context omitted.

> I guarantee that most of the low-quality research code is a result of a lack of discipline and experience in writing maintainable software. Bingo! Most research code is written by graduate students who never had a job before, so they do not know how to write maintainable software. You are definitely the exception, as you held a software dev job before going back to school.

Some researchers from top-10 schools still publish python2 code in 2020. I don't have an explanation for that. It's not even a lack of experience, but something on another level.

Mathematics doesn't suddenly stop working because your interpreter is a bit old.

Re: Research software code is likely to remain a tangled mess

#137

Post by hntrader »

These are some concepts that I believe in for research code.

Research code shouldn't be a monolith. Each hypothesis should be a script that follows a data pipeline pattern. If you have a big research question, think about what the most modular progression of steps would be along the path from raw data to final output, and write small scripts that perform each step (input is the output from the previous step). Glue them all together with the data pipeline, which itself is a standalone, disposable script. If step N has already been run, then running the pipeline script once again shouldn't resubmit step N (as long as the input hasn't changed since the last run).

This "intermediate data" approach is useful because we can check for errors each step on the way and we don't need to redo calculations if a particular step is shared by multiple research questions.

I was taught this by a good mentor and I've been using this approach for many years for various ML projects and couldn't recommend it more highly.

Re: Research software code is likely to remain a tangled mess

#138

Post by brakus127 »

Structure is great for a well understood problem space, but this is not usually the case when working working something novel. As a researcher your focus should be on learning and problem solving, not creating a beautiful code base. Imposing too many constraints early on can negatively impact your project later on. In the worse case, your code starts to limit the way you think about your research. I agree that there are some general best practices that should be applied to nearly all forms of coding, but beyond that it's a balance.

The same thinking should be used when adding regulation to an industry. Heavy regulation on a rapid developing industry can stifle innovation. Regulation (if needed), should be applied as our understanding of the industry increases.

Re: Research software code is likely to remain a tangled mess

#139

Post by bordercases »

brakus127 wrote: post #138

Structure is great for a well understood problem space, but this is not usually the case when working working something novel. As a researcher your focus should be on learning and problem solving, not creating a beautiful code base. Imposing too many constraints early on can negatively impact your project later on. In the worse case, your code starts to limit the way you think about your research. I agree that there…

Results need to be refined so that the way they were first formulated doesn't get in the way of their replication. At scale, this too becomes a cost to industry.

In the small, this isn't different from taking a lab notebook and making it clearer and better summarized so that it can be passed on to the poor sucker who has to do what you did after you move on to another project.

Furthermore, software projects that are put under the same iterative stress you imply for R&D inevitably go through a refactoring phase so that performance isn't affected in the long run.

Re: Research software code is likely to remain a tangled mess

#140

Post by disgruntledphd2 »

grahamlee wrote: post #135
cratermoon wrote: post #122

Earlier quoted context omitted.

Safe automatic refactoring requires the ability to do static analysis of the code. Many refactorings are harder in loosely-typed languages.

Refactoring tools were invented in Smalltalk and worked just fine.

This has always surprised me, since I learned it.

What are the features of Smalltalk that allowed this to happen? Conversely, what is stopping this from existing in more modern dynamic languages?

Post reply on HN