Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bad scientific code beats code following "best practices" (2014)

yosefk.com

Post reply on HN
Print view View original

291–300 of 333 posts

Re: Bad scientific code beats code following "best practices" (2014)

#291

Post by jacobolus »

alexmolas wrote: post #284
mglz wrote: post #272

I just handed in my PhD in computer science. Our department teaches "best practices" but adherence to them is hardly possible in research: 1) Requirements change constantly, since... it's research. We don't know where exactly we're going and what problems we encounter. 2) Buying faster hardware is usually an option. 3) Time spent on documentation, optimization or anything else that does not directly lead to results i…

> 3) Time spent on documentation, optimization or anything else that does not directly lead to results is directly detrimental to your progress. Here's is where I disagree. It's detrimental in the short term, but to ensure reproducibility and development speed in the future you need to follow best practices. Good science requires good engineering practices.

The point is, it's not prioritized since it's not rewarded. Grad students are incentivized to get their publications in and move on, not generate long-term stable engineering platforms for future generations.

Re: Bad scientific code beats code following "best practices" (2014)

#292

Post by pphysch »

cauch wrote: post #279
lynndotpy wrote: post #230

Scientist and programmer here, and my experiences are the opposite. I value keeping things "boringly simple", but I desperately wish there was any kind of engineering discipline. First is the reproducibility issue. I think I've spent about as much time simply _trying_ to get the dependencies of research code to run as I have done writing or doing research in my PhD. The simple thing is to write a requirements.txt fil…

My problem with this discussion is that a lot of people just say "I'm a scientist (or I'm working with scientists) and I'm observing X so I can say 'scientists blahblahblah'". Different scientific research fields are using widely different computer software environment, and have their own habits and traditions. The way a biologist uses programming has no reason to be similar to the way an astrophysicist does: they ha…

I don't think there is much variance in quality of software among (radically different) fields of science.

One of the most poorly engineered products I work with was created by a few academic CS guys. The core algorithms are sophisticated and ostensibly implemented well, but the overall product is a horrible mess.

The incentives of academia make this obvious. You need to write some code that plausibly works just enough to get a manuscript out of it, but not much else. Reproducibility is not taken that seriously, and "productization"/portability/hardening is out of the question.

Re: Bad scientific code beats code following "best practices" (2014)

#293

Post by pfisherman »

mglz wrote: post #272

I just handed in my PhD in computer science. Our department teaches "best practices" but adherence to them is hardly possible in research: 1) Requirements change constantly, since... it's research. We don't know where exactly we're going and what problems we encounter. 2) Buying faster hardware is usually an option. 3) Time spent on documentation, optimization or anything else that does not directly lead to results i…

Putting some effort into (3) can increase your citations (h-index). If people can’t use your software then they will just find some other method to benchmark against or build on.

Here you are not improving your time to get out an article, but reducing it for others - which will make your work more influential.

Re: Bad scientific code beats code following "best practices" (2014)

#294

Post by leksak »

jwagenet wrote: post #213
marmalade2413 wrote: post #184

Earlier quoted context omitted.

I was going to write a response but you've put what I would have said perfectly. The problem, at least in academia, is the pressure to publish. There is very little incentive to write maintainable code and finalise a project to be something accessible to an end user. The goal is to come up with something new, publish and move on or develop the idea further. This alone is not enough reason not to partake in practices…

Sometimes I don’t blame people for committing the ‘sin’ of leaving commented code; unless you know that code used to exist in a previous version, it may well have never existed.

It can be very warranted. For a client I'm working with now I'll routinely comment out big swaths of code as they change their mind back and forth every month or so on certain things. They won't even remember it used to exist.

Re: Bad scientific code beats code following "best practices" (2014)

#295

Post by dash2 »

mike_hearn wrote: post #275
dash2 wrote: post #257

Earlier quoted context omitted.

How are people supposed to do science without running statistical models?

This is asked in good faith of course, but that question really gets to the heart of what's been corrupting science. Statistical techniques can be very useful (ChatGPT!) but they aren't by themselves science. Science is about building a theoretical understanding of the natural world, where that theory can be expressed in precise language and used to produce new and novel hypotheses. A big part of why so much science…

Right, statistical models are not sufficient for science. I agree. But they are necessary. So I recur to my original question.

Re: Bad scientific code beats code following "best practices" (2014)

#296

Post by tesdinger »

jacobolus wrote: post #291
alexmolas wrote: post #284

Earlier quoted context omitted.

> 3) Time spent on documentation, optimization or anything else that does not directly lead to results is directly detrimental to your progress. Here's is where I disagree. It's detrimental in the short term, but to ensure reproducibility and development speed in the future you need to follow best practices. Good science requires good engineering practices.

The point is, it's not prioritized since it's not rewarded. Grad students are incentivized to get their publications in and move on, not generate long-term stable engineering platforms for future generations.

An experimental research system does not have to be a complete practical system, it can focus on a few things to prove a point, support a scientific claim.

Re: Bad scientific code beats code following "best practices" (2014)

#297

Post by edu_guitar »

One notable difference between scientific code and regular software development is that the code scientists write is an implementation of well defined/documented mathematical models, while in a, say, web application, there is no reference paper or research, the code _is_ the reference. That's why best practices are important, not for the person writing a piece of software now, but for the future. If you need to change scientific code, papers and specification make otherwise confusing structure more manageable.

Re: Bad scientific code beats code following "best practices" (2014)

#298

Post by nottorp »

hutzlibu wrote: post #249
nottorp wrote: post #212

Earlier quoted context omitted.

> as the field is still evolving and to find out the fastest way for your use case No one in the world at large cares about the fastest way, they care about the lowest budget :)

Depends. If gaming is what you do, the better the performance, the bigger the market. As then more people can play your game.

HN is funny. I've been told on a previous discussion that AAA publishers don't optimize for potatos. Now you tell me it's a business requirement.

Re: Bad scientific code beats code following "best practices" (2014)

#299

Post by tesdinger »

esafak wrote: post #192
dkarl wrote: post #189

Programmers want to embed domain terms everywhere. They look at scientific code and expect to see variables names containing "gravity," "velocity," etc. Scientists need code to conform to the way they examine, solve, and communicate problems. I asked for an explanation of a particular function and was sent a PDF and was told to look at a certain page, where I found a sequence of formulas. All of the notation matched…

The programmer's naming approach has the virtue of being self-explanatory, and thus more maintainable. Scientists don't care about maintainability. Their bar is reproducibility, and even for that they don't expect it to be as painless as an automated test.

even the variable names used by programmers are abbreviations for a longer description. longer than one letter, but still shorter than a sentence

Re: Bad scientific code beats code following "best practices" (2014)

#300

Post by bad_alloc »

alexmolas wrote: post #284
mglz wrote: post #272

I just handed in my PhD in computer science. Our department teaches "best practices" but adherence to them is hardly possible in research: 1) Requirements change constantly, since... it's research. We don't know where exactly we're going and what problems we encounter. 2) Buying faster hardware is usually an option. 3) Time spent on documentation, optimization or anything else that does not directly lead to results i…

> 3) Time spent on documentation, optimization or anything else that does not directly lead to results is directly detrimental to your progress. Here's is where I disagree. It's detrimental in the short term, but to ensure reproducibility and development speed in the future you need to follow best practices. Good science requires good engineering practices.

Never had a paper rejected for lack of reproducibility though. And as long as I am working for the PhD and not the long term career, it's still better to focus on the short term. I don't like it, but I feel that's where I ended up :(
Post reply on HN