Bad scientific code beats code following "best practices" (2014) • Hacker News Board

Re: Bad scientific code beats code following "best practices" (2014)

#181

Post by frostix » Sat, Jan 06, 2024, 2:09 PM UTC

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

>They are also less productive when coding than the scientists because they care too much about the quality of their work and not enough about getting shit done.

Ultimately I’d say the core issue here is that research is complex and those environments are often resource strapped relative to other environments. As such this idea of “getting shit done” takes priority over everything. To some degree it’s not that much different than startup business environments that favor shipping features over writing maintainable and well (or even partially) documented code.

The difference in research that many fail to grasp is that the code is often as ephemeral as the specific exploratory path of research it’s tied to. Sometimes software in research is more general purpose but more often it’s tightly coupled to a new idea deep seated in some theory in some fashion. Just as exploration paths into the unknown are rapidly explored and often discarded, much of the work around them is as well, including software.

When you combine that understanding with an already resource strapped environment, it shouldn’t be surprising at all that much work done around the science, be it some physical apparatus or something virtual like code is duct taped together and barely functional. To some degree that’s by design, it’s choosing where you focus your limited resources which is to explore and test and idea.

Software very rarely is the end goal, just like in business. The exception with business is that if the software is viewed as a long term asset more time is spent trying to reduce long term costs. In research and science if something is very successful and becomes mature enough that it’s expected to remain around for awhile, more mature code bases often emerge. Even then there’s not a lot of money out there to create that stuff, but it does happen, but only after it’s proven to be worth the time investment.

Re: Bad scientific code beats code following "best practices" (2014)

#182

Post by pas » Sat, Jan 06, 2024, 2:11 PM UTC

civilized wrote: post #164

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

These patterns appear in many fields. I take it as a sign that the tooling in the field is underdeveloped. This leads to a split between domain problem solvers, who are driven to solve the field's actual problems at all costs (including unreliable code that produces false results) and software engineers, who keep things tidy but are too risk-averse to attempt any real problems. I encourage folks with interests in bot…

journals, research universities/institutions, and grant orgs have the resources and gatekeeping role to encourage and enforce standards, train and support investigators in conducting real science not just pseudoscience, but these entities are actively disowning their responsibility in the name of empty "empowerment" (of course because rationally no one has a real chance of successfully pushing through a reform, so the smart choice is to just not rock the boat)

Re: Bad scientific code beats code following "best practices" (2014)

#183

Post by jampekka » Sat, Jan 06, 2024, 2:15 PM UTC

sseagull wrote: post #155

jampekka wrote: post #126

Earlier quoted context omitted.

I'm an ex-software developer/engineer and current scientist. In my experience TFA makes a good point, even though it's quite strawmanish. Most scientific code is horrible from any sane software developer perspective. The quality is so bad that I think a huge proportion of published results are plain wrong due to bugs in the analysis. These apply to my much of my code as well. But a lot of "software engineering" code…

I basically agree (except I will take static typing over dynamic typing any day). I work in computational chemistry, and scientists here don’t necessarily have problems actually coding (we’ve been doing it for 70+ years). But the “other stuff” is taking more and more time. Before, you wrote Fortran, put the files on disks or whatever, and sent them around. Now, you need to know: C++ and Python (and maybe Fortran too)…

> (except I will take static typing over dynamic typing any day)

Why? What exact typesystem do you prefer over Python's dynamic typing? A lot of its idioms, and probably e.g. NumPy/SciPy infrastructure would be about impossible with current static typesystems (look at the mess that is C++ scientific/ndarray libraries). Ditto for much of the autodiff and GPU stuff like pytorch.

Julia could perhaps get there, but the implementation has too many warts for it to take over.

Re: Bad scientific code beats code following "best practices" (2014)

#184

Post by marmalade2413 » Sat, Jan 06, 2024, 2:16 PM UTC

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

I was going to write a response but you've put what I would have said perfectly. The problem, at least in academia, is the pressure to publish. There is very little incentive to write maintainable code and finalise a project to be something accessible to an end user. The goal is to come up with something new, publish and move on or develop the idea further. This alone is not enough reason not to partake in practices such as unit tests, containerisation and versatile code but most academic code is written by temporary "employees". PhD's a in a department for 3-4 years, Post Doc's are there about the same amount of time.

For someone to shake these bad practices, they need to fight an uphill battle and ultimately sacrifice their research time so that others will have an easier time understanding and using their codes. Another battle that people trying to write "good" code would need to fight is that a lot of academics aren't interested in programming and see coding as simply as means to an end to solve a specific problem.

Also, another bad practice few bad practices to add to the list:

* Not writing documentation.

* Copying, cutting, pasting and commenting out lines of code in lieu of version control.

* Not understanding the programming language their using and spending time solving problems that the language has a built in solution for.

This is at least based on my own experience as a PhD student in numerical methods working with Engineers, Physicists, Biologists and Mathematicians.

Re: Bad scientific code beats code following "best practices" (2014)

#185

Post by fifilura » Sat, Jan 06, 2024, 2:18 PM UTC

civilized wrote: post #164

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

These patterns appear in many fields. I take it as a sign that the tooling in the field is underdeveloped. This leads to a split between domain problem solvers, who are driven to solve the field's actual problems at all costs (including unreliable code that produces false results) and software engineers, who keep things tidy but are too risk-averse to attempt any real problems. I encourage folks with interests in bot…

Can you elaborate on your thoughts regarding Wickham?

Re: Bad scientific code beats code following "best practices" (2014)

#186

Post by amadeuspagel » Sat, Jan 06, 2024, 2:18 PM UTC

> Files spread in umpteen directories

Tools and frameworks encourage this. Git and VS code are build around directories. In VS code the first thing in the sidebar is the explorer. When you press Ctrl+P you see an overview of files. File-System based routing.

But directories lack a crucial feature compared to text: Ordering. If I put everything in one file, I can order it in a way that makes sense. If I put everything in different files and directories, it's all going to be ordered alphabetically.

Re: Bad scientific code beats code following "best practices" (2014)

#187

Post by hyperthesis » Sat, Jan 06, 2024, 2:28 PM UTC

frostix wrote: post #181

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

>They are also less productive when coding than the scientists because they care too much about the quality of their work and not enough about getting shit done. Ultimately I’d say the core issue here is that research is complex and those environments are often resource strapped relative to other environments. As such this idea of “getting shit done” takes priority over everything. To some degree it’s not that much d…

maintainable prototypes are overengineered

Re: Bad scientific code beats code following "best practices" (2014)

#188

Post by 9dev » Sat, Jan 06, 2024, 2:29 PM UTC

galaxyLogic wrote: post #53

gregopet wrote: post #22

Sounds like the non-programmers are good at what they are supposed to be good at (solving the actual problem, if perhaps not always in the most elegant manner) while the programmers should be producing a highly maintainable, understandable, testable and reliable code base (and potentially have problems with advanced algorithms that rely on complicated theorems), but they are not. The OP has a case of bad programmers…

Right and I think "scientists" simply are more intelligent than average Joe Coder. Intelligent people produce better software. It is easy to learn some coding, not so easy to become a scientist. To becomes a scientist you must write and get your PhD-thesis approved, which must already be about scientific discoveries you have made while doing that thesis. Only people with above average IQ can accomplish something like…

Being intelligent in one domain doesn’t automatically make you good in any others. Exceptional biologists can be astoundingly bad at maths, and the other way around. Like most skills, being good at writing software requires not only intelligence, but lots of experience too. Maybe smarter people will pick it up faster, but they aren’t intrinsically better.

It’s a bit surprising you’d have to explain such a basic conclusion here.

Re: Bad scientific code beats code following "best practices" (2014)

#189

Post by dkarl » Sat, Jan 06, 2024, 2:39 PM UTC

Programmers want to embed domain terms everywhere. They look at scientific code and expect to see variables names containing "gravity," "velocity," etc.

Scientists need code to conform to the way they examine, solve, and communicate problems. I asked for an explanation of a particular function and was sent a PDF and was told to look at a certain page, where I found a sequence of formulas. All of the notation matched up, with the exception that superscripts and subscripts could not be distinguished in the code. To a programmer, the code looked like gibberish. To the scientists working on the code, it looked like a standard solution to a problem, or at least the best approximation that could be given in code.

You see the inverse problem when it comes to structuring code and projects: programmers see standard structures, expected and therefore transparent; scientists see gibberish. Scientists look at a directory called "tests" and think of a variety of possible meanings of the word, none of them what the programmer intended.

Re: Bad scientific code beats code following "best practices" (2014)

#190

Post by jampekka » Sat, Jan 06, 2024, 2:41 PM UTC

ohlokkru wrote: post #112

astrobe_ wrote: post #73

This is partly because, in my opinion, some "best practices" are superstitions. Some practice was best because of some issue with 80s era computing, but is now completely obsolete; problem has been solved in better ways or has completely disappeared thanks e.g. to better tooling or better, well, practices. e.g. Hungarian notation. Yet it is still passed down as a best practice and followed blindly because that's what…

[flagged]

You'll probably get downvoted for mentioning this, but capitalism is a big factor here.

It's not so much about selling books and farming engagement. It's about commodification of labor. The "best practices" tend to cater to the lowest common denominator to get at least something done. This lowers bargaining power of labor, drives down wages and increases capital's share of the pie.

This is inevitable in capitalism. Happens to all crafts. This is why we can have cheap crap but not nice things.