Bad scientific code beats code following "best practices" (2014) • Hacker News Board

Re: Bad scientific code beats code following "best practices" (2014)

#171

Post by skydhash » Sat, Jan 06, 2024, 1:46 PM UTC

thriftwy wrote: post #38

o11c wrote: post #23

Earlier quoted context omitted.

Wrong. How maintainable code is is measured in how well you know where to change something, and how certain you are that it did the right thing without side effects. The fatal error of the linked article is that bad scientific code often suffers from correctness problems - not just theoretical concerns, but the "negates the main point of this paper" kind of thing.

What if a new person takes your place and they do not know it and are not certain in anything?

When you hire a new person, there is usually some transfer of knowledge. Based on the above definition such transfer should be quick assuming that the person knows the platform and dependencies. A long time explaining the code is an indicator of needlessly complicated code

Re: Bad scientific code beats code following "best practices" (2014)

#172

Post by apelapan » Sat, Jan 06, 2024, 1:52 PM UTC

__loam wrote: post #28

canpolat wrote: post #21

Earlier quoted context omitted.

Working with a 300 line method is not fun, believe me. Everything is in one place and you don't have to change many files, yes, but due to the cognitive load, it's so much more effort to maintain it.

There's a happy middle path here I think. Long functions are hard to grok. Spreading the logic across 20 files also increases cognitive load. There's a balance to strike.

Long functions are not hard to grok, if they have a logical flow and stay reasonably close to a common level of abstraction (which can be high or low, doesn't matter). You just read top to bottom and follow the story.

20 files with 20 functions each does not cause high cognitive load, if the scope of each file and each function makes sense. You easily find the file+function you need, whenever you need to look something up and the rest of the time it is as if the rest didn't exist.

Good code can come in any shape. It is not shape itself that is important, it is the "goodness" that is important.

Re: Bad scientific code beats code following "best practices" (2014)

#173

Post by mihaaly » Sat, Jan 06, 2024, 1:52 PM UTC

9dev wrote: post #131

mihaaly wrote: post #100

Earlier quoted context omitted.

It is very true people should stick to the domain they know because otherwise they will have higher than average chance to f up. But that 'clear boundary' thing is a naive bollocks! No such thing! Both domain experts need to understand things beyond this imaginary and when precisely drawn then highly arbitrary boundary that is more like a gradient than a line normally (also not something relevant in a final good prod…

Perhaps I phrased this badly. My point wasn’t a clear boundary between professions, because you are right, that is difficult to impossible to draw. However, there’s a clear boundary between the goals of the code written by scientists vs. software engineers. Where a scientist aims to prove something, a software engineer builds code to produce business value. Both are trained very differently towards these goals.

I still disagree a bit. Their goals are the same: producing a product that fulfills the intended purpose and brings on value (financial or else or mixed). They may bring in their specialty learned beforehand (or even during) but this distribution of labour is not a goal but tool in reaching the common goal.

Also no such as a software engineer is trained to seeks business value while a scientist seeks proof with clear cut separation, not at all.

Re: Bad scientific code beats code following "best practices" (2014)

#174

Post by coldtea » Sat, Jan 06, 2024, 1:56 PM UTC

jakobnissen wrote: post #141

I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…

>I've found the problems that biologists cause are mostly 1. Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months

That's not on them though. That's on the state of the tooling in the industry.

Most of the time, dependencies could just be a folder you delete, and that's that (node_modules isn't very far from that). Instead it's a nightmare - and not for any good reason, except historical baggage.

The biologists writing scientific programs don't want "shared libraries" and other such BS. But the tooling often doesn't give them the option.

And the higher level abstractions like conda and pip and poetry and whatever, are just patches on top of a broken low level model.

None of those should be needed for isolated environments, only for dependency installation and update. Isolated environments should just come for free based on lower level implementation.

Re: Bad scientific code beats code following "best practices" (2014)

#175

Post by rramadass » Sat, Jan 06, 2024, 1:59 PM UTC

9dev wrote: post #62

Oh look, another tiring craftsmanship debate that other disciplines long figured out! A, say, physicist writing bad code could equally well be building a pergola for his garden. He doesn’t really know woodworking but god be damned if he couldn’t calculate the forces acting on the beams, and then add some screws - how hard can it be! And probably, he’ll even get the thing up, and it doesn’t look too bad even. Now get…

> It is a software engineer’s job to build quality software. A scientists job is to solve problems. There’s a clear boundary here, where the latter will deliver a concept to the former, who will eventually create a production-grade implementation off of that. Neither does a scientist have to build proper software, nor does a developer have to do cutting-edge research.

Precisely! See my relevant comment from another thread here - https://news.ycombinator.com/item?id=38821679

References:

1) Why science needs more research software engineers - https://www.nature.com/articles/d41586-022-01516-2

2) Research software engineering - https://en.wikipedia.org/wiki/Research_software_engineering

Re: Bad scientific code beats code following "best practices" (2014)

#176

Post by [deleted] » Sat, Jan 06, 2024, 1:59 PM UTC

9dev wrote: post #62

Oh look, another tiring craftsmanship debate that other disciplines long figured out! A, say, physicist writing bad code could equally well be building a pergola for his garden. He doesn’t really know woodworking but god be damned if he couldn’t calculate the forces acting on the beams, and then add some screws - how hard can it be! And probably, he’ll even get the thing up, and it doesn’t look too bad even. Now get…

[deleted]

Re: Bad scientific code beats code following "best practices" (2014)

#177

Post by _Wintermute » Sat, Jan 06, 2024, 2:00 PM UTC

galaxyLogic wrote: post #53

gregopet wrote: post #22

Sounds like the non-programmers are good at what they are supposed to be good at (solving the actual problem, if perhaps not always in the most elegant manner) while the programmers should be producing a highly maintainable, understandable, testable and reliable code base (and potentially have problems with advanced algorithms that rely on complicated theorems), but they are not. The OP has a case of bad programmers…

Right and I think "scientists" simply are more intelligent than average Joe Coder. Intelligent people produce better software. It is easy to learn some coding, not so easy to become a scientist. To becomes a scientist you must write and get your PhD-thesis approved, which must already be about scientific discoveries you have made while doing that thesis. Only people with above average IQ can accomplish something like…

In my experience getting a PhD doesn't require above average intelligence, it does require a lot of perseverance and a good amount of organisation though.

I honestly think most skilled tradespeople are more intelligent than me and my PhD holding colleagues.

Re: Bad scientific code beats code following "best practices" (2014)

#178

Post by Poliorcetes » Sat, Jan 06, 2024, 2:01 PM UTC

MichaelRo wrote: post #68

I've seen this: - Multiple/virtual/high-on-crack inheritance: add each function/class has 7 template specialization parameters and 3 macros which expand to templates which expand to macros - Lookup using dynamic structures from hell – dictionaries of names where the names are concatenated from various pieces at runtime, etc. think maps of maps of maps loaded from configs of configs - Dynamic loading and other grep-de…

reminds me of java spring library

and on top of it those DriverControllers and ManagerControllers keep getting depracated

Re: Bad scientific code beats code following "best practices" (2014)

#179

Post by hax0ron3 » Sat, Jan 06, 2024, 2:03 PM UTC

9dev wrote: post #62

Oh look, another tiring craftsmanship debate that other disciplines long figured out! A, say, physicist writing bad code could equally well be building a pergola for his garden. He doesn’t really know woodworking but god be damned if he couldn’t calculate the forces acting on the beams, and then add some screws - how hard can it be! And probably, he’ll even get the thing up, and it doesn’t look too bad even. Now get…

>It is a software engineer’s job to build quality software.

I think that most software engineers' jobs is not to build quality software, it is to make money. When you are trying to make money, the goal is not necessarily to make the best quality software that you can. Often, it is to make acceptably good software as soon as possible. A company that writes software that is half as good and ships it twice as fast might outcompete a company that writes software that is twice as good and ships it half as fast. I sometimes wish that my job was to build the best quality software that I can, but that is not the case. What I really get paid for is to make my employers' company successful, to make them money. And it's not that my employers don't care about making high quality software, it is just that if they cared about it too much they might get outcompeted by others who care less about it.

Re: Bad scientific code beats code following "best practices" (2014)

#180

Post by austin-cheney » Sat, Jan 06, 2024, 2:04 PM UTC

fsloth wrote: post #137

austin-cheney wrote: post #119

> Invariably, the biggest messes are made by the minority of people who do define themselves as programmers. After 15 years of writing JavaScript professionally I know that is a lie. The biggest messes are made by the majority of people hired that cannot really program.

I guess this could be an economics thing. Stereotypically maintenance of scientific codebases in general is not very lucractive, and the mental kick (IMHO you need to enjoy high performance numerical computing to be truly good at it) can be had for much better compensation doing stuff like cad or game engines. So I would imagine if the author has lots of experience of "professional programmers" maintaining their scie…

The goal of 95% of JavaScript in the wild is as mild as respond to user interactions and put text on screen. Its beyond trivial simple, but almost nobody is well trained to either the language or browser. As a result most people come in with assumptions of how things should work as determined by their education or experiences in unrelated languages and boy are most of those assumptions wildly incorrect. On top of that most JS developers tend to skew extremely young and are wildly insecure about complex data structures.

The result is a complete inability to program. Most people need really large tools to do more than 80% of the heavy lifting and they just write a few instructions on top of it. The perspective then becomes you need more advanced technologies to do cool things, because everything is too scary or mysterious otherwise.