Bad scientific code beats code following "best practices" (2014) • Hacker News Board

Re: Bad scientific code beats code following "best practices" (2014)

#281

Post by Xeamek » Sat, Jan 06, 2024, 7:23 PM UTC

danaris wrote: post #202

Xeamek wrote: post #92

Earlier quoted context omitted.

>Writing clean and maintainable code should be the best practice and writing obfuscated code for performance and security should be reserved for "few specialised cases" Except, we can (fairly objectively) reason about performance and security, while 'clean code' and 'maintainability' are arbitrary, with vague guidelines at best. Throwing out those first characteristics in name of the latter ones is just irrational. (…

There's probably a Someoneorother's Law or Something Fallacy about this, because it's a common problem, especially among people who fancy themselves More Rational (and thus More Intelligent) than others: You are assuming that the only things that matter are those that can be objectively measured (and measured simply and straightforwardly, with well-known metrics today). Developer frustration, which will increase when…

>You are assuming that the only things that matter are those that can be objectively measured.

No. Feelings do matter. But the problem is, what do You do when You have 2 people with conficliting feelings?

>Also, of course, there are at least some fairly well-accepted standards of clean code.

Are there though?

>even if some aspects of those aren't entirely agreed on by everyone, and painting them as completely arbitrary, subjective things is just wrong

Even if I grant You that there are some guidelines that are respected by overwhelming majority, that still doesn't prevent them from being arbitrary.

Re: Bad scientific code beats code following "best practices" (2014)

#282

Post by chaxor » Sat, Jan 06, 2024, 7:31 PM UTC

astrobe_ wrote: post #73

This is partly because, in my opinion, some "best practices" are superstitions. Some practice was best because of some issue with 80s era computing, but is now completely obsolete; problem has been solved in better ways or has completely disappeared thanks e.g. to better tooling or better, well, practices. e.g. Hungarian notation. Yet it is still passed down as a best practice and followed blindly because that's what…

It is important to have popular and powerful tools that can reduce amount of code for things like caching and building.

For example, Snakemake (os-independent make) with data version control based off of torrent (removing complication of having to pay for AWS, etc) for the caching of build steps, etc would be a HUGE win in the field. *No one has done it yet* (some have danced around the idea), but if done well and correctly, it could reduce the amount of code and pain in reproducing work by thousands of lines of code in some projects.

It's important for the default of a data version control to be either ipfs or torrent, because it's prohibitive to make everyone set up all these accounts and pay these storage companies to run some package. Ipfs, torrent, or some other centralized solution is the only real solution.

Re: Bad scientific code beats code following "best practices" (2014)

#283

Post by jayd16 » Sat, Jan 06, 2024, 7:33 PM UTC

thethimble wrote: post #276

jayd16 wrote: post #265

Earlier quoted context omitted.

Such as what? I don't really know of any such superstitions that are based on nothing. I see a lot of opinion/taste presented as something more, but I really can't think of superstitions.

OOP madness? XML? Web scale databases? Perhaps not superstition but certainly fundamentalist/hype-based thinking.

Chasing hyped up fads seems like the opposite of superstitions from the 80s, no?

Re: Bad scientific code beats code following "best practices" (2014)

#284

Post by alexmolas » Sat, Jan 06, 2024, 7:39 PM UTC

mglz wrote: post #272

I just handed in my PhD in computer science. Our department teaches "best practices" but adherence to them is hardly possible in research: 1) Requirements change constantly, since... it's research. We don't know where exactly we're going and what problems we encounter. 2) Buying faster hardware is usually an option. 3) Time spent on documentation, optimization or anything else that does not directly lead to results i…

> 3) Time spent on documentation, optimization or anything else that does not directly lead to results is directly detrimental to your progress.

Here's is where I disagree. It's detrimental in the short term, but to ensure reproducibility and development speed in the future you need to follow best practices. Good science requires good engineering practices.

Re: Bad scientific code beats code following "best practices" (2014)

#285

Post by dguest » Sat, Jan 06, 2024, 7:43 PM UTC

IlliOnato wrote: post #260

dguest wrote: post #152

Earlier quoted context omitted.

Most teams at CERN don't have a professional programmer available either. In a few of larger projects (those with a few hundred active contributors) there might be one or two more tech savvy people who profile the code regularly and fix the memory leaks. But few (if any) are professional programmers: most contributors are graduate students with no background in programming.

And this is scary. At least with "high-energy experiments" (like the one that discover Higgs) in colliders, a lot depends on so-called triggers, which dismiss 99.9% of information produced in a collision "on the spot", so that this information is never recorded and analyzed. They have to: there is way too much information produced. So the triggers try to identify "trivial" events and dismiss them immediately, relayin…

The triggers are the most fun part of the experiments!

The composition of teams working on triggers might be a bit of an exception in the "engineer : "data scientist" ratio. Most of the talent is still from a physics background but there's more of an engineering bent where around half the team can probably write performance critical code when they need to. Elsewhere that ratio is much lower.

Determining which data to save is a mix of engineering, algorithms, physics, bits of machine learning, and (for better or worse) a bit of politics. Surprisingly we're always desperate for more talent there.

As you say, the goal is to try to stop problems before they affect the data, but it's not always perfect. Sometimes we discover sampling biases after the data comes in and need to correct for them, and in the worst case we sometimes blacklist blocks of data.

Re: Bad scientific code beats code following "best practices" (2014)

#286

Post by danaris » Sat, Jan 06, 2024, 7:49 PM UTC

Xeamek wrote: post #281

danaris wrote: post #202

Earlier quoted context omitted.

There's probably a Someoneorother's Law or Something Fallacy about this, because it's a common problem, especially among people who fancy themselves More Rational (and thus More Intelligent) than others: You are assuming that the only things that matter are those that can be objectively measured (and measured simply and straightforwardly, with well-known metrics today). Developer frustration, which will increase when…

>You are assuming that the only things that matter are those that can be objectively measured. No. Feelings do matter. But the problem is, what do You do when You have 2 people with conficliting feelings? >Also, of course, there are at least some fairly well-accepted standards of clean code. Are there though? >even if some aspects of those aren't entirely agreed on by everyone, and painting them as completely arbitra…

> But the problem is, what do You do when You have 2 people with conficliting feelings?

Hopefully, you try to work it out like adults, rather than just declaring that your way is the only rational way, and anyone else's feelings need to pound sand.

Furthermore, this isn't primarily about "feelings" in the sense of "this hurt my feelings;" this is primarily about adding unnecessary stress to developers' lives. Stress is something that is scientifically proven to increase susceptibility to diseases and cancers, and reduce lifespans, so it seems to me that this should be enough objective and rational evidence that we should be genuinely trying to reduce it.

> Are there though?

Well, I think most people would agree that putting an entire C file on one line is a pain to work with, even if skipping the "unnecessary" whitespace does save a little space.

And naming your variables alphabetically based on the order you use them in (eg, `int alpha`, `char bravo`, `std::string charlie` makes the code hard to maintain.

"Well, but that's just obvious stuff! No one would ever do that!"

I guarantee you someone would do just about any boneheaded thing you can imagine in programming unless told not to, either out of spite or because their brain really just works that way.

Just because you've made a bunch of assumptions about how people would or should code doesn't mean that those assumptions are any less arbitrary than anything else.

> that still doesn't prevent them from being arbitrary.

...But that's the thing. They're not. Just because they're not deeply well-researched to ensure that this particular set of coding standards measurably increases performance and decreases stress while maintaining code doesn't mean that maintainability is an arbitrary thing. It just means that it hasn't been adequately studied yet.

...or maybe it just means you haven't[0] looked[1] enough[2] yet, and the research that's out there hasn't yet had time to coalesce into any kind of industry-wide action.

Furthermore, it sounds very much like you're saying that coding standards like K&R, or C++ Core Guidelines, or PSR-2, are entirely arbitrary. They're clearly specified, they're written down and easy to reference, they codify plenty of aspects of coding style—but are all coding standards, no matter how well-respected, completely arbitrary?

[0] https://www.researchgate.net/publication/299412540_Code_Read...

[1] https://www.hindawi.com/journals/sp/2020/8840389/

[2] https://www.researchgate.net/publication/303870101_Software_...

Re: Bad scientific code beats code following "best practices" (2014)

#287

Post by marcosdumay » Sat, Jan 06, 2024, 7:53 PM UTC

danaris wrote: post #202

Xeamek wrote: post #92

Earlier quoted context omitted.

>Writing clean and maintainable code should be the best practice and writing obfuscated code for performance and security should be reserved for "few specialised cases" Except, we can (fairly objectively) reason about performance and security, while 'clean code' and 'maintainability' are arbitrary, with vague guidelines at best. Throwing out those first characteristics in name of the latter ones is just irrational. (…

There's probably a Someoneorother's Law or Something Fallacy about this, because it's a common problem, especially among people who fancy themselves More Rational (and thus More Intelligent) than others: You are assuming that the only things that matter are those that can be objectively measured (and measured simply and straightforwardly, with well-known metrics today). Developer frustration, which will increase when…

> There's probably a Someoneorother's Law or Something Fallacy about this

I once got your for a couple of hours researching the origin of that famous phrase "you can't improve what you don't measure", so I could blame it correctly.

The idea is quite old, of course, and popular to the 19th century rationalists. But the format people keep teaching around today seems to be a strawman created by Deming, at the 80's, in a speech about how stupid that idea is.

Anyway, I guess we need some Othersomeoneorother's Law about how you just can't make a good point against an idea without someone taking your point, preaching it unironically, and making a movement in support of the idea.

Re: Bad scientific code beats code following "best practices" (2014)

#288

Post by IlliOnato » Sat, Jan 06, 2024, 7:56 PM UTC

mike_hearn wrote: post #271

IlliOnato wrote: post #247

Earlier quoted context omitted.

Wow. It's pretty unbelievable. It there a place where I can read the whole article? The one I found the funniest/crziest is "Bug reports were blown off by saying that they didn't matter because the "scientists" just ran their simulation lots of times and took the average", because this is exactly how some scientists I know think. This thinking is not limited to software. My father was by trade involved in building ex…

Yeah confusion between simulation and reality can be observed all over the place. Multiple runs can be needed if you're doing measurements of the natural world, but for a simulation that doesn't make sense (you can do Monte Carlo style stuff, but that's still replicable). You could see the lines being blurred in other ways. Outputs of simulations would be referred to as "findings", for example, or referenced in ways…

Thank you! This is very interesting.

> Outputs of simulations would be referred to as "findings"

Yeah, a recent brouhaha about creating (!) a traversable wormhole in a quantum computer comes to mind...

Re: Bad scientific code beats code following "best practices" (2014)

#289

Post by BobbyJo » Sat, Jan 06, 2024, 8:18 PM UTC

lynndotpy wrote: post #230

Scientist and programmer here, and my experiences are the opposite. I value keeping things "boringly simple", but I desperately wish there was any kind of engineering discipline. First is the reproducibility issue. I think I've spent about as much time simply _trying_ to get the dependencies of research code to run as I have done writing or doing research in my PhD. The simple thing is to write a requirements.txt fil…

The problem I've run into over and over with research code is fragility. We ran it on test A, but when we try test B nothing works and we have no idea why because god forbid there is any error handling, validation, or even just comprehensible function names.

Re: Bad scientific code beats code following "best practices" (2014)

#290

Post by auxym » Sat, Jan 06, 2024, 8:18 PM UTC

hyperthesis wrote: post #187

frostix wrote: post #181

Earlier quoted context omitted.

>They are also less productive when coding than the scientists because they care too much about the quality of their work and not enough about getting shit done. Ultimately I’d say the core issue here is that research is complex and those environments are often resource strapped relative to other environments. As such this idea of “getting shit done” takes priority over everything. To some degree it’s not that much d…

maintainable prototypes are overengineered

Indeed, and whatever code is used to publish a paper is a prototype, and unlikely to be reused, ever. Sometimes it is, but rarely.