Bad scientific code beats code following "best practices" (2014) • Hacker News Board

Re: Bad scientific code beats code following "best practices" (2014)

#311

Post by Nevermark » Sun, Jan 07, 2024, 10:05 AM UTC

Xeamek wrote: post #278

Nevermark wrote: post #234

Earlier quoted context omitted.

> Except, we can (fairly objectively) reason about performance and security, while 'clean code' and 'maintainability' are arbitrary, with vague guidelines at best. Ok, how does one best reason about performance and security with messy unmaintainable code? You barely need to try even shallow reasoning about a code base at all, before it's clean vs. messy, and maintainable vs. unmaintainable status will feel very objec…

>Ok, how does one best reason about performance and security with messy unmaintainable code? The same way one does it with 'clean' code: Using profiling tools. Security is a bit less straight forward, but still. >will feel very objective. Keyword: Feel. And while most people could probably agree on terrible code being terrible, the 'less terrible' the code is, the more this argument becomes a feeling. And then we hit…

Certainly you can point to numbers re performance. But reasoning about performance is another thing if the code is a mess.

Messiness and unmaintainability does not scale. You can argue its not a completely objective thing to measure. That is true. And yet objectively, it really matters.

Not everything that is important comes with a clear number. Which is why experience and good judgement matter too. Two more things without numbers.

Re: Bad scientific code beats code following "best practices" (2014)

#312

Post by astrobe_ » Sun, Jan 07, 2024, 10:11 AM UTC

jayd16 wrote: post #265

astrobe_ wrote: post #73

This is partly because, in my opinion, some "best practices" are superstitions. Some practice was best because of some issue with 80s era computing, but is now completely obsolete; problem has been solved in better ways or has completely disappeared thanks e.g. to better tooling or better, well, practices. e.g. Hungarian notation. Yet it is still passed down as a best practice and followed blindly because that's what…

Such as what? I don't really know of any such superstitions that are based on nothing. I see a lot of opinion/taste presented as something more, but I really can't think of superstitions.

I saw an example the other day that escapes annoyingly escapes my mind now, as it has been sort of overwritten by the "why the heck do some people name Makefiles with a capital M!?" pet peeve.

But I'd say a bit of everything listed in TFA. For instance global variables are the type of thing which makes a little voice say "if you do that, something bad will eventually happen". The voice of experience sometimes say things like that, though.

Re: Bad scientific code beats code following "best practices" (2014)

#313

Post by nottorp » Sun, Jan 07, 2024, 12:35 PM UTC

hutzlibu wrote: post #301

nottorp wrote: post #298

Earlier quoted context omitted.

HN is funny. I've been told on a previous discussion that AAA publishers don't optimize for potatos. Now you tell me it's a business requirement.

I told you it depends. There are AAA games for console and gaming computers and there are casual mobile games for example. Very different markets.

Oh the free to play stuff doesn't exist for me.

Re: Bad scientific code beats code following "best practices" (2014)

#314

Post by Xeamek » Sun, Jan 07, 2024, 1:05 PM UTC

Nevermark wrote: post #311

Xeamek wrote: post #278

Earlier quoted context omitted.

>Ok, how does one best reason about performance and security with messy unmaintainable code? The same way one does it with 'clean' code: Using profiling tools. Security is a bit less straight forward, but still. >will feel very objective. Keyword: Feel. And while most people could probably agree on terrible code being terrible, the 'less terrible' the code is, the more this argument becomes a feeling. And then we hit…

Certainly you can point to numbers re performance. But reasoning about performance is another thing if the code is a mess. Messiness and unmaintainability does not scale. You can argue its not a completely objective thing to measure. That is true. And yet objectively, it really matters. Not everything that is important comes with a clear number. Which is why experience and good judgement matter too. Two more things w…

Again though: What do You do when someone thinks Your code is a mess, but You think it's good?

Re: Bad scientific code beats code following "best practices" (2014)

#315

Post by maximus-decimus » Sun, Jan 07, 2024, 2:13 PM UTC

bsdpufferfish wrote: post #48

cube00 wrote: post #15

When scientific code is not required to be published with its research literature who really knows how bad it is?

A few responses come to mind. Who is requiring? What counts as code that needs to be published? But perhaps the most relevant response is that few people read papers, even fewer are going to look into their zip. The whole idea of papers is to condense a whole lot of work into concise digestible information.

Aren't papers also supposed to be reproducible? Having access to the original code would allow you to see why you're not getting the same results if it's caused by a bug.

Re: Bad scientific code beats code following "best practices" (2014)

#316

Post by madsbuch » Sun, Jan 07, 2024, 6:25 PM UTC

9dev wrote: post #238

kolinko wrote: post #160

Earlier quoted context omitted.

It’s like saying that a scientist doesn’t need to know how to write, and they should just pair up with ghostwriters/copywriters. Many professions have tool/skill requirements that are not related to that profession on a strict sense, but are still necessary to do the job properly. When I learned engineering we were taught how to draw diagrams by hand and write in a technical font. Computer code, for many science fiel…

> It’s like saying that a scientist doesn’t need to know how to write, and they should just pair up with ghostwriters/copywriters. And I'd say that illustrates my point even better! A scientist needs to know how to write a paper, but that doesn’t make them a great author. When they aim to write a book, they should get help from a professional publisher. Both tasks require writing text, but a book is very different fr…

> but that doesn’t make them a great author

This is also widely considered a huge issue in academia. The dissemination is so bad that a lot of great research is never read by others.

Academia is not a business. They barely can sustain themselves, therefore, they need to do things themselves. It is though to be a physicist. Mostly because you, besides being a great physicist, also need to be a good programmer and a good author.

Re: Bad scientific code beats code following "best practices" (2014)

#317

Post by madsbuch » Sun, Jan 07, 2024, 6:31 PM UTC

esafak wrote: post #232

kolinko wrote: post #160

Earlier quoted context omitted.

It’s like saying that a scientist doesn’t need to know how to write, and they should just pair up with ghostwriters/copywriters. Many professions have tool/skill requirements that are not related to that profession on a strict sense, but are still necessary to do the job properly. When I learned engineering we were taught how to draw diagrams by hand and write in a technical font. Computer code, for many science fiel…

It depends on what you consider a core skill. One has to specialize; that means leaving secondary tasks to others.

This is only true for systems where comparative advantage applies. Academia is not one of these places (as they explicitly don't want to be a part of the economy, which I understand).

Re: Bad scientific code beats code following "best practices" (2014)

#318

Post by cauch » Sun, Jan 07, 2024, 7:50 PM UTC

pphysch wrote: post #292

cauch wrote: post #279

Earlier quoted context omitted.

My problem with this discussion is that a lot of people just say "I'm a scientist (or I'm working with scientists) and I'm observing X so I can say 'scientists blahblahblah'". Different scientific research fields are using widely different computer software environment, and have their own habits and traditions. The way a biologist uses programming has no reason to be similar to the way an astrophysicist does: they ha…

I don't think there is much variance in quality of software among (radically different) fields of science. One of the most poorly engineered products I work with was created by a few academic CS guys. The core algorithms are sophisticated and ostensibly implemented well, but the overall product is a horrible mess. The incentives of academia make this obvious. You need to write some code that plausibly works just enou…

That's a strange comment. It looks like the goal of these software are not at all for "productization" and that even if the author wanted to, there will be no point of porting it or hardening it.

It feels like Software Developers are "brainwashed": a good software is a software that is good for what software developers need to do. But a good software is a software that is good for what people who needs it need to do. If the academic people don't need "productization" and still code for productzation, then they are doing a bad job.

Inversely, software made by software developers may be really bad in academic sector, as it is the subject of the article: they overengineer when not needed, they complicate things just for portability or future use that will never happen, ...

It's a bit like if someone says "these cooks who are preparing omelettes are bad cooks: they are mixing white and yolk together, while real cooks who do meringue always remove the yolk. So the proper way to cook eggs is by removing the yolk".

(another aspect that I've noticed is that software developers talk about unit test and code review and are offended if they see scientists not using them properly, but they don't even realise that the goal of these is fulfilled in an arguably more efficient way by other processes that exist in academia. For example, there is sometimes no unit test, but the creator of the algo is also the main user and they will notice directly if a new change has broken something. Or as another example, scientific collaboration often implies several teams working independently and writing their own implementation from scratch, so if team A and team B gets different results from the same inputs, they will investigate and find out the problem in the code)

Re: Bad scientific code beats code following "best practices" (2014)

#319

Post by pphysch » Sun, Jan 07, 2024, 9:38 PM UTC

cauch wrote: post #318

pphysch wrote: post #292

Earlier quoted context omitted.

I don't think there is much variance in quality of software among (radically different) fields of science. One of the most poorly engineered products I work with was created by a few academic CS guys. The core algorithms are sophisticated and ostensibly implemented well, but the overall product is a horrible mess. The incentives of academia make this obvious. You need to write some code that plausibly works just enou…

That's a strange comment. It looks like the goal of these software are not at all for "productization" and that even if the author wanted to, there will be no point of porting it or hardening it. It feels like Software Developers are "brainwashed": a good software is a software that is good for what software developers need to do. But a good software is a software that is good for what people who needs it need to do.…

> It's a bit like if someone says "these cooks who are preparing omelettes are bad cooks...

No, it's like serving the food on the floor without dishes. It doesn't matter whether the food is fancy or simple.

We need not defend the poor practices induced by the corrupt incentives of academia (publish or perish). Ideally, scientific methods and products of research should be highly reproducible (which implies a certain level of quality, in practice).

Re: Bad scientific code beats code following "best practices" (2014)

#320

Post by seadan83 » Sun, Jan 07, 2024, 9:41 PM UTC

cauch wrote: post #318

pphysch wrote: post #292

Earlier quoted context omitted.

I don't think there is much variance in quality of software among (radically different) fields of science. One of the most poorly engineered products I work with was created by a few academic CS guys. The core algorithms are sophisticated and ostensibly implemented well, but the overall product is a horrible mess. The incentives of academia make this obvious. You need to write some code that plausibly works just enou…

That's a strange comment. It looks like the goal of these software are not at all for "productization" and that even if the author wanted to, there will be no point of porting it or hardening it. It feels like Software Developers are "brainwashed": a good software is a software that is good for what software developers need to do. But a good software is a software that is good for what people who needs it need to do.…

Some of your points stand. Though, in the iron triangle of speed, quality, scope, it turns out that quality and speed are linked. Quality is a requirement for speed.

So, it seems more akin to making meringue with yolks. Eventually maybe it will work, but if you knew what you are doing and cared, it would be done better faster.

The criticism though of losing sight of the goal is valid. That happens.