Bad scientific code beats code following "best practices" (2014)
1–10 of 333 posts
Re: Bad scientific code beats code following "best practices" (2014)
#2TL;DR: a counter-productive rant against software engineers, claiming that bad code from software engineers is worse than bad code from scientists.
Did you consider hiring an experienced software engineer as a lead?
Re: Bad scientific code beats code following "best practices" (2014)
#3Read this on mobile and the identifier longWindedNameThatYouCantReallyReadBTWProgrammersDoThatALotToo overflowed into the margins - I regard this not as a bug but a feature which helped make the author’s point :-)
Re: Bad scientific code beats code following "best practices" (2014)
#4> I've been working, ... in an environment dominated by people with a background in math or physics who often have sparse knowledge of "software engineering". ... Invariably, the biggest messes are made by the minority of people who do define themselves as programmers.
Interesting switch in language here from "software engineering" to "programmers". There is of course a long history of debate on these terms, whether there is a meaningful distinction, and what qualifies as engineering versus programming.
Wherever you stand on this debate, there are a number of practices of software developers that tend to be used more towards the "engineering" side. Two of the most essential in my mind are peer code reviews and automated testing of changes (with tests, linters, type-checkers, code formatters, profilers, fuzzers, etc.).
This post doesn't talk about any of these practices or whether the so-called "programmers" messing up the scientific code are using them. I'd say if the people messing up the code are not actually advocating for using software development tools to write better code they are not actually applying software engineering practices to their code.
Re: Bad scientific code beats code following "best practices" (2014)
#5TL;DR: a counter-productive rant against software engineers, claiming that bad code from software engineers is worse than bad code from scientists. Did you consider hiring an experienced software engineer as a lead?
TFA seems to have hit a nerve.
Re: Bad scientific code beats code following "best practices" (2014)
#6TL;DR: a counter-productive rant against software engineers, claiming that bad code from software engineers is worse than bad code from scientists. Did you consider hiring an experienced software engineer as a lead?
You know, 96% of businesses (and by extension codebases) have to get by software wise without high priced software engineers. They couldn't afford it. The vast majority of running code is produced by people whose understanding of computer systems and programming goes as deep as how much documentation they need to ctrl+f through to get some specific tasks done.
Re: Bad scientific code beats code following "best practices" (2014)
#7> Simple-minded, care-free near-incompetence can be better than industrial-strength good intentions paving a superhighway to hell. The "real world" outside the computer is full of such examples.
Overengineering is insidious - "It is difficult to get a man to understand something, when his salary depends upon his not understanding it". A team can sell a solution better than a single person fixing something without making a big deal out of it. You get organizational clout and inertia on your side when you make something big and expensive.
And then complex systems are by nature hard to reason about and by extension hard to critique.
So many things come down to "complexity is the enemy".
Re: Bad scientific code beats code following "best practices" (2014)
#8If the non programmers commit correctness bugs and the programmers are just using patterns you don't like, maybe try to understand the patterns instead of balk at them.
Re: Bad scientific code beats code following "best practices" (2014)
#9Best practices tend to be overkill for small codebases that have few users, which encompasses the majority of scientific code.
Sheer tenacity is typically sufficient for scientific codebases.
Re: Bad scientific code beats code following "best practices" (2014)
#10> Simple-minded, care-free near-incompetence can be better than industrial-strength good intentions paving a superhighway to hell. The "real world" outside the computer is full of such examples. Overengineering is insidious - "It is difficult to get a man to understand something, when his salary depends upon his not understanding it". A team can sell a solution better than a single person fixing something without mak…
Prime Finance (at Amazon) did that.
Wrote fancy math on a PhD economist’s laptop.
Then when we added testing when developing a platform, we found out we’d been 5% off in allocating Prime revenue between organizations — and had the correct amount gone to Retail, the 2018 hiring freeze might have been avoided. (According to a very angry Wilke.)
Whoops.
Turned out we did need a team and all those guardrails, processes, code reviews, etc.
There’s a time and a place for “move fast and break things” — but real trouble can come from taking those academic practices into the real world.