How to write better scientific code in Python?

>First of all, the die function returns one sample at a time. It needs to be called N times to get N samples, which is slow.

This sort of thing is why I find the love of Python utterly baffling. Yes, the syntax is initially clean, but behind the scenes it is a total mess. If you just write the obvious thing the performance will be disgraceful. I don't know of another language where the for loop is considered harmful. You have to turn it into a hacked-together array language to get Python to exploit even one percent of the machine's real capabilities.

It reminds me of writing TI BASIC back in high school, where the classroom mythology was that leaving off the ending parentheses made programs run faster.

Don't write code designed to burn CPU with numerical calculations. There is hyper optimisation and then there is the right tool for the job. The speed loss from doing mathS in an interpreter is huge and frankly usually not worth paying.

Python like this is great for low CPU hyper responsive code, or for increasing efficiency of existing constructs. As a rule of advice to anyone who asks me this, go away and read about the benefits of compiled over interpreted code. There's to much to conver in 30s and frankly if you're worrying about extreme performance you shouldn't be using python... That's not too say I don't think python is awesome, but, write a module in something else :p

I've got a somewhat different critique of the article, in terms of the expected lifetime of the codebase. Part of this goes to language choice, feature choice in the language, etc. The author of the article skims this in the paragraphs above the Conclusion section.

Basically, Python is fundamentally, a moving target. You can use features today which may disappear in newer revisions of the language. This isn't theoretical, I've run into differences between 3.6 and 3.9 which caused me to add work-around code, so I didn't need to build functional versions for each different python. One may say "subclass!" but that is missing the point (and extending the cognitive load).

I've taken code I wrote in the very early 90s, compiled it, and run it against some of the data sets I had still sitting around on my disks (well, SSDs now) over the last year. These data sets were big endian, as I used to run on supers and the old style unix vendor workstations. My little endian machines were a) able to read the data thanks to a simple compiler option, b) compile the 30+ year old code, c) allow me to run this.

This is not because the language (fortran) has not evolved, but because it has been, and remains, remarkably stable over timescales comparable to peoples scien tific careers. Python, in contrast, doesn't remain stable over timescales of single digit years.

So if you code in (a very restricted subset of) Python, you may be able to get this type of stability, that should be a major feature of a stable research language. If you do any function calls, open files, create directories, etc. you need to code for the lowest common denominator, if you wish your code to be operational over even 5 year intervals.

I've got similar complaints about C++, that I ran into some 10yo boost that didn't compile on a recent compiler, even after setting the --std lower. I had to forward port that code to late model boost.

I didn't need to do that with the BLAS/GE calls in fortran.

If you think your work will be around in the long term, and you might need to use it in 10 years without redeveloping it, some of the other languages might be better fits.

Note: I have a similar, albeit more moderate critiques of Julia, in that something changed v0.7 to v1.x, that caused me 10 minutes of recoding in one case. Mildly annoyed, but given the far higher performance and far lower cognitive load of that language relative to the hyperoptimized (and likely still slow) version that article author wrote, I can handle that.

very similar to virtualenv by the sounds of it,, and doesn't really address things any more/less I would argue other than maybe snapshotting things which I'd argue isn't the same as understanding and testing.

Although I like the separation between deployment and OS that both solutions provide.

I beg to differ. Having dealt with the insanity of abbreviated variables/field names in cancer research, I can tell you that abbreviations often (1) do not immediately indicate what the actual underlying field is, and/or (2) will mislead, especially when variants are involved.

I don't find your second example exhausting to read at all - in fact I find it far _easier_ to read, especially to an astute reader (who is trying to make sense of a particular scientific literature).

No person will have any idea what `z = A*x-y` could mean 'in context', when it's so damn abstract of a reference.

Brevity in variable names is horrendous when cross-checking against papers. It is also a waste of time for scientists to reiterate what the legends/variable names mean, especially when their code could have used clear, unambiguous, verbose names instead.