Write it like natural language, rather than poorly named, succinct/hard to decipher variables.
That alone will put your code above 90% of scientific code.
How to write better scientific code in Python?
41–50 of 79 posts
Re: How to write better scientific code in Python?
#42The terrible irony of this post is that they use a completely wrong definition of expected value. They write down an integral...but then implement something totally different. You cannot calculate the EV of a random variable X by taking the mean of random samples drawn from the distribution of X (e.g. try it with the Cauchy distribution—see if you get something approaching the actual EV). The only thing they accompli…
Well the integral doesn’t work for a Cauchy distribution either so that seems a little unfair for criticism. And this kind of procedure is not unusual for more complex distributions that have no well-understood density function to integrate, like the results a physics simulation, aren’t they?
Re: How to write better scientific code in Python?
#43Former academic scientist, now software engineer. I think this article misses the point that rarely in science is the code the actual product/deliverable. The way a scientific code should be judged is it performant enough to answer my question in a reasonable amount of time, clear enough to another reader (which face it 99% of the time, in science the only other reader will likely be your future self or a student wit…
[deleted]
Re: How to write better scientific code in Python?
#44Having worked in the field long enough, I always feel the need to add: A) If you're writing high-performance code DON'T you _will_ hit bottlenecks which are non-trivial to the language unless you're a guru. If you're a scientist you're going to have to work to get to this level. Apart2) Python is excellent for saying "perform analysis A with variables B,C,D,E,F on X,Y,Z 30 times using this random seed library". What…
>B) For the love of whatever deity you worship. PLEASE DOCUMENT YOUR DEPENDENCIES CORRECTLY. This is non-trivial and doesn't mean "Works on CentOS6". I mean at least tested on Python x.y with depA version 1.2.3. Python isn't as bad a npm, but it's getting there...
That is why for R code, I really like the renv project.
https://www.rstudio.com/blog/renv-project-environments-for-r...
Re: How to write better scientific code in Python?
#45I think the title is a bit provocative / incorrect. There is nothing better about the scientific result of this.
The title could be, "How to improve engineering of scientific code".
Re: How to write better scientific code in Python?
#46I think to any audience other than fairly hardcore software engineers this is going to read a little... mad. The first code example was exceptionally clear, from then on, we get increasingly incomprehensible. "Better" it isnt. Scientific code is often write-once, run-once, done. And since mathematics/statistics is largely already formalised at the level of distributions, sampling, and so on -- we shouldn't expect to…
I think some degree of abstraction can lead to more clarity – especially if it hides a bunch of crap, that is done over and over again throughout the code. But these abstractions need to be very well chosen, clearly named and unless they really give you more clarity (e.g. by hiding some confusing details to bring the general point you are trying to make across more clearly) I would advice against them. If your scient…
I wish the article had discussed abstraction more, because it's especially tricky in "scientific" code.
The initial abstraction is actually pretty good, conceptually. You have a function that returns a value, like roll(sides=6), or an object you can manipulate to get a value: Die(sides=6).roll(). You then take those samples, somehow analyze them, and get a result. That code matches people's mental model very well.
If you were going to do this in C++, I'd stop here. Things only get more complicated because Python (Matlab and R too) penalize you heavily for leaving the BLAS sandbox.
Re: How to write better scientific code in Python?
#47This isn't a bad article; the title is just completely misleading. More appropriate would be:
"How to implement the simplest possible version of TensorFlow Probability [1], a probabilistic programming framework".
Re: How to write better scientific code in Python?
#48The terrible irony of this post is that they use a completely wrong definition of expected value. They write down an integral...but then implement something totally different. You cannot calculate the EV of a random variable X by taking the mean of random samples drawn from the distribution of X (e.g. try it with the Cauchy distribution—see if you get something approaching the actual EV). The only thing they accompli…
> You cannot calculate the EV of a random variable X by taking the mean of random samples drawn from the distribution of X
My understanding of statistics is rudimentary so forgive me but doesn't the sample mean of a normally distributed variable tend towards the expected value for the population?
Re: How to write better scientific code in Python?
#49Earlier quoted context omitted.
So "Python is totally performant" when it's Fortran :-)
I always feel this is a little unfair. Sure, a Python program and an R program, written the same way, using the same data structures, etc.. is going to usually show the R program is faster. But getting to that point is where the challenge is, and I feel that Python makes thinking about things like data structures and the algorithms you're using (in the case of external libraries) or writing much easier than other lan…
The catch, IMO, is that the need to stay in the numpy sandbox really constrains how you choose data structures and algorithms.
Storing N-dimensional points in a Point class, for example, is often so slow as to be a non-starter. Admittedly, it's not just a Python problem: struggles with array-of-structs vs. struct-of-array representations are pretty ubiquitous.
Re: How to write better scientific code in Python?
#50I agree with the others commenters who say the code "improvements" lead to unwarranted complexity and unreadability. Academic version of enterprise fizz-buzz this is.
IMHO, the quality of the code is not so important but more the documentation and testing around the code, and the use of best practices like git, versioning, etc. A good place to learn about these things is Patrick Mineault's The Good Research Code Handbook which is available here https://goodresearch.dev/ (intended for scientists who must produce code artifacts as part of their research).