Making Cython as easy as Python

This strategy should be more common! In the source tarball, there should be one main program which you start. The compiling should be an implementation detail of the main program.

The same for web applications. I never liked the idea of reintroducing a separate build (concat/minify/compress) process whenever the sources change. Instead, the production mode should be as simple as the debug mode: When the main page is requested per HTTP, first rebuild the client JS app as needed, then deliver it to the browser.

Caching the build result should be no different from any other caching of automatically-generated resources.

I use this approach for almost all new software that I write and I'm quite satisfied with that. The main program (e.g. "bin/mytool") is a wrapper shell script or Python script that runs the build as needed, then starts the program. In the simplest case, this is a wrapper around "make && ./run_tool".

Of course, the program should indicate the build with a small message to stderr, especially if the build takes some seconds or more. Moreover, it should be possible to trigger the build individually such that distros can build packages from that. On the other hand, running the program with "-h" or "--help" is a kind of build-only process, too.

I've seen some very impressive Julia libraries that perform almost at the level of C code, so I'm not sure if the gap that Julia needs to fill is that large.

Also, a quick question: which one is preferred nowadays, Cython or Numba?

Regarding the GPU, you might find an interest for MGL-MAT[1]: "MAT is a linear algebra library for working with multi-dimensional arrays which supports efficient interfacing to foreign and CUDA code. BLAS and CUBLAS bindings are available.". The author's blog is quite interesting to read (see for example [2]).

[1] http://cliki.net/mgl-mat

[2] I http://quotenil.com/On-the-Design-of-Matrix-Libraries.html

The problem with the Python ecosystem is that there exist a lot of libraries for scientific computation mostly incompatible with each other (blaze, numpy, numba, numexpr, dask...). Some of them try to reintroduce types in order to compile to native code, thereby losing the advantages of dynamic typing. In short, it is a mess, and everyone is trying to add on this because the existing ecosystem is big and the cost of rewriting everything is deemed too high, so people try to fix python limitations at the library level, sometimes making python more cumbersome to write tha a conventional statically typed language (I am looking at theano).

I don't think that Lua is that much of an improvement, and I do not have big hopes for the JVM. The growing number of projects trying to improve the situation with value types, off-heap allocations and C compatibility are a sign that the VM is not suitable for scientific computing and will not be much better for a while.

I instead hope that Nim will take a leading role for scientific computation. It has an easy syntax and is flexible enough to write libraries such as numpy. At the same time it is fast and C compatibility is trivial. Of course, there is a library ecosystem to build, but I think that this is more doable than trying to work against the language itself. I am trying to start this effort with https://github.com/unicredit/linear-algebra but the road is very long and I hope it takes off.

I cannot really comment on Julia as I do not know it welll enough

It's practical, but on the other hand, that goes against the security practice of not allowing the executable zone (be it memory or filesystem) to be writeable.

I personally prefer to precompile stuff, even running "python -m compileall" to create .pyc files, to avoid having to keep that path writeable by the user running the program.