Why static languages suffer from complexity

  import numpy as np
  import numpy.typing as npt

  x = np.array([1, 2, 3])
  def foo(x: npt.NDArray[np.int_]) -> bool:
      return True

However, we don't need to specify types until the point of conversion:

    let a = 1
    let b = a.float

Yeah that is not ideal. Looking at the code it seems logical at first glance to expect that `b` would be a `float`. In this case, the type hints are deceptive. Still, it's not as bad as JavaScript which doesn't even have an integer type! Just in case you haven't seen this classic: https://www.destroyallsoftware.com/talks/wat

Another gotcha I hit in Python is the scoping of for loops, e.g.,https://stackoverflow.com/questions/3611760/scoping-in-pytho...

Python takes a very non-obvious position on this from my perspective.

Ultimately, all these things are about the balance of correctness versus productivity.

I don't want to be writing types everywhere when it's "obvious" to me what's going on, yet I want my idea of obvious confirmed by the language. At the other end of the scale I don't want to have to annotate the lifetime of every bit of memory to formally prove some single use script. The vast majority of the time a GC is fine, but there are times I want to manually manage things without it being a huge burden.

Python makes a few choices that seem to be good for productivity but end up making things more complicated as projects grow. For me, being able to redefine variables in the same scope is an example of ease of use at the cost of clarity. Another is having to be careful of not only what you import, but the order you import, as rather than raise an ambiguity error the language just silently overwrites function definitions.

Having said that, as you mention, good development practices defend against these issues. It's not a bad language. Personally, after many years of experience with Nim I can't really think of any technical reason to use Python when I get the same immediate productivity combined with a static type checking and the same performance as Rust and C++ (also no GIL). Plus the language can output to C, C++, ObjC and JavaScript so not only can I use libraries in those languages directly, and use the same language for frontend and backend, but (excluding JS) I get small, self contained executables that are easily distributable - another unfortunate pain point with Python.

For everything else, I can directly use Python from Nim and visa versa with Nimpy: https://github.com/yglukhov/nimpy. This is particularly useful if you have some slow Python code bottlenecking production, since the similar syntax makes it relatively straightforward to port over and use the resultant compiled executable within the larger Python code base.

Perhaps ironically, as it stands the most compelling reason not use Nim isn't technical: it's that it's not a well known language yet so it can be a hard sell to employers who want a) to hire developers with experience from a large pool, and b) want to know that a language is well supported and tested. Luckily, it's fairly quick to onboard people thanks to the familiar syntax, and the multiple compile targets make it able to utilise the C/C++/Python ecosystems natively. Arguably the smaller community means companies can have more influence and steer language development. Still this is, in my experience, a not insignificant issue, at least for the time being.

It's not that simple. You also have to specify the effect set of the algorithm, meaning, assuming we do in place sort: every memory cell other that the input array are unchanged after termination. (If you return a fresh array, you have to specify something related). You also have to specify what happens for general sorting predicates, for example if the sorting predicate prints out the element it compares then this reveals (much of) the algorithm.

> The compression is stripping away the 'how'

The sorting example shows that this largely doesn't work in practise. In my experience for non-trivial programs you tend to have to carry around invariants whose complexity matches the size of the program you are verifying.

> I'm convinced that larger scale proof automation is way more essential than HoTT.

I strongly agree, but currently this proof automation is largely a hope, rather than a reality.

> crazy type hackery as seen in Haskell or Scala

Haskell and Scala have (local) type-inference. That makes those complex types (somewhat) digestible.

> dependent types just for refinement

If / when this really works well, so that you don't pay a price when you are not using complex types, then this would be very nice. I don't think PL technology is there yet. (I'm currently using a refinement typing approach in production code.)