There's an evaporative cooling effect. Ten years ago it was obvious that Python was going to have a very hard time in the multicore world. People who needed that performance and knew they needed that performance left somewhere in the intervening years. It has been obvious for a while that Python was not going to be capable of a general solution to that problem no matter what it did. Now those people are no longer in…
Why Python Is Terrible
81–90 of 125 posts
Re: Why Python Is Terrible
#82Can someone explain this part to me, please? I don't follow what's going on. > Python's use of reference counting defeated copy-on-write because even memory blocks holding variables that were read-only were actually written to in order to manipulate the reference counts, thereby blowing up the combined physical memory footprint of the workers. We solved this by smurfing the interpreter to use a magic reference count…
This would be a nonstarter if it required N+1 times the memory of the single process, so the OS uses an optimization called copy-on-write. When a process forks, all its physical memory is shared by the new process so it takes almost no new memory to start. If the new process writes to a memory page, that physical page is copied so it has its own version. (Thus “copy on write”.)
For most programs this works fine, but if you have a runtime that does garbage collection using a technique that requires writing to an object even if the code doesn’t change any of its values, trouble ensues. With reference counting, you have to write a new reference count for an object anytime a pointer to the object is assigned. If you store the reference count in the object, that means its physical page has to be copied. So now the CoW optimization totally doesn’t work, because just referencing an object causes it to take up additional new memory.
Ruby used to have this same problem, and after Ruby webservers became popular (hello Rails) they eventually incorporated a patch to move the GC information somewhere outside the actual object heap. Other systems like the JVM use similar techniques to store the bookkeeping bits somewhere other than the object field bits.
So what the OP did is patch the runtime so the objects created in the master process (pre-forking) have special reference counts that are never altered. This mostly works, because the master process generally does a bunch of setup so its objects were mostly not going to be garbage anyway.
Re: Why Python Is Terrible
#83Earlier quoted context omitted.
You are right, the article is wrong. It is most certainly not lazy. f(g(), h()) will always call both h and g regardless of whether f uses their results. Iteration in a sense can be "lazy", but that laziness is via data structures built on top of a strict core language. Python is not alone in having lazy iteratable data structures, but it leans into them relatively hard in its standard library. Many of us love this,…
Almost every mainstream language has a way to "lazily" (in the sense you mean) iterate over lists, so this can't be what the author was singling out. I think he's just confused.
Re: Why Python Is Terrible
#84I'm glad I never reported to him while at Google.
Re: Why Python Is Terrible
#85The author of the post seems like an evangelist for the Go programming language. > And, not to put too fine a point on it, but if you can code Python but not Go (or another decent programming language), you probably have no business writing software for a living.
What's funny is that (in a different Python rant from another author), it was pointed out that Google was the heaviest pusher of Python in the early 2000s. It probably would have been Java (from Android and elsewhere) had it not been all the legal stuff with Oracle brewing. So, Python here, Python there, Python everywhere... then Google invented Go and Dart and other shiny new toys and began pushing them everywhere.
Python (fronting C++ code) still plays a huge role at Google. I don't see that changing. Go has almost zero story for scientific computing.
Re: Why Python Is Terrible
#86Earlier quoted context omitted.
What's amazing is people who program for a living, who people would normally think of as being "experts", have so little knowledge of all the different types of programming that is done by, well, people who program for a living, and the tools they use to do that programming. Not only then is it gatekeeping, it's also a sign of an inexperienced programmer. I'll give one thing to Python programmers, they tend to work d…
You're saying that specialization is bad and that specialists can't be experts.
Re: Why Python Is Terrible
#87Re: Why Python Is Terrible
#88Every single "This language is good" or "This language is bad" take really needs to always come with a "for what, exactly." "This wrench is really bad for hammering nails!"
Agreed. But I also think it's fair to criticize general purpose languages on general grounds :) It's just that this article isn't very good at it.
It honestly just strikes me as very odd how e.g. even "just use multiple languages" is done a whole lot, but not really talked about as a good idea (as much as MY LANGUAGE IS GOOD AND YOURS IS NOT)
Re: Why Python Is Terrible
#89There's an evaporative cooling effect. Ten years ago it was obvious that Python was going to have a very hard time in the multicore world. People who needed that performance and knew they needed that performance left somewhere in the intervening years. It has been obvious for a while that Python was not going to be capable of a general solution to that problem no matter what it did. Now those people are no longer in…
If the only problem with Python was its slowness, I could live with it! Its other problems are worse (like tooling, bad type hinting, etc).
The semantics of Python are fundamentally slow. To fix that requires changing semantics. Such a change would dwarf 2 -> 3 in size and be effectively a new language, like "Perl 6" was.
Re: Why Python Is Terrible
#90Yes, pretty much agree with this word for word. It is very, very difficult to refactor a python application in any sort of reliable way. The standard way of error handling in python appears to be to present the user with a stack trace. Very user friendly (not!). Now people will say that, for instance, mypy can help with this. That is true but since projects can be started without type checking chances are that your p…
> The standard way of error handling in python appears to be to present the user with a stack trace. What do you expect it to do? Silently fail and move on?