71–80 of 251 posts
Earlier quoted context omitted.
What are "hunter-gatherer-style installs?"
I take that as "keep running pip install on yet another discovered dep until the script works." And heaven help you if you accidentally get conflicting versions installed in the same venv. (By conflicting, I mean two libraries that both depend on different versions of the same underlying library. Odds of this increase dramatically the more you use.)
Earlier quoted context omitted.
> I have become increasingly convinced Python as a language is a "trap" for any use of notable scal At what point do you embrace the trap and happily live with it? For example Dropbox runs many millions of lines of Python, has massive traffic and their server costs aren't completely out of line for the service they offer. Their code base is also over 10 years old. It seems to be working very well for them. I talked t…
My understanding is that Dropbox has been building performance-critical parts of their services in Go since 2014 at the least. https://twitter.com/jamwt/status/629727590782099456 Like many companies, they had initial scaling successes with Python,hit performance bottlenecks and looked for a way out with faster languages.
In a ton of cases you'll be completely fine serving a few million monthly page views of your SAAS app on a single $20-40 a month server using Flask, Django or whatever Python web framework you prefer. Performance will be really good too. Just talking about most apps where it's mainly reading and writing data to a DB.
Earlier quoted context omitted.
What are "hunter-gatherer-style installs?"
I take that as "keep running pip install on yet another discovered dep until the script works." And heaven help you if you accidentally get conflicting versions installed in the same venv. (By conflicting, I mean two libraries that both depend on different versions of the same underlying library. Odds of this increase dramatically the more you use.)
I'm militantly disinterested in the performance of Python until the deployment style of most Python projects is no longer "hunter-gatherer-style installs".
Best quote at the end: I wish I knew what went on in modern CPUs when it comes to branch prediction and inline caching because that is absolute magic. Seems spot on, and I really wonder as well. Always had the feeling that some decades ago it was still possible to outsmart the CPU (well, and the compiler/optimizer) and get performance improvements by thinking like a CPU, but these days this seems to have become impos…
how about getting rid of GIL?
Earlier quoted context omitted.
Given Python has the multiprocessing module, I always get confused when people talk about Python lack of support for multicore. What are the shortcomings of the multiprocessing module, that cause people to disregard it?
As a sibling mentions, the multiprocessing support is basically support for spawning new processes. It's better than nothing, but it's not very good for the performance. The real problem with Python multicore is that the main problem is solves is the one ctur is talking about, namely, "Oh crap, I used Python and it doesn't run fast enough for my needs... maybe I can run more processes?" Using a language that is alrea…
What I'm getting at is, let's say you have some system where you need highly performant real time processing of some data. Perhaps 10 years ago, you might have needed a complex multi-threaded java/c++ app to handle it. But perhaps now, you use, say, DynamoDB, kinesis, kafka, lambdas, or some other collection of services that you're basically gluing together with... python.
I'm not saying you're wrong. I'm just wondering, if you could truly factor in the costs in both developer time -- developing multi-threaded apps is usually pretty tricky after all -- and infrastructure costs and whatnot, where the boundary between "python is adequate" vs. "we really need go/java/c++" lies, and in which direction it's moving.
In retrospect, I guess it's sort of a silly question, since at the core, the services we'd be gluing together are no doubt written in faster languages. Perhaps a better way of framing it what % of problems can be adequately solved (cheaply) by just using language X, and how has that changed over time?
I'm militantly disinterested in the performance of Python until the deployment style of most Python projects is no longer "hunter-gatherer-style installs".
However, with pip's native freeze and install -r abilities, I'm also curious what you mean by hunter gatherer style installs. I've not yet had a problem with installing Python modules.
Earlier quoted context omitted.
If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs.
> If you get rid of the GIL you have to replace one big lock with a bunch of very small locks, which actually hurts performance for programs which only need 1 core, which is most programs. Python could do what most other languages do and allow programmers to explicitly add locks to the code that needs to be locked, rather than locking everything, all the time.