131–140 of 251 posts
I'm militantly disinterested in the performance of Python until the deployment style of most Python projects is no longer "hunter-gatherer-style installs".
Installs are now an scp.
Now of course, you gotta remember that installing something is usually more than copying the code, and there is nothing to help you with that, in any language.
Welcome to the world of deployment, which is why things like ansible and docker exist.
Earlier quoted context omitted.
Adding types after-the-fact can certainly be painful, but at least it's not an all-or-nothing choice, we can opt in to types for selected parts of a codebase. Personally I've got a lot of mileage out of Hypothesis for property-based testing. It's good at exercising edge-cases, and works particularly well when we sprinkle assertions through a codebase (where the "property" we're testing is simply "calling Foo doesn't…
> Adding types after-the-fact can certainly be painful, but at least it's not an all-or-nothing choice, we can opt in to types for selected parts of a codebase. One of the benefits of typing (that is rarely discussed) is that it deters people from writing code whose type would otherwise be crazy (e.g., "if someone passes the string 'foo' in for a parameter, then the return type is a string, otherwise it's a bytestrin…
I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…
I feel pretty similarly. Our product used to be a Python/Django monolith and over the years we've ended up pulling so much of the functionality out into services written in golang. It was fantastic for getting the product up and running relatively quickly, and we're still very happy to let Django take care of the frontend and database migrations. But when your codebase has gotten big, refactoring in Python feels like…
I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…
To an outsider who knows a bit of python because of Airflow and Spark, it seems Python has become a popular metaprogramming language that various disparate ecosystems have all adapated.
Pyspark is python but kind of its own language and much of the processing is happening outside the python runtime. I think you could say the same for people doing Numpy or Pandas.
Tensorflow probably even more so where I believe Python is the most popular interface, but you're really just programming a program to run somewhere else. Again this is the same with Airflow conceptually, though I believe the runtime is python.
If my hypothesis is overall correct, and that a majority (or large share) of python programmers aren't sharing the same ecosystem, libraries and packages, then shifting to a different runtime or language that just encapsulates a subset of the language is much less challenging.
This is the opposite problem of the JVM, where no one really likes Java the langauge, so everyone tries to create enjoyable (and productive) languages for the jvm to keep using the libraries and the runtime optimizations.
Earlier quoted context omitted.
Fair, maybe it's always a matter of picking the trap you want :) If you are optimizing for 0->1, Python is great. But if you want code to live and evolve over time with multiple authors, or ever will care about runtime performance, it's a dead end almost from the first line. But sometimes the productivity benefit is all that matters and you accept you may have to throw it all out later (or invest insanely in making i…
As with any language it's how you end up architecting your codebase. I do believe python lacks an authoritative resource for what is "good architecture" which leads to a lot of the code scaling problems.
Ugh. Python could be fast, but the python core team / GvR put so many ridiculous constraints on what can change that maybe at best it gets like 10% faster, some day. Anything meaningful means breaking compatibility with extensions, and they absolutely won't do that. I feel like GvR got burned in the python 2->3 transition, but he learned the wrong lesson. The lesson he seemed to learn was never break compatibility bu…
I wonder now that python made it to the big league, so to speak, if there shouldn't be more high level review of what exactly it should aim to be as an language and ecosystem going forward. Is the speed optimisation problem even well defined otherwise? E.g. tackling performant numerical computations via numpy and other such libraries seems to be a workable pattern. Even with compiled languages like C/C++ and fortran…
- iterators everywhere
- context managers
- generators
- namespaces
- powerful function arguments
- a good stdlib
- unicode handling
- string formatting
It's easy to get started, and then to get going.
Turns out a lot of people value that.
Python is often 1-2 orders of magnitude slower than java. That really is a lot yet Python and Java are about comparable in adoption.
Earlier quoted context omitted.
What are "hunter-gatherer-style installs?"
I take that as "keep running pip install on yet another discovered dep until the script works." And heaven help you if you accidentally get conflicting versions installed in the same venv. (By conflicting, I mean two libraries that both depend on different versions of the same underlying library. Odds of this increase dramatically the more you use.)
It's another story for conda, which flat-out refused to install my list of around 10 packages (that seems to stems from the fact that most packages are in conda-forge, and requirements are often inconsistent with the default repo).
Earlier quoted context omitted.
> It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... If you're a millionaire by that time, or have a strong established business, then it doesn't matter, it's a nice problem to have. Have your engineers have a go at it. See: Dropbox, Disqus, AirBnB, and others... And if you…
I work as a consultant for companies with billions in revenue and from what I see it definitely matters. Re-writes simply don't happen that often. Maybe for the pure venture capital SV tech companies you mentioned, but many companies are not in that bucket. Making good engineering choices from the beginning matters a lot.
If the choice is between a sub-optimally performant but working system, or a boondoggle that never gets off the ground, the “better engineering choice” is probably the former.