It's interesting to me that they are going down this path instead of the microservices path. This seems like something ripe for slowly breaking down into microservices. Someone made a change that took down production because of non-deterministic outcomes? How about break out whatever they were changing into it's own service? With proper fallbacks, breaking that part shouldn't take down all of production again. To be…
Python at Scale: Strict Modules
191–200 of 259 posts
Re: Python at Scale: Strict Modules
#192Earlier quoted context omitted.
It's a long way from being a realistic approach for anyone, including Dropbox, really.
What would you say are the biggest blockers to becoming realistic? I saw on the README they need tools in the Python ecosystem to start utilizing them, which I can help with starting with isort, beyond that I'd want to do whatever I can to help the project succeed.
To be more specific: getattr, operator overloading, descriptors, heterogeneous dicts, decorators, etc.
Type checking and metaprogramming are fundamentally at odds [1]. Dynamic languages like Python have more of a focus on the latter. They later added type checking, but it comes at the "cost" of ruling out the more idiomatic metaprogramming and reflection features. In other words, static typing makes your source code bigger.
Well, optional typing to some degree lets you have the best of both worlds -- you can skip type checking of the hard parts. But optional typing doesn't let you compile your program to make it faster -- you need a fully-typed program for that.
----
I'm doing something similar to mypyc with https://www.oilshell.org/ (I actually visited Dropbox and chatted with them about it back in the spring.)
The difference is that I'm compiling Oil's Python source to C++ rather than to Python-C extension modules. So it doesn't depend on the Python runtime. It's not done but it's working well so far, and it's given me a lot of appreciation for which dynamic features Python programs actually use! (both my own and others)
Also note that mypyc was used to speed up mypy, which is a type checker. A type checker is a very particular kind of program that's different than 99% of the use cases of Python. So success on speeding it up is super impressive but it's not clear it generalizes.
The same is true for Oil -- my translation work doesn't generalize to arbitrary Python programs. Lots of people have died on that hill because it's really hard. You have a hard tradeoff between the kinds of Python programs you can support and the speedup you can give them. There are 10-20 projects over the last 2 decades at various points along that spectrum. In addition to mypyc, Oil's strategy was also inspired by Shed Skin, which is an impressive but mostly dormant Python-to-C++ compiler.
----
So in short I would say the problem is that nobody will be able to agree on a subset. You will have a lot of different fragments of Python geared toward particular use cases.
But Python will very often be more appealing than any of those fragments because it has a bigger ecosystem. One thing that I've appreciated more and more while designing a language is how much the network effects and inertia matter. It's why we're still using C and C++ after almost 50 years. I'm sure every day there is still a lot more C++ written than Go, Rust, Swift, and D combined, etc.
Python has a similar network effect and it will be around basically forever in its current form. Software doesn't really get rewritten or reduced -- more stuff just gets added on top.
[1] I wrote some posts about that tradeoff here: http://www.oilshell.org/blog/tags.html?tag=metaprogramming#m...
Re: Python at Scale: Strict Modules
#193Re: Python at Scale: Strict Modules
#194Earlier quoted context omitted.
As orf said you have no idea about their codebase. And you have no idea what's included in that statement -- given that they talk about startup time, they most likely are taking into account the whole framework, a plethora of admin and analytics tools, lots of debugging / debug-only infrastructure, migrations, lots of tooling whose sole purpose is making it easier to work in large teams, etc… (And for the record, Lin…
> As orf said you have no idea about their codebase. I do too: It's Python and it's several million lines. Metaphor: you've got three pallets of goods and have hired three trucks to move them. I don't have to know how you wrapped the pallets to know that you brought two too many trucks. I don't have to know the details of what's included in "Instagram Server" et. al. to make this call (obviously) based on my experien…
Classic HN comment!
Re: Python at Scale: Strict Modules
#195/me laughs in Ansible/terraform
Re: Python at Scale: Strict Modules
#196Earlier quoted context omitted.
I much prefer frameworks/modules for which code is executed only once you invoke their "setup" function Django _does_ have a "setup" function. You can't import and use Django database connections outside of a running application without it. Flask also has a "run" method and does no i/o without it.
Every time I hear a comment alike parent's, it makes me think how many times a day I actually read a comment in the same fashion, but about something I actually know nothing about.
Re: Python at Scale: Strict Modules
#197More and more I want someone to create a new language that amounts to a strict subset of Python, with mypy built-in, and is compilable into machine code. Python has by far my favorite syntax, community, and in my experience leads to the greatest productivity. There just happens to be a lot of overly dynamic features, that aren't even used by most, but used just enough to hold back optimization and structural improvem…
Re: Python at Scale: Strict Modules
#198Earlier quoted context omitted.
Err, did you notice that Python is in the top 3 language in TIOBE (up from 4), and named "Language of the year"? What bizarro bubble do you live in?
Python is very healthy at the moment, and still growing! However, I think that makes it even more important as a community that we don't rest on our laurels and we fix the issues we do have. CPU concurrency is definitely one of those issues.
Yes, and each new version actually adresses that, one step at a time. (see https://bugs.python.org/issue35813 for example).
Re: Python at Scale: Strict Modules
#199I have a question about a detail in the article: > But if we moved the log_to_network call out into the outer log_calls function, [...] this would no longer compile as a strict module. My current understanding is that the log_calls method would NOT get executed during module load time!?! Why would having a side effect in this function violate the intention of __strict__ ?
> My current understanding is that the log_calls method would NOT get executed during module load time!?! That's incorrect. log_calls gets executed on import because it's a decorator, so equivalent to `hello_world = log_calls(hello_world)` at the top-level (which does also get executed). log_to_network in the _wrapped() definition doesn't get executed until hello_world gets called; but outside of the definition of _w…
Re: Python at Scale: Strict Modules
#200Earlier quoted context omitted.
Hopefully in every universe where Kotlin exists.
Kotlin is meaningless without Java. It is a fools errand to think Kotlin/Native would ever overtake Java. Only on Android it might have a future, if Fuchsia never becomes a thing.