Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python at Scale: Strict Modules

instagram-engineering.com

Post reply on HN
Print view View original

211–220 of 259 posts

Re: Python at Scale: Strict Modules

#211

Post by ainar-g »

ken wrote: post #20
ainar-g wrote: post #10

Earlier quoted context omitted.

It's a constant struggle against the current. Dynamically-typed languages are often “good enough for the time being”. I have the same issue explaining to our C/C++/Obj-C team why they should use static (Clang-Tidy, Infer, PVS-Studo) and dynamic (ASan, MSan, UBSan) analysis tools. They just keep giving me basically the same response of “I am a good programmer, and my code is good, and shame on you for even daring to t…

I'm confused. It should be easy to demonstrate the benefit, if there is one. Just show them the bugs! For me, it's not "status anxiety". It's simply not worth the effort. The last couple static analysis tools I ran on my programs, I spent a while getting the tool to not-crash (because even though the authors obviously had a static analysis tool themselves, they either didn't bother to run it on their own code, or it…

See, that's another thing that a lot of people don't understand about static analysis. It's not just there to find bugs in existing code, it's there to find bugs as you write or edit the code! Of course it won't find a lot in a tested code base. It's tested after all. But it immensely shortens debug time as you develop, and thus reduces testing time as well.

Re: Python at Scale: Strict Modules

#212

Post by gtirloni »

gtirloni wrote: post #195

> That means 20-60 seconds between a developer making a change and being able to see the results of that change in their browser, or even in a unit test. This, unfortunately, is the perfect amount of time to get distracted by something shiny and forget what you were doing /me laughs in Ansible/terraform

I hope the downvoters have awesome iteration cycles with Ansible/Terraform ;-)

Re: Python at Scale: Strict Modules

#213

Post by pjmlp »

coldtea wrote: post #208
pjmlp wrote: post #180

Earlier quoted context omitted.

Its adoption at banks and life science research labs tells another story. Naturally if the community finally gathers around PyPy, that might change.

> Its adoption at banks and life science research labs tells another story. In all 10 of the places it was adopted? Compared to Python with the thousands of deployments "at banks and life science research labs" it's a no show....

Compared to UNIX with the thousands of deployments "at a couple of universities campus and hobbists" it's a no show....

Re: Python at Scale: Strict Modules

#214

Post by weberc2 »

pjmlp wrote: post #181
nine_k wrote: post #69

Earlier quoted context omitted.

What Go adds in tooling and performance, it takes away in expressivity. What takes 3 lines in Python, takes 10-30 on Go.

Yeah, never understood what a Python Dev could find attractive in Go, besides not having to deal with C instead. Which is positive mind you, but they would be better served by adopting PyPy.

Pypy is super cool, but it doesn’t solve for maintainability and it only improves performance by one order of magnitude, leaving it 1-2 orders slower than Go. Besides, IMO, goroutines are so much nicer than Python’s async.

Re: Python at Scale: Strict Modules

#215

Post by weberc2 »

nine_k wrote: post #154
weberc2 wrote: post #151

Earlier quoted context omitted.

Yeah, but typically only locally. Like using a for loop instead of a list comprehension, or handling errors. So more keystrokes, but in most cases not more complexity. In some cases (generic programming), Python really is more expressive, but those ~5% of cases aren’t worth the tooling/perf/maintainability tradeoffs most of the time.

It depends. Code reviewers glazing over copy-pasted boilerplate blocks can more easily lose track of the whole, and miss an error which is obvious when the whole is expressed in 10 lines. There is some optimal range of expressive density for comfortable use by humans. APL or K likely above that level, and Go feels below it, not as low as COBOL, but still.

The opposite is true in my experience. Most of that boilerplate is brackets and indentation, which visually frame the interesting bits, drawing your eye to them. This is, of course, subjective, but I use both regularly and at worst this is not a problem for Go.

Re: Python at Scale: Strict Modules

#216

Post by weberc2 »

timothycrosley wrote: post #155
weberc2 wrote: post #151

Earlier quoted context omitted.

Yeah, but typically only locally. Like using a for loop instead of a list comprehension, or handling errors. So more keystrokes, but in most cases not more complexity. In some cases (generic programming), Python really is more expressive, but those ~5% of cases aren’t worth the tooling/perf/maintainability tradeoffs most of the time.

I'm curious what you see as the tooling benefits of go?

Build tooling (“go build” vs setup.py), type checker, text editor support (hovering over a symbol for the type and docstring), documentation generator / godoc.org, dependency management (pip is great but it’s not reproducible; go’s toolchain is only modestly better here IMO), no need for virtualenvs, etc. I’m sure I’m missing several.

Re: Python at Scale: Strict Modules

#217

Post by weberc2 »

timothycrosley wrote: post #57
weberc2 wrote: post #47

Earlier quoted context omitted.

Sounds like Go. ;) This is a cheeky remark, but I use Python and Go, and Go very much feels like an improved Python in most ways. Especially when it comes to static analysis, build tooling, distribution, performance, etc. In particular, I love that there are no venvs, pipenvs, virtualenvs, pyenvs, wheels, eggs, setuptools, easy_installs, etc.

I hate the fact that you may be right, because I really don't like Go in many ways: - I hate it's module system and package eco-system story. - I don't like its syntax. - I don't like its error handling. - I'd much prefer gradual typing. - I want to maintain the ability to use interactive interpreters. - I don't like the fact that instead of being community driven it is Google driven. But, anecdotally, I see go being…

These are all fair points. I really enjoy Python, but there are too many things I fight with on a regular basis that simply aren’t issues in Go. It could be so much better if (1) there was a better type system (mypy is unnecessarily shoehorned into the syntax and still very broken—can’t even express recursive types like JSON), (2) a good way to constrain the dynamism so performance could be improved, and (3) a better environment/package management and distribution story (so far pantsbuild.org and PEX files are the best I’ve found). Then there are a long tail of more minor issues, like async/await vs goroutines, real parallelism, etc.

Re: Python at Scale: Strict Modules

#218

Post by k_sze »

Another thing that I would like to see in some kind of strict mode is the ability to mark explicit exports like in JavaScript modules. I often want to import multiple things globally at the top of a module because they are shared by multiple class or function definitions that I am writing. However, such imports end up being exposed to and usable by the consumers of my module, even though the consumers should really have imported those things at their source instead of via my module.

There are currently maybe two ways to tackle this “problem”, without a strict mode:

1. Don’t import at the global module scope; but that’s a bit tedious.

2. Import with rename, like `import os as _os`, and then leave it to the principle of “we’re all consenting adults”. I.e. if anybody imports and used things that start with an underscore, it’s clearly their fault, not mine.

Re: Python at Scale: Strict Modules

#219

Post by rurban »

I like that idea, it's just not that easy. How to do define module versions and inheritance, when you are not allowed to do global assignments in the module. declarations only, and no IO or global side effect is fine, but declaring versions and inheritance need to be allowed in global scope.

I added these ideas here: https://github.com/perl11/cperl/issues/406

Re: Python at Scale: Strict Modules

#220

Post by gtirloni »

gtirloni wrote: post #212
gtirloni wrote: post #195

> That means 20-60 seconds between a developer making a change and being able to see the results of that change in their browser, or even in a unit test. This, unfortunately, is the perfect amount of time to get distracted by something shiny and forget what you were doing /me laughs in Ansible/terraform

I hope the downvoters have awesome iteration cycles with Ansible/Terraform ;-)

I'd love to hear your feedback
Post reply on HN