Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python at Scale: Strict Modules

instagram-engineering.com

Post reply on HN
Print view View original

181–190 of 259 posts

Re: Python at Scale: Strict Modules

#181

Post by pjmlp »

nine_k wrote: post #69
weberc2 wrote: post #47

Earlier quoted context omitted.

Sounds like Go. ;) This is a cheeky remark, but I use Python and Go, and Go very much feels like an improved Python in most ways. Especially when it comes to static analysis, build tooling, distribution, performance, etc. In particular, I love that there are no venvs, pipenvs, virtualenvs, pyenvs, wheels, eggs, setuptools, easy_installs, etc.

What Go adds in tooling and performance, it takes away in expressivity. What takes 3 lines in Python, takes 10-30 on Go.

Yeah, never understood what a Python Dev could find attractive in Go, besides not having to deal with C instead.

Which is positive mind you, but they would be better served by adopting PyPy.

Re: Python at Scale: Strict Modules

#182

Post by pjmlp »

dunefox wrote: post #116
coldtea wrote: post #71

Earlier quoted context omitted.

Countless companies, huge and small -- from Apple and Amazon, to Google and your friendly local startup, plus all the enterprise world that's not a .NET shop... In what parallel universe is not Java immensely popular or not used for green projects?

Hopefully in every universe where Kotlin exists.

Kotlin is meaningless without Java.

It is a fools errand to think Kotlin/Native would ever overtake Java.

Only on Android it might have a future, if Fuchsia never becomes a thing.

Re: Python at Scale: Strict Modules

#183

Post by kissgyorgy »

clintonb wrote: post #144
cpeterso wrote: post #100

Earlier quoted context omitted.

I like Python, but I often wonder how many developers use Python because they actually use dynamic language features versus just liking the languages' clean syntax and library ecosystem. I'm surprised languages that offer both REPL (for development) and AOT native compilation (for production), like OCaml, are not more popular. Evidence that syntax matters, I guess. :) mypy and mypyc are interesting but their compile-…

Don’t underestimate inertia. I’ve worked with Python and Django for seven years. I know the libraries in the ecosystem. I know the framework. It’s far easier for me to start a project with Django than to learn another framework or language.

You would be surprised if you start learning Go. It's far easier to learn that whole language that some part of Python (for example asyncio)

Re: Python at Scale: Strict Modules

#184

Post by dagw »

TylerE wrote: post #131
coldtea wrote: post #123

Earlier quoted context omitted.

In my assessment, not really. They are surprisingly spot on (for the top spots and singling out contenders). Not to mention the language landscape is almost static at the top. Nobody's gonna came and take Python, Java, C, C++, JS, out in the next 10-15 years... Only a huge self-blunder, like the Perl 5 -> 6 transition, and only at much more volatile time (when paradigms change, e.g. when web dev changed from CGI, Per…

Well, for instance, TIOBE has Rust at #34, below such mainstays as ABAP, COBOL, and Scratch. It has Groovy above Ruby.

Well, for instance, TIOBE has Rust at #34, below such mainstays as ABAP, COBOL

Do you believe that to be incorrect? I think you're probably underestimating how widely used those languages are in massive, "boring" companies around the world. Rust may be the new cool kid, and may even be the future, but the number of companies around the world that have adopted Rust for anything significant today is minuscule.

It has Groovy above Ruby.

Again, Java is everywhere, and many Java shops have added Groovy to their workflow where it makes sense. Ruby is barely used outside a small of number of tech companies.

Re: Python at Scale: Strict Modules

#185

Post by pjmlp »

rswail wrote: post #173
lern_too_spel wrote: post #78

Earlier quoted context omitted.

For large scale projects, Java and C++ remain the go-to languages. I've seen a little bit of Go start to show up but no others. Other languages are used for libraries (Rust, C), only at certain employers (OCaml, Erlang), or for small-scale projects (nearly everything else).

On C++, inertia is a wonderful thing. Java has the benefits of extensive dependency injection and JVM/ecosystem tools that lower the risk of deployment of code. .NET also provides the controlled "managed code" environment of CLR. Why any enterprise would use C++ for standard "business" or "large scale" programming makes no sense to me. Enterprises want stability, not speed to market. Most of their infrastructure chan…

As ex-C++ dev, that lives in the Java/.NET worlds since 2005, because they still don't cover all use cases where C++ might be needed.

So while you might not write it as full stack C++, a couple of native libraries might be required as dependency, to access OS features, give some help to the AOT/JIT compilers, or in Java's case implementation of more machine friendly data structures.

Re: Python at Scale: Strict Modules

#186

Post by pjmlp »

adev_ wrote: post #162
WatchDog wrote: post #134

Earlier quoted context omitted.

Why would you pick Go or C++ over Java? C++ is a hydra of complexity, sure it has it's place, but it's not nearly as productive as Java for your typical web application. Go is almost the opposite, so simple it lacks features like generics. The last time I used Go it had fundamental usability issues around dependency management(although I think recent versions have improved on vendoring a little).

> C++ is a hydra of complexity, sure it has it's place, but it's not nearly as productive as Java for your typical web application. Modern C++ well is as productive ( probably even more productive ) as Java. The main issue with C++ is recruitment, C++ engineers are rare because C++ is barely teached.

Any good university has C++ related classes.

The problem with modern C++ is having many devs to actually make use of it.

Many devs don't care to follow up on modern language X best practices, rather just typing away something that works.

Re: Python at Scale: Strict Modules

#187

Post by miki123211 »

bcherny wrote: post #32
miki123211 wrote: post #21

This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're w…

> I wish there was a language that let you move gradually from one end to the other, exactly when you need to. This is precisely what gradually typed languages — like TypeScript, Flow, and typed Pythons — solve! I talked about this on Software Engineering Radio last week: https://www.se-radio.net/2019/10/episode-384-boris-cherny-on... .

It's not just about static typing, though. Macros, metaprogramming, being able to reach as deep as you want to, ugly code full of side effects, global state etc. All of those might actualy benefit you when your project is small, and they make development way faster (see Rails). Later, however, they're a definite impediment.

Re: Python at Scale: Strict Modules

#188

Post by moksly »

timothycrosley wrote: post #18

More and more I want someone to create a new language that amounts to a strict subset of Python, with mypy built-in, and is compilable into machine code. Python has by far my favorite syntax, community, and in my experience leads to the greatest productivity. There just happens to be a lot of overly dynamic features, that aren't even used by most, but used just enough to hold back optimization and structural improvem…

Aside from having a two decade history of using C#, types is the only thing preventing us from going full Python. Even so, the dynamic types in Python are more often a benefit than a disadvantage because Python is so great at handling them automatic.

We build our employee database, and from there our IDM, from a singel XML file in a really shitty format + three txt files in even worse formats (they are single line output files from an old mainframe system predating sap). We used to do it in a rather complicated Microsoft SSIS workflow with a lot of C# services. All in all it’s a 30 minute nightly runtime. I recently replaced it with around 500 lines of Python and a 1-5 minute Runtime (sometimes at the beginning of a school year we’ll see changes to around 1000 positions).

Python eats the XML like it wasn’t shit. It takes things like terrible date formats, we’re talking the output of a SAP free-text box shitty, and ports then seamlessly into a SQL date field. This alone was a nightmare in C# and Python just does it.

Still, after two decades of strict types it feels dangerous.

Re: Python at Scale: Strict Modules

#189

Post by bjoli »

tln wrote: post #147

Avoiding module side effects and making classes and modules immutable seem like two separate concerns

Not really. Mutation in general, and in modules in particular, inhibit a lot of reasoning about the code, and thus stops a whole lot of optimizations from being possible. Guile (a scheme dialect) recently got declarative modules for that reason, where a top level binding cannot change (i.e. you cannot set! a binding, but you can wrap in it a mutable container and change the contents of that container). This makes procedure calls and variable lookup a lot faster. Andy Wingo wrote about it here: https://wingolog.org/archives/2019/06/26/fibs-lies-and-bench... .

Those optimizations won't mean much for cpython, since Cpython doesn't try to run things fast, but for something like pypy this could be a big deal.

Re: Python at Scale: Strict Modules

#190

Post by miki123211 »

DonaldPShimoda wrote: post #103
miki123211 wrote: post #21

This is yet another example of the divide between wizarding and engineering[1]. When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. In short, you're w…

> When you're a small startup, what matters is the expressiveness of your language, and the ability do do a lot of things very very quickly. Type safety, performance, readability, those things don't matter. You're just a bunch of engineers who know the whole codebase inside out, you're pretty certain of what you're doing. I'm not familiar with this use of the term "expressiveness". My understanding is that expressive…

My definition of expressiveness is basically similar to what Paul Graham (@pg) says. He also calls this "powerfulness" in his famous "Beating the averages" essay[1]. In short, a more expressive language is a language that lets you express more with less. Rails, with its "has_many :books" is very expressive, Assembly is the other end.

The wizarding/engineering spectrum was coined by the article I've linked to[2]. I think the post is exactly about that, first Instagram was wizarding and they had a suitable language for wizarding, now they're engineering, but their language is still only good for wizarding.

As I've said in a sister comment, it's not just about static typing, but metaprogramming/macros/side effects everywhere etc. There's more to the expressiveness/powerfulness than just types. While gradual typing is certainly an improvement, I think we need more research in this direction.

[1] http://www.paulgraham.com/avg.html

Post reply on HN