Faster Python with Guido van Rossum • Hacker News Board

Re: Faster Python with Guido van Rossum

#111

Post by arc619 » Thu, Oct 07, 2021, 4:57 PM UTC

amelius wrote: post #48

dom96 wrote: post #40

Earlier quoted context omitted.

If anybody is looking for an alternative to Python that is also a great 0->1 language but doesn't have the same wall as Python described here, check out Nim[1]. 1 - https://nim-lang.org

People's main reason for using Python is the ecosystem.

Nim can use Python's ecosystem in both directions with nimpy https://github.com/yglukhov/nimpy and nimporter https://github.com/yglukhov/nimpy

This lets you gradually transition hot path Python modules to Nim, get compiled performance generally on par with C and Rust, whilst enjoying strong, static typing with great type inference.

In my experience (6-7 years 4 of which are full time) Nim strikes the perfect balance of the productivity you get with Python with high performance at the same time.

Also the metaprogramming features are incredible and, importantly, don't use a language subset but use the base Nim language itself.

Re: Faster Python with Guido van Rossum

#112

Post by jerf » Thu, Oct 07, 2021, 4:58 PM UTC

alex_smart wrote: post #99

ferdowsi wrote: post #62

Earlier quoted context omitted.

My understanding is that Dropbox has been building performance-critical parts of their services in Go since 2014 at the least. https://twitter.com/jamwt/status/629727590782099456 Like many companies, they had initial scaling successes with Python,hit performance bottlenecks and looked for a way out with faster languages.

That you can incrementally build performance-critical parts of your system in other languages is a huge success story for Python. I don't understand in what world you can view that as a negative. If you were NOT able to do that, that would have actually been a reason to not build your app in Python.

"That you can incrementally build performance-critical parts of your system in other languages is a huge success story for Python."

Even 5-10 years ago, that's not a "huge success story"; that's table stakes for any serious language.

Re: Faster Python with Guido van Rossum

#113

Post by igouy » Thu, Oct 07, 2021, 4:59 PM UTC

giovannibonetti wrote: post #46

indymike wrote: post #35

Earlier quoted context omitted.

> This means performance will fall further and further behind compiled languages. This is exactly as it should be: the nature of the interpreter is that it has to do a lot of the work a compiler does when it compiles whenever a new command is run (and in Python's case, that is before you cover the whole "everything is an object" part). Yes, some of it can be mitigated, but the expectation that an interpreter keep pac…

> The developer experience with an interpreter is fantastic, By the way, there are some (high-level) compiled languages with REPLs like Lisp, Haskell and Elixir.

"GHC compiles Haskell code either directly to native code or using LLVM as a back-end."

Does elixirc compile to "native code" ?

Does the Haskell REPL use an interpreter not the GHC compiler?

http://downloads.haskell.org/~ghc/latest/docs/html/users_gui...

Re: Faster Python with Guido van Rossum

#114

Post by jerf » Thu, Oct 07, 2021, 4:59 PM UTC

amelius wrote: post #44

jerf wrote: post #32

Earlier quoted context omitted.

As a sibling mentions, the multiprocessing support is basically support for spawning new processes. It's better than nothing, but it's not very good for the performance. The real problem with Python multicore is that the main problem is solves is the one ctur is talking about, namely, "Oh crap, I used Python and it doesn't run fast enough for my needs... maybe I can run more processes?" Using a language that is alrea…

> Python is now only incrementally better than modern static languages in some dimensions The main benefit of Python now over those newer languages is its ecosystem.

It isn't just "newer languages". Many older languages have picked up over the years.

Re: Faster Python with Guido van Rossum

#115

Post by amelius » Thu, Oct 07, 2021, 5:00 PM UTC

arc619 wrote: post #111

amelius wrote: post #48

Earlier quoted context omitted.

People's main reason for using Python is the ecosystem.

Nim can use Python's ecosystem in both directions with nimpy https://github.com/yglukhov/nimpy and nimporter https://github.com/yglukhov/nimpy This lets you gradually transition hot path Python modules to Nim, get compiled performance generally on par with C and Rust, whilst enjoying strong, static typing with great type inference. In my experience (6-7 years 4 of which are full time) Nim strikes the perfect balance…

That's interesting but it looks like Nim has no support for Qt yet (where Python has PyQt and PySide).

Re: Faster Python with Guido van Rossum

#116

Post by arc619 » Thu, Oct 07, 2021, 5:00 PM UTC

arc619 wrote: post #111

amelius wrote: post #48

Earlier quoted context omitted.

People's main reason for using Python is the ecosystem.

Nim can use Python's ecosystem in both directions with nimpy https://github.com/yglukhov/nimpy and nimporter https://github.com/yglukhov/nimpy This lets you gradually transition hot path Python modules to Nim, get compiled performance generally on par with C and Rust, whilst enjoying strong, static typing with great type inference. In my experience (6-7 years 4 of which are full time) Nim strikes the perfect balance…

Oops wrong link for nimporter. Actual link: https://github.com/Pebaz/Nimporter

Re: Faster Python with Guido van Rossum

#117

Post by agumonkey » Thu, Oct 07, 2021, 5:02 PM UTC

ferdowsi wrote: post #62

nickjj wrote: post #50

Earlier quoted context omitted.

> I have become increasingly convinced Python as a language is a "trap" for any use of notable scal At what point do you embrace the trap and happily live with it? For example Dropbox runs many millions of lines of Python, has massive traffic and their server costs aren't completely out of line for the service they offer. Their code base is also over 10 years old. It seems to be working very well for them. I talked t…

My understanding is that Dropbox has been building performance-critical parts of their services in Go since 2014 at the least. https://twitter.com/jamwt/status/629727590782099456 Like many companies, they had initial scaling successes with Python,hit performance bottlenecks and looked for a way out with faster languages.

i find this pattern utterly funny

it's always the same

prototype in an easy language

tighten the bolts closer to the metal after some time

Re: Faster Python with Guido van Rossum

#118

Post by sorokod » Thu, Oct 07, 2021, 5:04 PM UTC

nicolaslem wrote: post #7

ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

Isn't it that maintaining large codebases in the long run is just hard? What language is not a trap?

Digging a well is hard, doesn't mean you have to this with a spoon.

Re: Faster Python with Guido van Rossum

#119

Post by fulafel » Thu, Oct 07, 2021, 5:05 PM UTC

amelius wrote: post #10

Qem wrote: post #6

Earlier quoted context omitted.

Given Python has the multiprocessing module, I always get confused when people talk about Python lack of support for multicore. What are the shortcomings of the multiprocessing module, that cause people to disregard it?

Python's multiprocessing module forces you to serialize all your data, or use ctypes. This means it's either going to be slow, or it's like writing code in a hybrid of Pythonic and C-style code. Which of course goes against the idea of using the simplest tool for the job.

You can use shared memory without ctypes: https://docs.python.org/3/library/multiprocessing.shared_mem...

But you still do have to jump through some hoops then. But like the example in the docs show there are shareable primitives, you can share numpy arrays, etc.

Re: Faster Python with Guido van Rossum

#120

Post by alex_smart » Thu, Oct 07, 2021, 5:06 PM UTC

jerf wrote: post #112

alex_smart wrote: post #99

Earlier quoted context omitted.

That you can incrementally build performance-critical parts of your system in other languages is a huge success story for Python. I don't understand in what world you can view that as a negative. If you were NOT able to do that, that would have actually been a reason to not build your app in Python.

"That you can incrementally build performance-critical parts of your system in other languages is a huge success story for Python." Even 5-10 years ago, that's not a "huge success story"; that's table stakes for any serious language.

Sure, but that is the feature that enables people to build apps in slow, interpreted languages like Python in the first place. The parent comment had somehow made it sound like that was a negative.

I did not mean "huge win" as in it is something unique to Python. I meant it in the sense that it greatly adds to the value proposition of the language.