Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Python keeps growing, explained

github.blog

Post reply on HN
Print view View original

391–400 of 459 posts

Re: Why Python keeps growing, explained

#391

Post by _aavaa_ »

dragonwriter wrote: post #357
_aavaa_ wrote: post #272

Earlier quoted context omitted.

Have gone through the exercise, I know this is false. Not everything can be pushed into numpy, and you can still be left with lots of loops in python.

That's what numba [0] is for (can also help with the NumPy stuff in certain cases.) [0] https://numba.pydata.org/

Oh, I'm familiar with numba and while it certainly helps, it has plenty of it's own issues. You don't always get a performance gain and you only find this out at the end of a refactoring. Your code can get less readable if you need to transport data in and out of formats that it's compatible with (looking at you List()).

To say nothing of adding yet another long dependancy chain to the language (python 3.11 is still not supported even though work started in Aug of last year).

I do wonder if the effort put into making this slow language fast could have been put to better use, such as improving a language with python's ease of use but which was build from the beginning with performance in mind.

Re: Why Python keeps growing, explained

#392

Post by syntheweave »

di456 wrote: post #257
Yoric wrote: post #214

Earlier quoted context omitted.

I fully agree with the description. What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about…

> the features that make Python quite good at prototyping make it rather bad at auditing for safety and security What's an example that makes it bad? Is it a case of the wrong tool for the job? For example I understand that garbage collection languages shouldn't be used with real time systems like flight controllers.

What audits need most is some ability to analyze the system discretely and really "take it apart" into pieces that they can apply metrics of success or failure to(e.g. pass/fail for a coding style, numbers of branches and loops, when memory is allocated and released).

Python is designed to be highly dynamic and to allow more code paths to be taken at runtime, through interpreting and reacting to the live data - "late binding" in the lingo, as opposed to the "early binding" of a Rust or Haskell, where you specify as much as you can up front and have the compiler test that specification at build time. Late binding creates an explosion of potential complexity and catastrophic failures because it tends to kick the can down the road - the program fails in one place, but the bug shows up somewhere else because the interpreter is very permissive and assumes what you meant was whatever allows the program to continue running, even if it leads to a crash or bad output later.

Late binding is very useful - we need to assume some of it to have a live, interactive system instead of a punchcard batch process. And writing text and drawing pictures is "late binding" in the sense of the information being parsed by your eyes rather than a machine. But late binding also creates a large surface area where "anything can happen" and you don't know if you're staying in your specification or not.

Re: Why Python keeps growing, explained

#393

Post by _aavaa_ »

slt2021 wrote: post #355
dagw wrote: post #163

Earlier quoted context omitted.

I've rewritten real world performance critical numpy code in C and easily gotten 2-5x speedup on several occasions, without having to do anything overly clever on the C side (ie no SIMD or multiprocessing C code for example).

2-5x speedup is not a lot, I would say it is not worth it to rewrite from py to C if you don't have an order of magnitude improvement. Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left

2-5x IS a lot. It's the speed difference between the current iPhone 14 and an iPhone XS-iPhone 6. That's 4-8 years of hardware improvements.

And the parent was talking about numpy code which is better than stock python, who knows how far back normal python would send you.

Re: Why Python keeps growing, explained

#394

Post by hot_gril »

kyawzazaw wrote: post #387
brundolf wrote: post #329

Earlier quoted context omitted.

It's really funny that one of the subheadings under "Why is Python so popular?" is "It has high corporate demand."

That's a perfectly relevant thing, no?

Yes. The entire reason I like using Java at my day job is that the rest of the company uses it the most and supports it well. I would never use Java on my own, but that's a different situation.

Re: Why Python keeps growing, explained

#395

Post by _aavaa_ »

pbourke wrote: post #283
dagw wrote: post #163

Earlier quoted context omitted.

I've rewritten real world performance critical numpy code in C and easily gotten 2-5x speedup on several occasions, without having to do anything overly clever on the C side (ie no SIMD or multiprocessing C code for example).

I’ve done the same but moved from vanilla numpy to numba. The code mostly stayed the same and it took a couple hours vs however long a port to C or Rust would have taken.

For a package whose pitch is "Just apply one of the Numba decorators to your Python function, and Numba does the rest." a few hours of work is a long time.

Re: Why Python keeps growing, explained

#396

Post by hot_gril »

samwillis wrote: post #31

I have been a heavy Python user now about 15 years, but for me now I'm increasingly reaching for modern JavaScript and particularly TypeScript to do the things I would have traditionally done with Python. ES modules, fat arrow expressions, and all the other nice new syntax and library features have made the language so more pleasant to use. In many ways the ergonomics of TypeScript in particular are far superior to P…

Same. I analyzed a little why I find JS (I don't like TS) easier to deal with for some tasks:

- Particular support for web frontends or backends, both very broad categories.

- JS concurrency is easier to deal with. Focused entirely on promises with the nice async/await syntax on top, unlike Python which slapped on too many different ways to do this.

- By far easier package management and imports. Python's is so annoying that any project you download is gonna have you spin up a Docker container for it.

- ES6 added a lot of array/etc managing that JS was lacking before.

- Freeform objects with {key: value} syntax are convenient, despite maybe seeming weird at first. Python OOP somehow got really complicated over the years.

- Inline functions (I use fat-arrow but regular way is also fine). I never got why Python, despite being common in function-oriented programming, didn't let you do an inline def.

- Curly braces. Indentation shouldn't affect logic flow.

- No Python3 vs 2 drama.

Specific use cases have been Express backends with node-postgres and lots of SQL, and React (or RN) frontends.

Re: Why Python keeps growing, explained

#397

Post by hot_gril »

fredley wrote: post #41
samwillis wrote: post #31

I have been a heavy Python user now about 15 years, but for me now I'm increasingly reaching for modern JavaScript and particularly TypeScript to do the things I would have traditionally done with Python. ES modules, fat arrow expressions, and all the other nice new syntax and library features have made the language so more pleasant to use. In many ways the ergonomics of TypeScript in particular are far superior to P…

I am going this way too. I recently tried to onboard a few contractors with limited python experience. It took several deep sessions to work out why they couldn't get an environment set up. After 10+ years I've never come across the install certificates command. There's still no way to get a dev environment with a specific version of Python working with one command that works reliably everywhere. pyenv isn't even pac…

> There's still no way to get a dev environment with a specific version of Python working with one command that works reliably everywhere.

Closest thing to this is Docker nowadays. It's so annoying. npm just works.

Re: Why Python keeps growing, explained

#398

Post by jononomo »

dagw wrote: post #163
ofrzeta wrote: post #156

Earlier quoted context omitted.

I'd say if you do data-intenstive computation with Numpy you are not leaving much on the table due to Python.

I've rewritten real world performance critical numpy code in C and easily gotten 2-5x speedup on several occasions, without having to do anything overly clever on the C side (ie no SIMD or multiprocessing C code for example).

2-5x speedup barely seems worth re-writing something for, unless we're talking calculations that take literally days to complete, or you're working on the kernel of some system that is used by millions of people.

Re: Why Python keeps growing, explained

#399

Post by jononomo »

_aavaa_ wrote: post #393
slt2021 wrote: post #355

Earlier quoted context omitted.

2-5x speedup is not a lot, I would say it is not worth it to rewrite from py to C if you don't have an order of magnitude improvement. Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left

2-5x IS a lot. It's the speed difference between the current iPhone 14 and an iPhone XS-iPhone 6. That's 4-8 years of hardware improvements. And the parent was talking about numpy code which is better than stock python, who knows how far back normal python would send you.

I'm in the camp that 2-5x performance improvement is not really worth re-writing Python code in C for.

Re: Why Python keeps growing, explained

#400

Post by sethrin »

mixmastamyk wrote: post #322
sethrin wrote: post #313

Earlier quoted context omitted.

The most similar language to Python is Ruby, not Rust or Go, and Ruby is better at the things you've listed. Which would bring us back to the previous point that it's not actually about ease of use. Someone else identified Ruby's fatal flaw as not having good C tooling, and I think that's probably accurate.

Ruby has odd, once unique syntax that looks strange to anyone raised on C-like syntax, including js and java. Python is similar, simply removing redundant braces. So no ruby was not better at things most folks care about, i.e. being easy to learn.

You seem to have confused personal familiarity with ease of learning.
Post reply on HN