Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster Python with Guido van Rossum

softwareatscale.dev

Post reply on HN
Print view View original

151–160 of 251 posts

Re: Faster Python with Guido van Rossum

#151

Post by Cthulhu_ »

BiteCode_dev wrote: post #143
ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

A trap for your dream world were you suddenly get google size ? Because I have a 1 million unique users video streaming service still running python 2.7, using a few servers. The thing has a mobile version serving a different media on the fly, encodes user uploaded videos, features comments, tagging, and even has machine learning detection of content now. It is still maintained by one single person, and he is not a p…

It's a variation on cargo cult; you WISH you had the scaling problems that could be solved by using something a bit more performant.

But solve the problem first, and solve it fast. It's why so many startups from the 2010's on used Ruby; they were productive in it, solving a real problem, rolling out features fast.

I've worked in a few projects where they dove onto the cargo cult, of wishing they had the kind of requests and load that e.g. a microservices architecture might help with. Massively overpriced projects too, because they hired lots of consultants and self-employed people.

The one time there was a guy who rocked up in a Maserati (used, lol); he took two weeks to build a page that was just a centered bit of text and a button, and it didn't even work.

Re: Faster Python with Guido van Rossum

#152

Post by prionassembly »

ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

I recently learned just enough Rust to use its ndarray crate and rewrite bottleneck bits of code. I used to think the whole "write inner loops in $fastlang, use $scriptlang as glue" was something of a lame "cope" (and I say this as someone who doesn't know $fastlang), but I'm seeing upwards of 100X time performance improvements for code that can't be hammered into numpy idioms. I can provide code examples.

Edit: code example https://gist.github.com/asemic-horizon/2830ed3637cfd278e7937...

Re: Faster Python with Guido van Rossum

#153

Post by munificent »

wil421 wrote: post #14
ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

Do you have the opposite experience? A language used by FAANG that doesn’t have performance issues at their scale. They seem to invest a lot in JavaScript which is not what I’d call a performant language and as you’ve said Python. I’ve used Python for more scripting type tasks and creating basic APIs to get specific jobs done. I’m very weary to go “all in” on Python for an app based on all the feedback about performa…

> They seem to invest a lot in JavaScript which is not what I’d call a performant language

A lot of them have huge amounts of code that runs in browsers, so they essentially have to invest in JS whether they like it or not.

> All that being said I’ve never heard someone say we used X language and it scaled remarkably without issues.

Sometimes you have to go by what people don't say. Humans tend to talk most about problems or surprises. The former are actionable and the latter are more interesting to talk about because of their rarity. That means when things work well as expected, you get silence.

I remember when people constantly talked about how Java was too slow. Then there was a period of time where people talked a lot about how Java was fast enough because that was an interesting change. Now people don't talk about Java performance much at all, which is a good sign (but easy to overlook), that Java performance is now consistently reliably good for most users.

Re: Faster Python with Guido van Rossum

#154

Post by zerkten »

kjksf wrote: post #129
nickjj wrote: post #72

Earlier quoted context omitted.

Sure, Rust too for some of the desktop app components. But the takeaway is you only need to do that when you're operating at mega scale and actually hit these bottlenecks. In a ton of cases you'll be completely fine serving a few million monthly page views of your SAAS app on a single $20-40 a month server using Flask, Django or whatever Python web framework you prefer. Performance will be really good too. Just talki…

I'm a solo dev. I write all my backend code in Go (used Python before Go existed) because I can deploy Go executable on the cheapest render.com server and it runs using fraction of the available resources (~50 MB out of 512 MB available and literally 0.01% CPU use). If I used Python (or Ruby, node or even Java) I would likely have to go higher (and pay more) because those languages are not only slow, they are also me…

Microscopic cost optimization is frequently not a problem, but it's great if you can get it for low effort with Go. It's highly contextual whether it'll matter that much in both the short and long-term. You're situation as a solo dev is special and can't be easily extrapolated to efforts involving many people. The things that make solo dev great are hinderances to collaborative work efforts (by design in many cases.)

As a frame of reference, ASP.NET services in C# can exceed the throughput and performance of an equivalent Django app, but Django (and Python) may be the best choice for some people. What's not considered here is the experience of the developer and whether they can avail of the platform capabilities fast. Do they need to rewrite libraries? How much work is needed to be as proficient as you are with Go? The language and platform choice is rarely a differentiator here.

Over the long-term you can see some patterns. Python companies are able to scale up their teams more quickly and cheaply in some markets than Go ones. You choose to optimize hosting cost while they optimize other costs. They may need to avail of libs like Pandas and these are readily available. I don't know what the situation is with Go today, but similar libs weren't available originally and the Go community didn't have a big foothold in data science.

Re: Faster Python with Guido van Rossum

#155

Post by uyt »

overgard wrote: post #90

Ugh. Python could be fast, but the python core team / GvR put so many ridiculous constraints on what can change that maybe at best it gets like 10% faster, some day. Anything meaningful means breaking compatibility with extensions, and they absolutely won't do that. I feel like GvR got burned in the python 2->3 transition, but he learned the wrong lesson. The lesson he seemed to learn was never break compatibility bu…

> nor will they ever endorse something like PyPy

What's their beef with PyPy?

Re: Faster Python with Guido van Rossum

#156

Post by igouy »

CraigJPerry wrote: post #130
Mikeb85 wrote: post #104

Earlier quoted context omitted.

Did you look at the source of the programs where Python was competitive? Python basically using C types and GMP to write C in Python...

I suspect that site is some kind of in-joke. E.g. read the regex-redux example in say c#, now look at the go version, now the python version… It’s not really what you’d expect for a site called computer language benchmarks game. Each example just calls out to a very fast c library (pcre2) to perform the heavy lifting regardless of which language is being “benchmarked”. Seems a pretty pointless site. The other example…

> Each example just calls out to a very fast c library…

No.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Re: Faster Python with Guido van Rossum

#157

Post by overgard »

Joker_vD wrote: post #103
overgard wrote: post #90

Ugh. Python could be fast, but the python core team / GvR put so many ridiculous constraints on what can change that maybe at best it gets like 10% faster, some day. Anything meaningful means breaking compatibility with extensions, and they absolutely won't do that. I feel like GvR got burned in the python 2->3 transition, but he learned the wrong lesson. The lesson he seemed to learn was never break compatibility bu…

The interpreter that breaks the backward compatibility makes that code run infinitely slower (i.e. it does not run at all) instead of running faster. And it doesn't matter if the compatibility was broken for some thing you actually care about: if your code can't run at all until the broken third-party dependency is fixed by its maintainer, you simply won't upgrade, will you?

I work in the Node ecosystem, and while they break compatibility every once in a while I can't say I've ever even really noticed. There's a way to do it without it being a catastrophy.

Re: Faster Python with Guido van Rossum

#158

Post by joconde »

throwaway894345 wrote: post #148
joconde wrote: post #132

Earlier quoted context omitted.

If a person like that is able to harm the team, I'd argue that it's a management problem. Lazy coding is only an issue when you allow people to be lazy...

I think you could make the same argument about code styling or really just about anything. If you had sufficiently quality coworkers, these things probably wouldn't be an issue, but it's a lot easier to just implement the technical solution (a code formatter or type checker).

Yeah but the technical solution won't save you; laziness usually applies to everything, so they'll find something else to mess up.

The technical solution is a very useful way of preventing mistakes by well-intentioned people though.

Re: Faster Python with Guido van Rossum

#159

Post by overgard »

uyt wrote: post #155
overgard wrote: post #90

Ugh. Python could be fast, but the python core team / GvR put so many ridiculous constraints on what can change that maybe at best it gets like 10% faster, some day. Anything meaningful means breaking compatibility with extensions, and they absolutely won't do that. I feel like GvR got burned in the python 2->3 transition, but he learned the wrong lesson. The lesson he seemed to learn was never break compatibility bu…

> nor will they ever endorse something like PyPy What's their beef with PyPy?

As far as I can tell, it's because it's not based on CPython. For whatever reason, to GvR Python == CPython, other interpreters don't matter. It's a shame, because python the language is fine, but CPython the interpreter is janky and slow.

Re: Faster Python with Guido van Rossum

#160

Post by ithrow »

BiteCode_dev wrote: post #143
ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

A trap for your dream world were you suddenly get google size ? Because I have a 1 million unique users video streaming service still running python 2.7, using a few servers. The thing has a mobile version serving a different media on the fly, encodes user uploaded videos, features comments, tagging, and even has machine learning detection of content now. It is still maintained by one single person, and he is not a p…

Even faster for 0->1, use Javascript, big ecosystem as Python but same language in front-end and back-end. Faster runtime also.
Post reply on HN