Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster Python with Guido van Rossum

softwareatscale.dev

Post reply on HN
Print view View original

161–170 of 251 posts

Re: Faster Python with Guido van Rossum

#161

Post by overgard »

joconde wrote: post #136
overgard wrote: post #90

Ugh. Python could be fast, but the python core team / GvR put so many ridiculous constraints on what can change that maybe at best it gets like 10% faster, some day. Anything meaningful means breaking compatibility with extensions, and they absolutely won't do that. I feel like GvR got burned in the python 2->3 transition, but he learned the wrong lesson. The lesson he seemed to learn was never break compatibility bu…

There is no way I'm doing anything serious in a language that decides to make us rewrite part of our work regularly. We have actual work to do... I think you underestimate how important stability is.

Not really suggesting any amount of rewriting of python code, just that extensions might need to be recompiled and/or not depend on reaching into internals they shouldn't touch. To users it should mostly be invisible.

Re: Faster Python with Guido van Rossum

#162

Post by throwaway894345 »

joconde wrote: post #158
throwaway894345 wrote: post #148

Earlier quoted context omitted.

I think you could make the same argument about code styling or really just about anything. If you had sufficiently quality coworkers, these things probably wouldn't be an issue, but it's a lot easier to just implement the technical solution (a code formatter or type checker).

Yeah but the technical solution won't save you; laziness usually applies to everything, so they'll find something else to mess up. The technical solution is a very useful way of preventing mistakes by well-intentioned people though.

You seem to think the debate is about whether typing alone can deter lazy employees from wreaking havoc, or otherwise solving broad organizational problems. No one claimed anything like that.

The claim is very narrow: typing can inhibit certain kinds of bad code and guide people toward better solutions. Whether someone is writing bad code because they're lazy or inexperienced, typing keeps some of the bad code from entering the code base.

Of course it's not a replacement for good management, hiring practices, etc.

Re: Faster Python with Guido van Rossum

#163

Post by wassenaar10 »

Qem wrote: post #6
amelius wrote: post #3

Python needs to go multicore, like OCaml. And add a modern concurrent garbage collector. One of the difficulties is of course to not break existing C extensions.

Given Python has the multiprocessing module, I always get confused when people talk about Python lack of support for multicore. What are the shortcomings of the multiprocessing module, that cause people to disregard it?

The performance overhead of multiprocessing is pretty high, to the point that you really have to take care to ensure that whatever problem you're solving with multiprocessing is large and parallel enough to justify it, otherwise your multiprocessing solution may very well be slower than your single-process one.

That's not to say it's useless, it's a nice tool to have in the standard library, but if you're coming from one of the many mainstream programming languages with lightweight threading libraries which make it much easier to take advantage of parallelism it's easy to get frustrated at both python's standard threading and multiprocessing libraries for their respective shortcomings, at least in their CPython implementations.

Re: Faster Python with Guido van Rossum

#164

Post by m_mueller »

ithrow wrote: post #160
BiteCode_dev wrote: post #143

Earlier quoted context omitted.

A trap for your dream world were you suddenly get google size ? Because I have a 1 million unique users video streaming service still running python 2.7, using a few servers. The thing has a mobile version serving a different media on the fly, encodes user uploaded videos, features comments, tagging, and even has machine learning detection of content now. It is still maintained by one single person, and he is not a p…

Even faster for 0->1, use Javascript, big ecosystem as Python but same language in front-end and back-end. Faster runtime also.

Depends. Mainly on what libraries you can make use of. For anything numerical or also ETL processes that deal with various document formats python is way more mature.

Re: Faster Python with Guido van Rossum

#165

Post by bilater »

ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

uhmm Instagram runs on Django

Re: Faster Python with Guido van Rossum

#166

Post by max-ibel »

ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

As a counterpoint:

Micropython seems to make embedded code more readable and bug free than Arduino C-based programs, judging from a few projects I looked into.

Yes, it would probably be a mistake to write a large distributed, high performance database (say, for a login system handling billions of users) in python, but many, if not most, projects never hit the wall you are referring to.

In that sense, I feel that writing something in the most scalable language (if such a thing existed) would just be premature optimization at the highest level.

Re: Faster Python with Guido van Rossum

#167

Post by ithrow »

m_mueller wrote: post #164
ithrow wrote: post #160

Earlier quoted context omitted.

Even faster for 0->1, use Javascript, big ecosystem as Python but same language in front-end and back-end. Faster runtime also.

Depends. Mainly on what libraries you can make use of. For anything numerical or also ETL processes that deal with various document formats python is way more mature.

Since you don't know if your product will workout and you want to go fast you use a third-party service first. Of course, for some type of stuff, you might have to go with python but if both can equally do the job well in the backend and you need a web[mobile] frontend the...

Re: Faster Python with Guido van Rossum

#168

Post by willcipriano »

BiteCode_dev wrote: post #143
ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

A trap for your dream world were you suddenly get google size ? Because I have a 1 million unique users video streaming service still running python 2.7, using a few servers. The thing has a mobile version serving a different media on the fly, encodes user uploaded videos, features comments, tagging, and even has machine learning detection of content now. It is still maintained by one single person, and he is not a p…

I haven't worked at Google's scale, but I've heard a lot of critism about tools and programming languages from people who create architectures that are inherently slow. When a user login requires 8 http calls to distinct microservices or something else silly like that the problem isn't a lack of static types. So far in my career I haven't had a programming language ever be the reason why something can't scale, most often the problem is the initial developers lacking a understanding of relational databases and cache locality.

Re: Faster Python with Guido van Rossum

#169

Post by wenc »

prionassembly wrote: post #152
ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

I recently learned just enough Rust to use its ndarray crate and rewrite bottleneck bits of code. I used to think the whole "write inner loops in $fastlang, use $scriptlang as glue" was something of a lame "cope" (and I say this as someone who doesn't know $fastlang), but I'm seeing upwards of 100X time performance improvements for code that can't be hammered into numpy idioms. I can provide code examples. Edit: code…

Thanks for providing code. I’ve written code like that before and the first thing that jumps out at me is that you have for loops in Python code which isn’t slow but will never be as fast as loops in a static compiled language. Sometimes for loops are the most readable way of expressing a calculation but as a numerical computation person my instinct is to look for opportunities to vectorize by rewriting loops into matrix notation on paper and then expressing them as array calculations.

Otherwise you’re right — whenever you have “for” loops you’re always going to end up ahead if you rewrite those in C, Rust, Julia or Fortran. And that’s exactly what authors of high performance numerical codes do.

Re: Faster Python with Guido van Rossum

#170

Post by takeda »

Cthulhu_ wrote: post #151
BiteCode_dev wrote: post #143

Earlier quoted context omitted.

A trap for your dream world were you suddenly get google size ? Because I have a 1 million unique users video streaming service still running python 2.7, using a few servers. The thing has a mobile version serving a different media on the fly, encodes user uploaded videos, features comments, tagging, and even has machine learning detection of content now. It is still maintained by one single person, and he is not a p…

It's a variation on cargo cult; you WISH you had the scaling problems that could be solved by using something a bit more performant. But solve the problem first, and solve it fast. It's why so many startups from the 2010's on used Ruby; they were productive in it, solving a real problem, rolling out features fast. I've worked in a few projects where they dove onto the cargo cult, of wishing they had the kind of reque…

Also, a lot of developers forget to it, but even if you start with a fast language down the line you will end up rewriting it. The problem is that requirements change a lot and because of that your application will often end up being a mess. For similar reason RDBMS is also better than NoSQL when starting a project.

When rewriting you already know how the project is now, so you can design your app around that, which makes things simpler and faster.

Post reply on HN