Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster Python with Guido van Rossum

softwareatscale.dev

Post reply on HN
Print view View original

241–250 of 251 posts

Re: Faster Python with Guido van Rossum

#241

Post by joconde »

overgard wrote: post #161
joconde wrote: post #136

Earlier quoted context omitted.

There is no way I'm doing anything serious in a language that decides to make us rewrite part of our work regularly. We have actual work to do... I think you underestimate how important stability is.

Not really suggesting any amount of rewriting of python code, just that extensions might need to be recompiled and/or not depend on reaching into internals they shouldn't touch. To users it should mostly be invisible.

> extensions might need to [...] not depend on reaching into internals they shouldn't touch

Is that the problem? I thought I read somewhere that the C/C++ extension API would have to be broken to allow for removing the GIL, for example. The API, not just internals that shouldn't be touched by user code.

If it were only a rebuild, I don't think anyone would object; pip packages are built once per Python version anyway. Breaking the API would be a different story for libraries that make heavy use of C++ extensions.

Re: Faster Python with Guido van Rossum

#242

Post by joconde »

throwaway894345 wrote: post #162
joconde wrote: post #158

Earlier quoted context omitted.

Yeah but the technical solution won't save you; laziness usually applies to everything, so they'll find something else to mess up. The technical solution is a very useful way of preventing mistakes by well-intentioned people though.

You seem to think the debate is about whether typing alone can deter lazy employees from wreaking havoc, or otherwise solving broad organizational problems. No one claimed anything like that. The claim is very narrow: typing can inhibit certain kinds of bad code and guide people toward better solutions. Whether someone is writing bad code because they're lazy or inexperienced, typing keeps some of the bad code from e…

> typing can inhibit certain kinds of bad code and guide people toward better solutions

I argue that somebody who would try something like your example above is either lazy or trying to wreak havoc. In my experience, people like that won't be deterred by strict typing, they'll keep typing nonsense and copying dangerous lines from StackOverflow until the thing outputs what they want. Having them write code that's complicated or important enough to require typing checks will result in disaster anyway, the only solution is to keep them out of the codebase, or review (and often rewrite) all their work.

I agree that typing is a very good guardrail for well-intentioned people with a minimum of competence, but in that case there's nothing to lose by making typing optional: they'll follow it anyway, and the added flexibility is often useful when well-used.

An inexperienced dev who wants to learn also won't need to be forced to follow typing guidelines, we just have to explain it to them...

Re: Faster Python with Guido van Rossum

#243

Post by mixmastamyk »

CraigJPerry wrote: post #138

A question i’ve often wondered about is how much slower could you make python before users would leave it behind? Python is often 1-2 orders of magnitude slower than java. That really is a lot yet Python and Java are about comparable in adoption.

I’ve never had an issue with a Python performance, not in twenty years. Possibly because of using it for what it’s good at? Every year it gets faster as well. I did have an issue with the performance of Java twenty years ago, but that was a long time ago of course. Computers at an order of magnitude faster now and java startup improved.

Re: Faster Python with Guido van Rossum

#244

Post by m4rtink »

bborud wrote: post #235
m4rtink wrote: post #232

Earlier quoted context omitted.

Good point about scientists being quite terrible and software mainteinability - I've encountered it quite a few times myself and from what I've heard its not much better elsewhere. I blame the publish-or-perish mechanics and grant approval process not actually motivating the participants in any way to write maintainable or reusable code. Like, I don't say they should make it good enough for industry to consume, but a…

I see your point about using shared libraries to fix problems in multiple applications at once, but there is a flip side to that: what you're running isn't what you built so you may not be able to predict what your software will do when installed on different users computers.

That's true - but I'm not sure one can always 100% assure the environment is the same - or even should!

Ideally one should make the software as robust as possible so that it runs with high probability even in unforeseen environments - and if it fails, it should do that noticeably so that the developers will learn about it and can fix it.

It pretty much works like this in Fedora - lot of the various API breakages and regressions show up when the bits and pieces first land in the rolling distro called Rawhide, but thats fine as hardly anyone runs Rawhide as a daily driver. By the time the next Fedora release is branched from Rawhide and goes via Beta and RC all these issues are ironed out and the end result is pretty stable yet very up to date.

It's usually all the proprietary or heavy bundling software that have the biggest problems with dependency updates as they don't go with the flow or avoid updating the dependency for so long to end up with an insane jump to do.

Re: Faster Python with Guido van Rossum

#245

Post by KronisLV »

igouy wrote: post #236
KronisLV wrote: post #223

Earlier quoted context omitted.

In this context I believe that if a benchmark does exactly that your software would do in a production environment (shuffling JSON around and accessing a DB), then such a benchmark is probably a rough indicator of how well any stack would work, as long as the source code is also written in an idiomatic fashion. Of course, if you can, take benchmarks with a grain of salt and ideally do some prototyping and load testin…

> … does exactly that your software would do… Is "exactly" really the situation, or is it that the benchmark does something kind-of like one of the use cases? https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

In the case of the TechEmpower benchmarks, i'd argue that it's actually pretty much the same thing as real world applications: https://github.com/TechEmpower/FrameworkBenchmarks

For example, look at a full platform Java Spring Boot example here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/mast...

The controller code, the model code and even the repository code are all very reflective of the primitives you'd find in production code.

The only differences would be when you add additional complexity, such as validations and other code into the mix, such as additional service calls etc., but if even the simpler stuff in Python (as an example) would be noticeably slower, then it stands to reason that it wouldn't get magically faster on the basis of less overhead alone.

The exceptions to this which i could imagine would be calls to native code, since the aforementioned Python has an amazing number crunching or machine learning support, but that is hardly relevant to web development in most cases.

Otherwise we risk running into the "No true Scotsman" fallacy: https://en.wikipedia.org/wiki/No_true_Scotsman

In such a case, we'd adamantly claim that these benchmarks are simply not good enough even for educated guesses and we'd have to build out our full system to benchmark it, which no one actually has the resources for - in practice you'd develop a system and then would have to do a big rewrite, as was the case with PHP and Facebook, as well as many other companies.

On the other hand, premature optimization and dogmatic beliefs are the opposite end of the spectrum: you don't need to write everything in Rust if using Ruby would get your small application out the door in a more reasonable amount of time.

There are no easy answers, everything depends on the context (e.g. small web dev shop vs an enterprise platform to be used by millions) and the concerns (e.g. resource usage and server budget vs being the first to market, as well as developer knowledge of tech stacks, for example market conditions of PHP vs Rust).

Regardless, in my eyes, there's definitely a lot of value in these attempts to compare idiomatic interpretations of common logic for web development primitives (serving network requests in a known set of formats).

Re: Faster Python with Guido van Rossum

#246

Post by throwaway894345 »

joconde wrote: post #242
throwaway894345 wrote: post #162

Earlier quoted context omitted.

You seem to think the debate is about whether typing alone can deter lazy employees from wreaking havoc, or otherwise solving broad organizational problems. No one claimed anything like that. The claim is very narrow: typing can inhibit certain kinds of bad code and guide people toward better solutions. Whether someone is writing bad code because they're lazy or inexperienced, typing keeps some of the bad code from e…

> typing can inhibit certain kinds of bad code and guide people toward better solutions I argue that somebody who would try something like your example above is either lazy or trying to wreak havoc. In my experience, people like that won't be deterred by strict typing, they'll keep typing nonsense and copying dangerous lines from StackOverflow until the thing outputs what they want. Having them write code that's comp…

> An inexperienced dev who wants to learn also won't need to be forced to follow typing guidelines, we just have to explain it to them..

Any engineer who would be qualified to write these kinds of guidelines would tell you not to waste time drafting and enforcing guidelines because type checkers exist. And if they're a really good engineer, they'll politely rebuke you for attributing poor code quality to the author's moral character, as this is a self-limiting outlook and generally toxic behavior.

Re: Faster Python with Guido van Rossum

#247

Post by igouy »

KronisLV wrote: post #245
igouy wrote: post #236

Earlier quoted context omitted.

> … does exactly that your software would do… Is "exactly" really the situation, or is it that the benchmark does something kind-of like one of the use cases? https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

In the case of the TechEmpower benchmarks, i'd argue that it's actually pretty much the same thing as real world applications: https://github.com/TechEmpower/FrameworkBenchmarks For example, look at a full platform Java Spring Boot example here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/mast... The controller code, the model code and even the repository code are all very reflective of the primitives you…

(Incidentally, thank you for the pleasant conversation.)

> … which no one actually has the resources for…

That webpage references "JSMeter: Characterizing Real-World Behavior of JavaScript Programs".

iirc they instrumented a web browser to collect "real-world" data and demontrated that wasn't like the behavior of benchmark code.

That webpage references "A comparison of three programming languages for a full-fledged next-generation sequencing tool" — "reimplemented … in all three languages and benchmarked their runtime performance and memory use."

That's in-practice.

> … you don't need to write everything in Rust if using Ruby…

Performance doesn't matter until it matters.

As-in the home page quote and reference — "It's important to be realistic: most people don't care about program performance most of the time."

https://benchmarksgame-team.pages.debian.net/benchmarksgame/

Re: Faster Python with Guido van Rossum

#248

Post by bborud »

m4rtink wrote: post #244
bborud wrote: post #235

Earlier quoted context omitted.

I see your point about using shared libraries to fix problems in multiple applications at once, but there is a flip side to that: what you're running isn't what you built so you may not be able to predict what your software will do when installed on different users computers.

That's true - but I'm not sure one can always 100% assure the environment is the same - or even should! Ideally one should make the software as robust as possible so that it runs with high probability even in unforeseen environments - and if it fails, it should do that noticeably so that the developers will learn about it and can fix it. It pretty much works like this in Fedora - lot of the various API breakages and…

I agree that there is little to be gained from requiring perfection, I think there is considerable value in trying to do everything that is within reason. And there are a lot of reasonable things one can do to be nice to users. Allowing the user to install a piece of software without having to alter their system at all is a good thing.

And this brings us back to where this discussion started: it takes real effort to write non-trivial Python code that will run just by installing the program itself. The path of least resistance is to dump all the problems in the lap of the user, which many Python programmers tend to do in practice. That's not very nice to the consumer.

I think Python could benefit from developing empathy with the consumer.

Re: Faster Python with Guido van Rossum

#250

Post by SoylentOrange »

ctur wrote: post #4

I have become increasingly convinced Python as a language is a "trap" for any use of notable scale, be the scale about number of developers, codebase size, or performance requirements. It's a great 0->1 language and great at simple glue, but eventually you hit a wall and have to keep investing larger and larger amounts of people or computing resources to get continued returns... all due to fundamental design decision…

What do you use nowadays?
Post reply on HN