Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Python keeps growing, explained

github.blog

Post reply on HN
Print view View original

361–370 of 459 posts

Re: Why Python keeps growing, explained

#361

Post by haswell »

brundolf wrote: post #328
CE02 wrote: post #169

One thing I’d add to this conversation, though I’m certain it’s already been stated: As many have mentioned, there is a large subset of the user base that uses Python for applied purposes in unrelated fields that couldn’t care less about more granular aspects of optimization. I work as a research assistant for international finance faculty and I would say that compared to the average Hackernews reader, I’m technologi…

I'm one of Python's biggest critics (to me it's a Monkey's Paw of software development), but I think this is exactly the appropriate situation to use it. It's great for one-off fancy calculations, system scripts, ideally with no dependencies and/or a short lifetime

> to me it's a Monkey's Paw of software development

This piqued my curiosity. I've worked with Python on and off for the last ~20 years, and while I'm not a fanboy or apologist, and use other tools when appropriate, there's also a reason it remains in my toolbox and sees regular use while many other tools have come/gone/been replaced by something better.

Can you share an example scenario where it's a Monkey's Paw? My suspicion is that this is more of an org issue than a tech issue?

Re: Why Python keeps growing, explained

#362

Post by gary_0 »

pxc wrote: post #356
gary_0 wrote: post #296

Earlier quoted context omitted.

> Hello world is just `print "Hello, world!"` SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello, world!")?

That's because that issue of the webcomic is like 15 years old. It was true of the version of Python which was current at the time: https://stackoverflow.com/questions/6182964/why-is-parenthes...

Yes, I know. But some people still have the reflex from Python 2 and feel bitter when the error message says "I know what you want, and I'm not giving it to you."

Re: Why Python keeps growing, explained

#363

Post by redsaz »

slt2021 wrote: post #355
dagw wrote: post #163

Earlier quoted context omitted.

I've rewritten real world performance critical numpy code in C and easily gotten 2-5x speedup on several occasions, without having to do anything overly clever on the C side (ie no SIMD or multiprocessing C code for example).

2-5x speedup is not a lot, I would say it is not worth it to rewrite from py to C if you don't have an order of magnitude improvement. Because if you compare the benefit to the cost of rewrite from py to C and cost of maintaining/updating C code and possible C footguns like manual memory safety, etc - then there is no benefit left

It can be worth it. What matters is how much time it saves your users over the course of using the app vs the time it took to develop it. So, if:

#-of-users * total-time-saved-per-user > time-spent-optimizing

Then it's worth it. You can even multiply by cost of user per time unit and cost of developer per time unit, to see how much money was saved.

Even in cases where its the same person on both sides, it can still work out. There's an xkcd comic about it, even.

Re: Why Python keeps growing, explained

#364

Post by brundolf »

haswell wrote: post #361
brundolf wrote: post #328

Earlier quoted context omitted.

I'm one of Python's biggest critics (to me it's a Monkey's Paw of software development), but I think this is exactly the appropriate situation to use it. It's great for one-off fancy calculations, system scripts, ideally with no dependencies and/or a short lifetime

> to me it's a Monkey's Paw of software development This piqued my curiosity. I've worked with Python on and off for the last ~20 years, and while I'm not a fanboy or apologist, and use other tools when appropriate, there's also a reason it remains in my toolbox and sees regular use while many other tools have come/gone/been replaced by something better. Can you share an example scenario where it's a Monkey's Paw? My…

Dependency management/tooling. Python (philosophically) treats the whole system as a dependency by default, in contrast with other modern languages that operate at the project/workspace level. This means it's very hard to isolate separate projects under the same system, or to reproducibly get a project running on a different system (even the same OS, because the system-wide state of one machine vs the next matters so much).

People work around these issues with various kludges like virtual environments, Docker (just ship the whole system!), and half a dozen different package managers, each with their own manifest format. But this is a problem that simply doesn't exist in Go, JavaScript, Rust, and others.

For code that never needs anything except the standard library, or for a script that never needs to be maintained or run on a different machine, Python is fine. Maybe even nice. But I've watched my coworkers waste so many hundreds of developer-hours just trying to wrangle their Python services into running locally, managing virtual environments, keeping them from trampling on each other's global dependencies, following setup docs that don't work consistently, and fixing deployments that fail every other week because the house is built on sand.

Re: Why Python keeps growing, explained

#365

Post by yamtaddle »

moffkalast wrote: post #113
loudmax wrote: post #108

Earlier quoted context omitted.

You won't get high performance out of Python directly, but there are a lot of Python libraries that use C or a powerful low level language underneath. The heavy lifting in so much of machine learning is CUDA, but most people involved in ML are writing Python.

Sure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast. If people write program logic in python it will run at python speeds. Otherwise you're not really writing python, like nobody says some linux native program is bash because it happens to be launched from a bash script.

> Sure, but what's not really python per se. One could also call C++ libraries from java via JNI and pretend java is super fast.

But that's how every scripting language obtains good-not-just-decent performance. A strong culture of dropping down to C for any halfway-important library is why PHP's so hard to beat in real-world use, speed-wise (whatever its other shortcomings).

Re: Why Python keeps growing, explained

#366

Post by namaria »

noloblo wrote: post #344
ironman1478 wrote: post #335

Earlier quoted context omitted.

A lot of these problem spaces can get away with single threaded performance because maybe they're generating a report or running an analysis once a day or at even slower frequency. I work in a field where numerical correctness and readability is important for prototyping control algorithms (I work on advanced sensors) and python satisfies for those properties for our analysis and prototyping work. When we really want…

debugging dynamic python and imperative stateful python after a certain code base size >10k LOC gets extremely painful for any meaningful scale you are better served by basic FP hygiene as evidenced in haskell elixir CL/racket or rust/golang

I agree. But most people just need a pick up truck, not forming railway consists.

Re: Why Python keeps growing, explained

#367

Post by aflag »

dragonwriter wrote: post #353
mlboss wrote: post #342

How difficult it would be to transpile python program into a more performant language using a LLM ? This would solve so many issues. Write a code in python and convert it into C++/Rust/Assembly for performance.

> How difficult it would be to transpile python program into a more performant language using a LLM ? It'd be a lot easier (and more reliable) just to use a traditional compiler, rather than an LLM imitating a compiler. The problem is maintaining the full scope of Python semantics there is usually no advantage to this. Compiling parts of a python codebase, with restricted semantics, can give you significant gains in…

I think the idea is that a person can read a project in python and re-implement it more efficiently in C++. The LLM could do the same thing. However, I don't think LLMs are quite that powerful.

Re: Why Python keeps growing, explained

#368

Post by yamtaddle »

artagnon wrote: post #87
qsort wrote: post #27

Earlier quoted context omitted.

I use Python all the time both for my personal stuff and for some side-projects at work, so this isn't a dunk on Python, but honestly it feels like a circular thing: it's popular because it's popular. I wouldn't say it's friendlier than the alternatives, Perl and Shell scripts sure, but not when compared to Javascript, Ruby or Lua. Now, if you're talking about libraries, support, etc. then sure, Python wins hands dow…

I think Lua was always seen as a bit obscure, and not enough people invested in the language to write useful utilities. It has a solid C foreign function interface, and the compiler is quite fast, which leaves me puzzled about why it never gained traction. I think it's an embedded scripting language in the majority of use cases (e.g. NeoVim, LuaLaTeX, scripting in some game engines). The story of Ruby is altogether d…

> I think Lua was always seen as a bit obscure, and not enough people invested in the language to write useful utilities. It has a solid C foreign function interface, and the compiler is quite fast, which leaves me puzzled about why it never gained traction. I think it's an embedded scripting language in the majority of use cases (e.g. NeoVim, LuaLaTeX, scripting in some game engines).

- Lua's standard library is so weak that it makes most other batteries-not-included languages look like they have large, robust, and helpful standard libraries.

- It's got a bit of the quirkiness and gotcha-ability of JavaScript but without its being a language that's impossible to avoid due to capture of a mega-popular platform, which is what propelled JavaScript to ubiquity despite its being kinda shit and unpleasant to work with.

- Tooling's not as good as many other languages.

(FWIW sometimes I write Lua regardless, because it's the right tool for the job)

Re: Why Python keeps growing, explained

#369

Post by tejtm »

faizshah wrote: post #314
bombcar wrote: post #310

Earlier quoted context omitted.

I still use perl for some of that stuff, or even awk, but those are barely reusable or readable.

This is a letter to the general community: please stop writing these scripts in perl and bash one liners. That one off script you thought would only be used once or twice at this nonprofit has been in continuous use for 12 years and every year a biologist or journalist runs your script having no idea how it actually works. Eventually the script breaks after 8 years and some poor college student interns there and has…

one line spaghetti ... I remain unsympathetic.
Post reply on HN