Live data from Hacker News

“Python's batteries are leaking”

pyfound.blogspot.com

221–230 of 420 posts

Re: “Python's batteries are leaking”

#221

Earlier quoted context omitted.

Yup. Totally agree with you on that. And yes this one is a constructive proposal. My problem is on the like part. Where shall we draw the line and how do we decide? To me this is a far more interesting discussion. (Maybe it has happened. I don’t go to many conferences these days so I might be missing something here. ) She mentioned http.client vs requests, datetime vs. moments etc, which are also quite correct to me.…

> Where shall we draw the line and how do we decide? Why does there need to be a line? As long as the package manager is part of the core distribution (even if it is itself an upgradable package) why not moving everything into packages, even if some are maintained by the core team and have the stable version at time of distribution release included with the core distribution—but perhaps installed only on demand?

How would you get standards like unittest?

pip install unitest? Oops I spelled it wrong wonder what I just installed?

Re: “Python's batteries are leaking”

#222
post #44

When I first used python like 20 years ago I was blown away by how much functionality was blown in, and it can be annoying using languages where even the most basic functionality involves downloading 50 packages from the internet, but on the other hand the standard library does seem to be a mess now.

I found that attractive when I first learned Python, but when I (much more recently) started picking up Rust, I was blown away by how easy and normal it is to use external packages: the build tool and package manager are the same thing and shipped with the language, the hello-world-equivalent docs assume you're using it, and even the Rust compiler and standard library themselves can (carefully) depend on external pac…

Node is ostensibly batteries included, but they’re weird, arcane batteries powered by tears. Therefore, to get anything done easily usually requires an external package.

Re: “Python's batteries are leaking”

#223

> six is non-optional for writing code for Python 2 and 3 I maintain a Python 2 & 3 compatible project that has no external dependencies.

Seconding this, I contribute to (and recently became a maintainer of) a relatively popular package that supports Python 2 and 3. We have 3 dependencies, and none of them are six.

To be fair, one of our three dependencies does in fact depend on six, but we don't use six anywhere in our code. This dependency of ours is used only in one specific area of the codebase, and in fact I (the person who added the dependency) wish that we didn't have to use it at all.

Re: “Python's batteries are leaking”

#224

Note that similar issues were raised with Ruby stdlib, which is being addressed in part with “Gemification” of stdlib, so that all of stdlib (targeted for 3.0, though it's been going on since 2.4)[0] is being moved out to externally-updatable packages that are included by default (default and bundled gems), so that it is still “batteries included” but the batteries are at least replaceable. Amber's suggestion seems t…

Yes and no. Until the Ruby maintainers give the gem versions of the internal modules different names you will get bizarro conflicts when using a gem after the "batteries included" version has already loaded.

I don't know how many hours I've spent battling Psych errors because of this very thing, but it's way too many. Calling the gem something, anything else, would solve the issue.

It's great that they're unbundling a lot of things, but there's still some serious friction between external and internalized versions of these gems.

For Ruby, EventMachine sub-universe is really in bad shape. EventMachine is creaky and old. Event-aware packages are in short supply and are usually woefully out of date, unmaintained.

Re: “Python's batteries are leaking”

#225
post #75
post #17

Earlier quoted context omitted.

From my own past experience with depression, it’s not as simple as “I’ll just stop doing all the things that have kept me in motion until now.” This is not meant to speculate on van Rossum’s health, just trying to point out that there are situations where apparent rudeness may be out of the person’s immediate control.

You can say the same thing about anyone doing anything. It's just giving excuses for bad behavior.

Yes. It is an excuse, even for poor behavior.

Re: “Python's batteries are leaking”

#226

Earlier quoted context omitted.

the problem is it can fall apart quickly. the XML parsing in the standard library is limited and slow, so most people consume lxml instead [0]. so it depends on the case. counterpoint: e.g. pathlib being in included is great. it was at least inspired by 3rd party libraries, but the features are relatively stable and the scope defined, and relatively few dependencies, and so moving it into the standard library is a wi…

ElementTree is in the stdlib. It isn't slow and has incremental parsing and so on. It's also a nice API for dealing with XML.

> ElementTree is in the stdlib. It isn't slow and has incremental parsing and so on.

I had enough trouble using it efficiently that I went and wrapped Boost property tree[0] and can happily churn out all sorts of data queries (including calling into python for the sorting function from the C++ lib) in almost no time.

I was taking daily(ish) updates of an rss feed and appending it to a master rss file but sorting was pretty slow using list comprehensions so now I convert it automagically to json and append it as is. No more list comprehensions either, just hand it a lambda and it outputs a sorted C++ iterator.

Though I probably should've just thrown the data into a database and learned SQL like a normal person...

[0] https://github.com/eponymous/python3-property_tree

Re: “Python's batteries are leaking”

#227
post #207

Earlier quoted context omitted.

Python cannot be atomized effectively, and the issue is political. The problem is that I cannot count on being able to install new software in many environments. If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever. Consequently, the standard libraries need to be very com…

> If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever. Can you explain this more? What kind of place do you work? I've had some experience with large, bureaucratic companies, but nothing ever so far as "you can't install any other libraries."

>What kind of place do you work?

Not OP but same. I'm currently debating with myself whether I should attempt to install PUTTY. Given that port 22 is blocked and it's not needed for my core role it'll be dicey if I get challenged.

Pulling executable code off some repo...no way that is ever officially passing muster. People might do it anyway, but on a personal risk basis.

>Can you explain this more?

Place that are heavy on confidential financial info basically. Practically everything I touch is confidential client data. So employer is naturally jumpy about what's on my laptop software wise.

Ironically the above comes full circle...need putty to get onto a VM in cloud where there are no restrictions and crucially no client data. Nobody cares what I do there - hell they'll even pay for it thanks for MSDN enterprise

Re: “Python's batteries are leaking”

#228
post #125

Earlier quoted context omitted.

> What happens if you discover a security vulnerability in your stdlib? As the user of the stdlib, it really depends on what the security issue is. For most users, if they can verify they're not using module X, then there's no problem and no rush for them to upgrade. In a language like Python that's very dynamic, you also might be able to download a hotfix .py file provided by the language maintainers (or any other p…

> Actually users do. Not all users, but your sweeping statement isn't true of all users either. Experience and reading reports tell me that many if not most users actually upgrade, especially when there's a significant security issue and especially when upgrading is easy. 2.7.14 to 2.7.15 was a problem for no one. Sure, some people definitely upgrade. But there's a big difference between requiring someone to actively…

Wouldn't new projects also be using the latest stable released version of the language (and hence standard lib) that's shipped with their distro or wherever they're getting it from, especially in the deployment environment? Or at least I think they'd be as likely to do so as getting the newest versions of libs from the lib repository...

Unless, and maybe this is a source of disagreement, it's still the standard in Node land to specify dependencies as "whatever the latest version is, I don't care"? That approach has teeth. After you've been bitten a few times you learn to say "no more" and version pin. There are other nice benefits to version pinning too (like reproducible builds). It's pretty common in Java land, even though some people still specify "version x or higher". It's a tradeoff.

One downside is that if you don't occasionally check maven central or wherever for new versions you might miss a security issue, and if you're copying dependency specifying files over to new projects the issue will persist. This is solvable with tooling -- http://www.mojohaus.org/versions-maven-plugin/ comes to mind and I was pleasantly surprised to get an email from Github that even a toy project of mine pointed to a version of a JSON parser with a known issue.

> What if we come up with a newer way to do async? Do we add yet another API?

Sure. Especially if people were just fine using the older API. For the fs situation, putting it in its own package would be great, up until the point I have to modify years-old code that's been working just fine through upgrades until someone decided to break things... for JS that's even more intolerable since nothing in the type system forbids the function updating to accept a promise or a callback. That (as would be a new API for async) would be a non-breaking change, it only provides more and the requirements for old code are the same.

If it's possible to break up the core in a way that preserves legacy code, I'm all for it. Java's Project Jigsaw did this to an extent with its module system. Another way to help that is to separate what's standard (such as with an official ANSI or ISO standard) for the language and what a particular implementation of the standard bundles with its releases on top of the standard. I recall at one point the old IO fork of Node promised to never break "core javascript APIs", but the industry consortia that defines what that means makes me think it was more a hollow promise.

To take an example from a language that has official standards, C, across decades of versions of C compilers and standard libraries from various vendors, if my code is C89 then it compiles with compilers that support C89. People might be offended that a function like strcpy exists, but it's there, I can use it if I want, or a separate library like bstring, or maybe the compiler provides their own __builtin__strcpy_chk that I can use directly or switch between with a flag (_FORTIFY_SOURCE). This might end up being less work for the compilers, too, since they don't need a security bulletin saying "some uses of strcpy can lead to a buffer overflow, removing it in v2.1.2, update immediately to strcpy2". They can (and did) just provide a new strcpy2, made available a seamless upgrade (I believe even Node has done this with some shims for deprecated APIs) if you don't want to change the source text, and warn people about the old one.

Re: “Python's batteries are leaking”

#229

Earlier quoted context omitted.

It’ll be interesting to see how Go (Rust, and other new languages) evolve and if they can avoid some level of package decay when they reach the age of Python, Java, etc.

I’ve been working with go a lot lately and they seem really focused on not letting this happen. Every single thing in the language and standard library seem completely focused on minimalism and compiler time. The standard lib is unlikely to change all that much and people are not picking the language for a bunch of convenience features. Third party package problems will be an issue at some point but that’s more due t…

I think this is mostly due to the 1.x guarantee that anything old will compile on anything new. I don’t know if Python or others formalized the same guarantee, but I don’t remember any issues 2.2 - 2.15.x (of course 2-3 is infamous). However 3.3+ (when I joined the py3 crew from py2) I don’t recall any either.

My experience with Go has been equally pleasant (though it does require more boilerplate for commonly accepted reasons in the community), but in no way unique.

Re: “Python's batteries are leaking”

#230

The Python standard library has been a huge help for me. Evaluating which third party packages to trust and handling updates is a hassle. (Would love a solution for this. Does anyone have a curated version of PyPI?) I’m surprised that people want to slim it down other than for performance on a more constrained system. As an aside, why doesn’t the Python standard library extend/replace features with code from successf…

> Does anyone have a curated version of PyPI? I'm not entirely sure what you're looking for, but have you tried https://www.enthought.com/product/enthought-deployment-manag... ? Edit: example $ edm envs create tester36 --version 3.6 $ edm shell -e tester36 (tester36) $ edm install ipython matplotlib pyqt

That’s interesting. Thanks!
Post reply on HN