Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pypi.org is running a survey on the state of Python packaging

pypi.org

Post reply on HN
Print view View original

121–130 of 193 posts

Re: Pypi.org is running a survey on the state of Python packaging

#121

Post by greymalik »

smeagull wrote: post #113
IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

I think some of the pain has been Apple's fault. Requiring conda for some official packages means you always have two competing ecosystems on one machine - which is just asking for pain.

What packages are you referring to? I’ve been doing professional Python dev on a Mac for the past three years and have never had a reason to use conda, so I’m curious what I’m missing.

Re: Pypi.org is running a survey on the state of Python packaging

#122

Post by mistrial9 »

orf wrote: post #110
mistrial9 wrote: post #90

Earlier quoted context omitted.

very unfortunately the direct burden of python2 is placed on the packagers.. users of Python 2 like their libs (me) and have no horse in this demonization campaign

Pay for support for Python 2 then? At which point it’s a burden on the person you are paying. Or don't, in which case you're complaining that people are demonizing you because they are not doing your work for free?

this is battle-fatigue in action! I did not complain, in fact I am faced with the serious burden that is placed on packagers of python, on a regular basis, and have put many cycles of thought into it.. It is obvious that packagers and language designers are firmly in one end of a sort of spectrum.. while many users, perhaps engineering managers in production with sunk costs, are firmly at the other.. with many in between .. more could be said.. no one is an enemy on this, it is complicated to solve it with many tradeoffs and moving parts.. evidence in the topic today.

Re: Pypi.org is running a survey on the state of Python packaging

#123

Post by qbasic_forever »

IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

Libraries that use native code in JS or C-bindings in Go are equally annoying to get going if the documentation from the author is sub-par. In the JS world you'll get pages and pages of node-gyp errors that are completely incomprehensible until you realize the author never mentioned that you needed a half dozen little development library dependencies installed. Native C/C++ library code interfacing just sucks in general because there is zero package or dependency management in that world too.

Re: Pypi.org is running a survey on the state of Python packaging

#124

Post by nomel »

shrimpx wrote: post #38

I like poetry for its simplicity but I can’t tell how “official” it is in the python ecosystem. I hope it doesn’t die out. I think it’s the simplest possible way to maintain deps and publish to PyPI if you don’t have any weird edge cases.

The state of python packing changes so frequently that I wouldn't even be able to answer this survey, without looking up best practices. We're supposed to be using yaml files now, right?

Re: Pypi.org is running a survey on the state of Python packaging

#125

Post by thrown_22 »

IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

Installing Python only applications is trivial. What you're complaining about is all the missing code in other languages which isn't controlled by Python and depends on the OS to provide.

This is why we created Linux distributions in the first place. It is not the place of every language to reinvent the wheel - poorly.

Re: Pypi.org is running a survey on the state of Python packaging

#126

Post by ciupicri »

nomel wrote: post #124
shrimpx wrote: post #38

I like poetry for its simplicity but I can’t tell how “official” it is in the python ecosystem. I hope it doesn’t die out. I think it’s the simplest possible way to maintain deps and publish to PyPI if you don’t have any weird edge cases.

The state of python packing changes so frequently that I wouldn't even be able to answer this survey, without looking up best practices. We're supposed to be using yaml files now, right?

I think you meant TOML files.

P.S. By the way https://peps.python.org/pep-0518/#other-file-formats

Re: Pypi.org is running a survey on the state of Python packaging

#127

Post by dec0dedab0de »

zo1 wrote: post #53
nomdep wrote: post #34

Earlier quoted context omitted.

The PSF are not engineers looking for a better developer experience, but politicians looking for power. That’s why the pipenv fiasco a few years ago

What was the pipenv fiasco?

Pipenv is pretty nice, I still use it. Kenneth Reitz really has a knack for interfaces and making things easier for developers to use.

The fiasco was that he was not the best at maintaining projects, and used his popularity from tablib and requests to get pipenv recommended by the PyPA well before it was ready for general use. Then around the same time there was a few scandals I don't remember the details of, something about a developer being mad that Kenneth kept the money being donated to requests. It ended up with all of his popular projects being maintained by others.

I wish the response from the PyPA was to go full in on pipenv and just keep improving it at the same rate it was in the beginning. Instead it stagnated. Poetry came out fragmenting the ecosystem even more. And quite a few developers gave up on virtual environments in favor of docker.

pipenv started having regular releases again and have kept it going for the past few years. I like it enough to not want to put any effort into switching, but in 2017/2018 it really felt like the next big thing. It makes me sad to think that we all lost out because of politics.

Re: Pypi.org is running a survey on the state of Python packaging

#128

Post by Const-me »

mcdermott wrote: post #94

Installing packages, creating a manifest of dependancies, managing virtual environments, packaging, checking/formatting code, etc... should be built into the Python toolchain (the python binary itself). Needing to chose a bunch of third party tools to make it work... makes Python, well... un-pythonic.

Not sure about formatting code, I think that's a job for IDE or text editor, not the runtime.

Absolutely agree about the rest of them.

Re: Pypi.org is running a survey on the state of Python packaging

#129

Post by chlorion »

cycomanic wrote: post #99
ziml77 wrote: post #52

Earlier quoted context omitted.

I tend to just give up on a package if it requires a C toolchain to install. Even if I do end up getting things set up in a way that the library's build script is happy with, I'll be inflicting pain on anyone else who then tries to work with my code.

I know this is unpopular opinion on here, but I believe all this packaging madness is forced on us by languages because Windows (and to a lesser degree osx) have essentially no package management. Especially installing a tool chain to compile C code for python is no issue on Linux, but such a pain on Windows.

It may be unpopular but it's correct!

Every language tries to re-implement the package manager, but it ends up breaking down as soon as you need to interact with anything outside of that specific language's ecosystem. The only solution for interacting with the "outside" (other languages, toolchains, etc) is a system level, language agnostic package manager of some kind.

Linux distros package management is far from perfect but it's still miles ahead of the alternatives!

I very highly recommend people to learn how to write and create Linux packages if they need to distribute software. On Arch for example this would be creating PKGBUILDs, Gentoo has ebuilds, and other distros have something similar to these things.

Re: Pypi.org is running a survey on the state of Python packaging

#130

Post by westurner »

dane-pgp wrote: post #46
westurner wrote: post #42

Earlier quoted context omitted.

CT: Certificate Transparency logs log creation and revocation events. The Google/trillian database which supports Google's CT logs uses Merkle trees but stores the records in a centralized data store - meaning there's at least one SPOF Single Point of Failure - which one party has root on and sole backup privileges for. Keybase, for example, stores their root keys - at least - in a distributed, redundantly-backed-up…

For another example of how Ethereum might be useful for certificate transparency, there's a fascinating paper from 2016 called "EthIKS: Using Ethereum to audit a CONIKS key transparency log" which is probably way ahead of its time. Abstract: https://link.springer.com/chapter/10.1007/978-3-662-53357-4_... PDF: https://jbonneau.com/doc/B16b-BITCOIN-ethiks.pdf

Certificate Transparency: https://en.wikipedia.org/wiki/Certificate_Transparency

/? "Certificate Transparency" Blockchain https://scholar.google.com/scholar?q=%22Certificate+Transpar... https://scholar.google.com/scholar_alerts?view_op=list_alert...

- Some of these depend upon a private QKD [fiber,] line

- NIST PQ algos are only just now announced: https://news.ycombinator.com/item?id=32281357 : Kyber, NTRU, {FIPS-140-3}?

/? Ctrl-F "Certificate Transparency" https://westurner.github.io/hnlog/ :

"Google's Certificate Transparency Search page to be discontinued May 15th, 2022" https://news.ycombinator.com/item?id=30781698

- LetsEncrypt Oak is also powered by Google/trillian, which is a trustful centralized database

- e.g. Graph token (GRT) supports Indexing (search) and Curation of datasets

> And what about indexing and search queries at volume, again without replication?

My understanding is that the s Sigstore folks are now more open to the idea of a trustless DLT? "W3C Verifiable Credentials" is a future-proof standardized way to sign RDF (JSON-LD,) documents with DIDs.

Verifiable Credentials: https://en.wikipedia.org/wiki/Verifiable_credentials

# Reproducibile Science Publishing workflow procedures with Linked Data:

- Sign the git commits (GPG,)

- Sign the git tags (GPG+Sigstore, ORCID & DOI (-> W3C DIDs), FigShare, Zenodo,)

- Sign the package(s) and/or ScholarlyArticle & their metadata & manifest ( Sigstore, pkg_tool_xyz,CodeMeta RDF/JSON-LD, ),

- Sign the SBOM (CycloneDx, Sigstore,)

- Search for CVEs/vulns & Issues for everything in the SBOM (Dependabot, OSV,)

- Search for trusted package hashes for everything in the SBOM

- Sign the archive/VM/container image (Docker Notary TUF, Sigstore,)

- Archive & Upload & Restore & Verify (and then Upgrade Versions in the) from the dependency specifications, SBOM, and/or archive/VM/container image (VM/container tools, repo2docker (REES),)

- Upgrade Versions and run unit, functional, and integration tests ({pip-tools, pipenv, poetry, mamba}, pytest, CI, Dependabot,))

Post reply on HN