Live data from Hacker News

How to make a Python package in 2021

antonz.org

191–200 of 215 posts

Re: How to make a Python package in 2021

#191

The number of extra tools used in this article boggles my mind. Are you writing a simple library? Create a setup.py. Copy paste an existing setup.py and modify it to suit your purposes. Now you have a working, pip installable python package. Want to publish to PyPI? Use twine. It's standard and it's simple. You don't need complicated tooling for simple projects.

I actually think one should do the exact opposite to what you're suggesting. I've experienced modern package management through Cargo and anything below that level now seems like returning to stone age. In the python ecosystem, poorly defined packages is a wide problem. You never know what you get in the next version upgrade. So my suggestion: burn all the "simple" python packaging tools in a big fire and move eveyth…

Came here to about how it would be better to use poetry, thanks for spelling it out for me.

Re: How to make a Python package in 2021

#192
post #181

Earlier quoted context omitted.

Future-proof... Poetry 1.1 broke compatibility with 1.0. 1.1 lockfiles would crash Poetry 1.0, and 1.0 lockfiles would be thrown away by Poetry 1.1. It does not correctly verify hashes (if at all) [1]. You can't add packages without updating all your dependencies. Monorepos are not supported. PEP-508-compliant Git dependencies cause the tool to crash with TypeError [2]. I think Poetry is the right direction, I use it…

I didn't say it was a silver bullet... I said "Don't Use setup.py".

Fair enough. I saw many mentions of Poetry and merged them in my head when I finally replied. Apologies.

Re: How to make a Python package in 2021

#193
post #177

Earlier quoted context omitted.

72MB? On a development machine? Are you on dialup? Poetry doesn't fragment the ecosystem. Unlike setuptools it uses pyproject.toml, which can be read by other tools, and is the correct way of storing package configuration. A package built using Poetry is installable without Poetry in the exact same way as one built using setuptools.

Last time I tried poetry it was so broken that it was not even usable. I may try again later.

I had a tough time with it ~3 years ago, but now it works great for me

Re: How to make a Python package in 2021

#194
post #3

Just use Poetry [1]. It's popular and works well. [1] https://python-poetry.org/

How does it compare to pipenv?

In addition to everything else already said, pipenv is mind-bogglingly slow and buggy. Like 30 minutes and a timeout error to install pyspark.

Re: How to make a Python package in 2021

#195

Earlier quoted context omitted.

Yes. Also, check pip-tools by jazzband.

Which, like any jazzband project, you must not use for a commercial project, as stated by their CoC, the Contributor Covenant with a vague & bizarre modification about ethical use at the end. You are simply not allowed to contribute in any way on any kind of paid time, nor are you allowed to pay someone to contribute. This is of course not advertised for when they tell you to give them your OSS projects, for which yo…

Do you mean

> Other unethical or unprofessional conduct

from https://jazzband.co/about/conduct ? Or something else?

Re: How to make a Python package in 2021

#196

Earlier quoted context omitted.

I don't know anything about pipenv drama, so by morbid curiosity I looked for it and this is the first thing I found: https://github.com/pypa/pipenv/issues/2228 This is one of the most ridiculous issues I've read. If the rest of the "drama" is like that, then eh.

The drama was surrounding false advertising so to speak. Pipenv promised a lot but did not quite deliver, much like the earlier days of MongoDB. But more importantly, it pretended or at least heavily implied it was an official PSF-affiliated project, when it was not. How that claim was substantiated was also subject to drama.

It also had no releases for over a year, even though the master branch was getting frequent updates and the performance of the last release was atrocious (and I'm not sure if it has improved much).

I did like the emojis in the logs though.

Re: How to make a Python package in 2021

#197
post #92
post #67

Earlier quoted context omitted.

It's also not ready yet, missing critical features like editable installs. Right now, you still need a shim setup.py. Until pyproject.toml can actually replace setupy.py, I see little incentive to start using it: It's just one more file to add. The one exception is if the package actually has build requirements, e.g. for Cython modules.

The only reason we use pyproject.toml is because of the stupid black formatter that refuses to support setup.cfg, which every other python tool under the sun supports.

Yeah, it's annoying. But Black has so little configuration that I'm ok with hardcoding command-line parameters in my Makefile/tox.ini/precommit hooks (`--skip-string-normalization --line-length 79`)

Re: How to make a Python package in 2021

#198

Earlier quoted context omitted.

I actually think one should do the exact opposite to what you're suggesting. I've experienced modern package management through Cargo and anything below that level now seems like returning to stone age. In the python ecosystem, poorly defined packages is a wide problem. You never know what you get in the next version upgrade. So my suggestion: burn all the "simple" python packaging tools in a big fire and move eveyth…

I was hoping to see this blog post highlight poetry usage. Any good resource for this? I've already seen the official page.

I learned with this article. And it teaches how to combine Poetry with Pyenv effectively.

https://blog.jayway.com/2019/12/28/pyenv-poetry-saviours-in-...

Re: How to make a Python package in 2021

#199
I work in a very small company where we are building the plane as we learn to fly, and I'm the only person there with any programming experience. I've been working on trying to improve my hobbyist-level (at best) knowledge of the Python ecosystem over the last year or so.

I've gotten to the point where a number of smaller tools I've put together can now be used in larger projects. I learned the hard way that just copying files around makes it hard to know which version is in that project, and upgrading, especially once there is more than one file, becomes a lot tougher. I learned the harder way that trying to link the same file into multiple projects is a great way to really screw things up.

About 4 or 5 months ago I made a real effort to try to learn how to use virtual environments (pipenv) and packaging to make it so that if I update one of the smaller tools, I don't clobber all the downstream projects that rely on it. I wanted to make it so that when I update something to add features or change things, I can go back and fix older projects it's used in at my leisure. I haven't even begun to touch on unit testing, and I have no clue what linting is. Things are kind of working so far, but it feels very hacky and fragile, and I know it can (and SHOULD) be better.

All of this stuff around packaging and being able to install those packages very daunting, and trying to stumble on the right tutorials is extremely frustrating. The vast majority of them assume I want to share my stuff with the world on PyPI, or that I have servers available to me to create private PyPI indexes, but I don't. Yet I still want my packages to "resolve their own dependencies" when I install or upgrade them.

And when it comes to learning things like testing, the few tutorials I've looked at either use different tools to do it, or their examples are so oversimplified that when I look at my own code, I don't know where to begin.

I say all of this because looking at this tutorial, it's more of the same. I want to make my code better. I want to make it easier to use those smaller projects in larger projects. But then it says things like "Every solid open-source project runs cloud tests after each commit, so we will too," but it doesn't do anything to explain what that is or why it should be done, besides "everyone does it, so you should, too."

I think what makes it even harder is that when something like this gets shared, there are so many conflicting opinions. Some people say to just use setuptools, other say that setuptools is on its way out and to use pyproject.toml (or some other tool) instead. It's all just so... hard!

I'm sorry. This is coming off a bit ranty, and that's not what I intended. I'm just feeling frustrated and I'm not sure of a better way to express that I need help with finding help. There are even a lot of things that I'm sure I need help with, but I just don't know what they are. It makes it really hard to verbalize what I need to another person, let alone to get the right words into a search engine to take me there.

Re: How to make a Python package in 2021

#200
post #11

Earlier quoted context omitted.

How do you handle version pinning? hash checking? CI? testing on multiple platforms? multiple python versions? deployment? credential management? package data? version bumps? Sure, experts know how to do all these things because they spent many days learning them, but I'd rather outsource to a tool.

You don't pin anything for a package. I'm not aware of any "standard" CI that a package tool could set up. I guess you mean testing on multiple versions, in which case tox will help. Deployment for a package is handled by twine. Package data? What about it? Version bumps should always be manual but I recommend setuptools-scm. You seem to be confusing packages with "apps". It's very important to understand the clear d…

> You seem to be confusing packages with "apps".

I'm not confusing libraries with applications. Pinning dependency versions enables a repeatable test environment.

Post reply on HN