How to make a Python package in 2021 • Hacker News Board

Re: How to make a Python package in 2021

#201

Post by jimmaswell » Thu, Apr 08, 2021, 8:20 PM UTC

1337shadow wrote: post #99

jimmaswell wrote: post #39

Earlier quoted context omitted.

There would be no room for error if we just put the libraries in with the project as files instead of adding all these extra steps. Nobody seems to like this simple, bulletproof method anymore for some reason though.

That's exactly what a package manager does

A package manager is a whole separate program with config files that adds an extra build step and works like a black box. I mean just having the libraries entirely present in the repository. If an update is needed then someone pastes it in and commits it. This also lets you organize how you want.

Re: How to make a Python package in 2021

#202

Post by clawlor » Thu, Apr 08, 2021, 9:16 PM UTC

cixinlui wrote: post #130

zomglings wrote: post #8

The number of extra tools used in this article boggles my mind. Are you writing a simple library? Create a setup.py. Copy paste an existing setup.py and modify it to suit your purposes. Now you have a working, pip installable python package. Want to publish to PyPI? Use twine. It's standard and it's simple. You don't need complicated tooling for simple projects.

This is bad advice. Do not create a setup.py for a new package. (Keeping setup.py for an old package can be okay.) The author is correct that you want a tool such as flit or poetry which will work with pyproject.toml. Setting up a basic package will be no harder than using setuptools, and it is much more future-proof. You won't have to copy-paste some other crufty setup config either. It is fair that you don't need a…

Unfortunately, the pyproject.toml format still doesn't support editable installs. So, setup.py is still required if you need this feature.

Re: How to make a Python package in 2021

#203

Post by neolog » Thu, Apr 08, 2021, 9:39 PM UTC

jimmaswell wrote: post #201

1337shadow wrote: post #99

Earlier quoted context omitted.

That's exactly what a package manager does

A package manager is a whole separate program with config files that adds an extra build step and works like a black box. I mean just having the libraries entirely present in the repository. If an update is needed then someone pastes it in and commits it. This also lets you organize how you want.

That's often called "vendoring" these days.

Re: How to make a Python package in 2021

#204

Post by neolog » Thu, Apr 08, 2021, 9:40 PM UTC

blondin wrote: post #46

staticassertion wrote: post #23

Earlier quoted context omitted.

TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.

yea, it is. but it's a sane thing to do. recommending poetry for a beginner is a bad idea. (nothing against that package). python is a mature, old, software system. three times older than go or rust. way older than zig. these modern languages have learned from the field as a whole and implemented tools that people are taking for granted nowadays. as with any mature software system, people & companies have established…

> recommending poetry for a beginner is a bad idea.

Strongly disagree. There are so many footguns with low-level tools like pip that I can't recommend it to anybody but an expert (but an expert doesn't need my recommendation anyway).

Re: How to make a Python package in 2021

#205

Post by xtracto » Thu, Apr 08, 2021, 9:45 PM UTC

midrus wrote: post #164

2021 and still there is no clear winner or official way to install dependencies and create packages. The worst part of Python.

Is the 2.7 vs 3.?? not an issue anymore in the Python world?

Re: How to make a Python package in 2021

#206

Post by jftuga » Thu, Apr 08, 2021, 10:08 PM UTC

Which is the more standard way to use venv for Python 3.8 or newer?

1) venv at the same level:

create new project on GH

git clone https://github.com/user/myproject

python -m venv myproject

cd myproject

(activate venv)

____

2) clone first, venv in subdirectory:

create new project on GH

git clone https://github.com/user/myproject

cd myproject

python -m venv venv

(activate venv)

___

3) something completely different? if so, what?

Re: How to make a Python package in 2021

#207

Post by neolog » Fri, Apr 09, 2021, 12:01 AM UTC

cixinlui wrote: post #135

goerz wrote: post #67

Earlier quoted context omitted.

It's also not ready yet, missing critical features like editable installs. Right now, you still need a shim setup.py. Until pyproject.toml can actually replace setupy.py, I see little incentive to start using it: It's just one more file to add. The one exception is if the package actually has build requirements, e.g. for Cython modules.

I don't recognize this. I use poetry all the time, without setup.py and the local installs are editable. I have published half a dozen packages which don't have a setup.py and they all work fine.

You can't install a poetry package editably into another environment. It's really a missing feature in the pyproject spec. There's an open issue about it under one of the pypa repos. Someone just needs to do the work of implementing it in pip/setuptools.

Re: How to make a Python package in 2021

#208

Post by neolog » Fri, Apr 09, 2021, 12:55 AM UTC

xapata wrote: post #90

neolog wrote: post #87

Earlier quoted context omitted.

I don't know how much the original author had to do with Python's success in the last 20 years. The success of Python in data science is because of NumPy/Scipy/Pandas. Things like packaging that needed leadership never got any.

The leadership was that the science community would benefit from a science-specific tool, like Conda, and that making one ring to rule them all would be too difficult. Also, yes, Guido and many other early contributers have been been active for 30ish years.

Conda isn't really science specific, just like pip isn't webdev specific.

Re: How to make a Python package in 2021

#209

Post by Loranubi » Fri, Apr 09, 2021, 6:08 AM UTC

xtracto wrote: post #205

midrus wrote: post #164

2021 and still there is no clear winner or official way to install dependencies and create packages. The worst part of Python.

Is the 2.7 vs 3.?? not an issue anymore in the Python world?

2.7 is dead.

Re: How to make a Python package in 2021

#210

Post by nerdponx » Fri, Apr 09, 2021, 7:10 AM UTC

takluyver wrote: post #110

erezsh wrote: post #64

Earlier quoted context omitted.

How does flit compare to poetry? The seem to be both doing the same thing, and in a very similar way.

Poetry does much more than Flit, like resolving dependencies, creating a lock file, and managing an environment where you can run your code. In particular, Poetry is meant to support application development (where you want to have a fixed version of your dependencies) as well as library development. Flit is more aimed at being the simplest possible thing to put a package on PyPI, if that's all you want to do. It expe…

I feel like this is a good place to mention Pip-tools [0] which can generate a lockfile of sorts from a standard requirements.txt (or a setup.py). Specifically, it resolves all dependencies (including hashes), and writes them to a new "requirements" file that you can read with the usual `pip install -r`.

The nice part about Pip-tools versus Flit or Poetry or Pipenv is that Pip-tools lets you keep using Setuptools if you want to, or if you're unable to switch to one of the others for some reason (and valid reasons do exist).

[0]: https://pypi.org/project/pip-tools