Live data from Hacker News

Makefile Tricks for Python Projects

ricardoanderegg.com

21–30 of 35 posts

Re: Makefile Tricks for Python Projects

#21
post #17

Earlier quoted context omitted.

__pypackages__ is gonna be so nice when it lands.

Hate to be the bearer of bad news but the PEP has been rejected https://peps.python.org/pep-0582/

Nooooooooooo it was the chosen one.

Its quite sad to see the rationale as making it too complicated when "upgrading to venvs" when it should kill them outright. Why using python depends at all on weird shell semantics is beyond me.

Re: Makefile Tricks for Python Projects

#22
post #18

Don’t do this… unless you have a very specific and non-standard usecase that requires it, like the author: > I work using git worktrees to have multiple branches checked out at the same time. By using this trick, I can run different commands and make sure the imported package is from that branch, instead of having to python3 -m pip install -e . around all the packages For most usecases, you can wire up your imports a…

> if you’re on MacOS to auto-use your per-project venv.

Just chuck the venv name into a .python-version file then pyenv will do that for you, on any OS.

Re: Makefile Tricks for Python Projects

#25

I recently discovered just ( https://just.systems ), a make alternative that’s much more ergonomic IMO, and have been using it in my personal projects. It’s task-oriented, not goal-oriented, so not a perfect replacement, but works well for my needs.

Author here. I like `just` too, but it's missing a few features: * It can't run tasks in parallel * It doesn't handle file dependencies. i.e: it's just a task runner, not a build tool. * It's another tool you have to install, make comes pre-installed in most UNIX-like operating systems.

There’s a problem hidden.

MacOS make is outdated (or not GNU flavor not sure) and, for example, doesn’t support parallel tasks by default or some more advanced recipes.

Asking people to use gmake on a standard Makefile is much more difficult than just asking to install Justfile runner.

Re: Makefile Tricks for Python Projects

#27
post #18

Don’t do this… unless you have a very specific and non-standard usecase that requires it, like the author: > I work using git worktrees to have multiple branches checked out at the same time. By using this trick, I can run different commands and make sure the imported package is from that branch, instead of having to python3 -m pip install -e . around all the packages For most usecases, you can wire up your imports a…

Author here.

I agree my setup is not simple (or even recommended as the default), that's why I call them "tricks".

> you can wire up your imports and script entry points in poetry, and manage venvs there too. Or just use pyenv-virtualenv if you’re on MacOS to auto-use your per-project venv.

As someone who has tried most of the current Python tools (including poetry and pyenv), I wouldn't say that "wiring up" imports and entry points is much simpler or footgun-free. It also requires using specific tools, which may not be possible depending on the project.

Also, multiple venvs is not equivalent, since you still need to manage those venvs and install the packages (+ dependencies). The `PYTHONPATH` trick can save a lot of time when the dependency tree is complex, and you just want a single venv.

For example, a workflow like:

  git worktree add -b feature_one
  #
  # edit some files
  #
  BRANCH=feature_one make run-service
  
  # in a different terminal, assuming `master` is also checked-out as a worktree
  BRANCH=master make run-service
With the `PYTHONPATH` trick, you can do this in a few seconds. Now you have the same service running two different versions of your package, without having to reinstall anything or caring about using the correct venv.

Again, I know this is a hack and it can be very project-specific. But it has worked great for me, and maybe it can be helpful to others.

Re: Makefile Tricks for Python Projects

#28
post #25

Earlier quoted context omitted.

Author here. I like `just` too, but it's missing a few features: * It can't run tasks in parallel * It doesn't handle file dependencies. i.e: it's just a task runner, not a build tool. * It's another tool you have to install, make comes pre-installed in most UNIX-like operating systems.

There’s a problem hidden. MacOS make is outdated (or not GNU flavor not sure) and, for example, doesn’t support parallel tasks by default or some more advanced recipes. Asking people to use gmake on a standard Makefile is much more difficult than just asking to install Justfile runner.

100% agree, and this bothers me a lot. I like `just`, but for some reason I keep coming back to `make`.

> or not GNU flavor not sure

On my macOS laptop

  which make
  # /usr/bin/make

  /usr/bin/make --version
  # GNU Make 3.81
> doesn’t support parallel tasks by default

I haven't verified this. I've checked `make --help` using the built-in make on my Mac (GNU Make 3.81), it mentions the `--jobs` option to run multiple jobs in parallel.

Re: Makefile Tricks for Python Projects

#29
There are quite a few things here that I've used for a few years with option projects at work - overall I like it.

A big part of that is due to an endless stream of people who couldn't be bothered to learn what a venv was or how to use it, but... yeah, that's what work is like sometimes. The makefile pretty much ended those issues, absolutely worth the effort put into it.

Re: Makefile Tricks for Python Projects

#30

Or use https://pydoit.org and a virtualenv and be happy.

Looks like it’s pretty much abandoned: > doit is under active development. Version 0.36.0 released on 2022-04.

The Github repo shows some commits being made as recently as 4 months ago: https://github.com/pydoit/doit
Post reply on HN