Ahhhh yes they did it. They delivered workspaces and editable dependencies, which is literally all I needed.
I can't wait to set this up, I'm very confident they'll iron out any remaining bugs or gotchas (e.g. CUDA) quickly.
Uv 0.3 – Unified Python packaging
31–40 of 60 posts
Re: Uv 0.3 – Unified Python packaging
#32I love the idea of Rye/uv/PyBi on managing the interpreter, but I get queasy that they are not official Python builds. Probably no issue, but it only takes one subtle bug to ruin my day. Plus the supply chain potential attack. I know official Python releases are as good as I can expect of a free open source project. While the third party Python distribution is probably being built in Nebraska.
We'd love it if there _were_ official portable Python binaries, but there just aren't. We're not just distributing someone else's builds though, we're actively getting involved in the project, e.g., we did the last five releases.
We've invested quite a bit of effort into finding system Python interpreters though and support for bringing your own Python versions isn't going anywhere.
Re: Uv 0.3 – Unified Python packaging
#33How's the work on a type checker coming along?
Re: Uv 0.3 – Unified Python packaging
#34How's the work on a type checker coming along?
Imagine mypy with the speed of uv…
Re: Uv 0.3 – Unified Python packaging
#35Does anyone know if Astral has plans to build an LSP like pyright? There are many projects that try to replicate pyright's functionality, pylyzer comes to mind, but don't have sufficient coverage (e.g. missing Generic support). Having a team like Astral's behind creating a fast and good LSP for Python would be great.
Re: Uv 0.3 – Unified Python packaging
#36Does uv handle cases where dependencies go beyond Python? For example, many bioinformatics-related conda packages would come with external binaries written in C++ or other languages. Would uv be able to pull from conda?
Re: Uv 0.3 – Unified Python packaging
#37Why is it called “uv”?
Re: Uv 0.3 – Unified Python packaging
#38I love the idea of Rye/uv/PyBi on managing the interpreter, but I get queasy that they are not official Python builds. Probably no issue, but it only takes one subtle bug to ruin my day. Plus the supply chain potential attack. I know official Python releases are as good as I can expect of a free open source project. While the third party Python distribution is probably being built in Nebraska.
I'm from Nebraska. Unfortunately if your Python is compiled in a datacenter in Iowa, it's more likely that it was powered with wind energy. Claim: Iowa has better Clean Energy PPAs for datacenters than Nebraska (mostly due to rational wind energy subsidies).
Anyways, software supply chain security and Python & package build signing and then containers and signing them too
Conda-forge's builds are probably faster than the official CPython builds. conda-forge/python-feedstock//recipe/meta.yml: https://github.com/conda-forge/python-feedstock/blob/main/re...
Conda-forge also has OpenBLAS, blos, accelerate, netlib, and Intel MKL; conda-forge docs > switching BLAS implementation: https://conda-forge.org/docs/maintainer/knowledge_base/#swit...
From "Building optimized packages for conda-forge and PyPI" at EuroSciPy 2024: https://pretalx.com/euroscipy-2024/talk/JXB79J/ :
> Since some time, conda-forge defines multiple "cpu-levels". These are defined for sse, avx2, avx512 or ARM Neon. On the client-side the maximum CPU level is detected and the best available package is then installed. This opens the doors for highly optimized packages on conda-forge that support the latest CPU features.
> We will show how to use this in practice with `rattler-build`
> For GPUs, conda-forge has supported different CUDA levels for a long time, and we'll look at how that is used as well.
> Lastly, we also take a look at PyPI. There are ongoing discussions on how to improve support for wheels with CUDA support. We are going to discuss how the (pre-)PEP works and synergy possibilities of rattler-build and cibuildwheel
Linux distros build and sign Python and python3-* packages with GPG keys or similar, and then the package manager optionally checks the per-repo keys for each downloaded package. Packages should include a manifest of files to be installed, with per-file checksums. Package manifests and/or the package containing the manifest should be signed (so that tools like debsums and rpm --verify can detect disk-resident executable, script, data asset, and configuration file changes)
virtualenvs can be mounted as a volume at build time with -v with some container image builders, or copied into a container image with the ADD or COPY instructions in a Containerfile. What is added to the virtualenv should have a signature and a version.
ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ :
rpm-ostree rebase ostree-image-signed:registry:
rpm-ostree rebase ostree-image-signed:docker://
So, when you sign a container full of packages, you should check the package signatures; and verify that all package dependencies are identified by the SBOM tool you plan to use to keep dependencies upgraded when there are security upgrades.
e.g. Dependabot - if working - will regularly run and send a pull request when it detects that the version strings in e.g. a requirements.txt or environment.yml file are out of date and need to be changed because of reported security vulnerabilities in ossf/osv-schema format.
Is there already a way to, as a developer, sign Python packages built with cibuildwheel with Twine and TUF or sigstore to be https://SLSA.dev/ compliant?
Re: Uv 0.3 – Unified Python packaging
#39Love that you’ve kept the workspace support from Rye. Is there any support for workspace _builds_? That is, resolving the graph of interdependencies between packages to build a wheel/Dockerfile/whatever?
Fwiw I’m building a thing [1] that does this. Current docs suggest Rye but will s/rye/uv/ shortly. It’s basically just some CLI commands and a Hatch/PDM plugin that injects needed stuff at build-time.
Re: Uv 0.3 – Unified Python packaging
#40https://xkcd.com/927/
Unfortunate that python dev these days requires dependency manager managers to set up an entire for-purpose python just for each project. As opposed to being able to run on the system python (like every other scripting language). A victim of it's popularity. Now since Uv is in Rust we'll need a dependency manager manager manager for it on any OS that's not rolling to compile since what rustc is changes every 3 months…
Python is hardly unique there. Other trends include using docker images for tools (using all kinds of for-purpose frozen version language runtimes)