Python 3.12.0 from a supply chain security perspective • Hacker News Board

Re: Python 3.12.0 from a supply chain security perspective

#11

Post by Aachen » Thu, Oct 05, 2023, 10:53 AM UTC

jsjohnst wrote: post #10

crabbone wrote: post #6

In what situation, when it comes to deployed products is any of this relevant? Having used Python for decades, across multiple organizations starting from mega-corps and down to five programmers I've never used built Python binaries for any project that required Python. It's not hard to build your own, and it gives you better control of what's included (Python has a handful of optional compile-time dependencies, whic…

> but you already chose to suffer by not having control of your tools ~Gaslight much?~ Edit: How about not insulting people who don’t share your point of view?

> In contemporary language, gaslighting is a colloquialism describing the subjective experience of having one's reality repeatedly questioned

I understand it's even more contemporary to start using the word for anything where you could otherwise respond "don't be a dick", but I'm not sure what alternative word we could still use to mean gaslighting if gaslighting gets hijacked for a general negative meaning

Re: Python 3.12.0 from a supply chain security perspective

#12

Post by jsjohnst » Thu, Oct 05, 2023, 11:20 AM UTC

Aachen wrote: post #11

jsjohnst wrote: post #10

Earlier quoted context omitted.

> but you already chose to suffer by not having control of your tools ~Gaslight much?~ Edit: How about not insulting people who don’t share your point of view?

> In contemporary language, gaslighting is a colloquialism describing the subjective experience of having one's reality repeatedly questioned I understand it's even more contemporary to start using the word for anything where you could otherwise respond "don't be a dick", but I'm not sure what alternative word we could still use to mean gaslighting if gaslighting gets hijacked for a general negative meaning

Fair point! I was basing it more on this definition:

> to grossly mislead or deceive (someone) especially for one's own advantage

But even that doesn’t fit ideally, so I edited the post.

Re: Python 3.12.0 from a supply chain security perspective

#13

Post by nextlevelwizard » Thu, Oct 05, 2023, 11:41 AM UTC

crabbone wrote: post #6

In what situation, when it comes to deployed products is any of this relevant? Having used Python for decades, across multiple organizations starting from mega-corps and down to five programmers I've never used built Python binaries for any project that required Python. It's not hard to build your own, and it gives you better control of what's included (Python has a handful of optional compile-time dependencies, whic…

I have worked with Python for well over a decade and only time I have built my own is to test out the up coming version before it is readily available for my distro.

On Windows I just download from python.org and on Mac I get whatever homebrew gets me.

Never have I even thought about need to optimise my python executable.

Re: Python 3.12.0 from a supply chain security perspective

#14

Post by Denvercoder9 » Thu, Oct 05, 2023, 11:54 AM UTC

carreau wrote: post #4

As a curiosity, what would it entail to make the two tgz byte-for-byte identical ? There was/is some discussion in setuptools about how to normalize the tarball ( https://github.com/pypa/setuptools/issues/2133#issuecomment-... ) coudl something similar be applied to Building Python itself ?

> As a curiosity, what would it entail to make the two tgz byte-for-byte identical ?

It can't be that complicated. The tarballs autogenerated by GitHub (using `git archive`) were byte-for-byte identical for years, until GitHub upgraded git and things broke because entire ecosystems had started to rely on that.

[1] https://news.ycombinator.com/item?id=34586917

Re: Python 3.12.0 from a supply chain security perspective

#15

Post by stabbles » Thu, Oct 05, 2023, 12:11 PM UTC

carreau wrote: post #4

As a curiosity, what would it entail to make the two tgz byte-for-byte identical ? There was/is some discussion in setuptools about how to normalize the tarball ( https://github.com/pypa/setuptools/issues/2133#issuecomment-... ) coudl something similar be applied to Building Python itself ?

The suggestion there (uid = gid = 1000; uname = user; gname = users) isn't great.

Just use uid = gid = 0, and omit uname/gname.

If you're distributing software via a tarball, the uid/gid bits are meaningless. They only make sense when you archive / backup a directory and plan to extract on the same system.

If you set them to anything other than 0, it may happen that when the tarball is extracted as root user, ownership is changed to the uid/gid of the tarinfo provided those exist on the system. That's a lot of fun!

Python itself in fact tries to chown files when extracting a tarfile (under sudo).

If you set uid = gid = 0, then at least when extracting as root, the files remain owned by root.

Re: Python 3.12.0 from a supply chain security perspective

#16

Post by SethMLarson » Thu, Oct 05, 2023, 12:42 PM UTC

wallrat wrote: post #5

Is there any effort to integrate SLSA with PyPI? GitHub recently announced[1] that npm support for SLSA is GA now. [1] https://github.blog/changelog/2023-09-26-npm-provenance-gene...

Great question! PyPI already supports Trusted Publishers [1], which gets you most of the benefits of SLSA build provenance (provable link between artifacts and a public software repository). Implementing Trusted Publishers is the recommended first step for ecosystems looking to implement build provenance [2].

[1] https://docs.pypi.org/trusted-publishers/

[2] https://github.com/ossf/wg-securing-software-https://docs.py...

I don't think there's a big effort /right now/ to implement complete SLSA build provenance for PyPI and expose it for users to verify.

Re: Python 3.12.0 from a supply chain security perspective

#17

Post by SethMLarson » Thu, Oct 05, 2023, 12:47 PM UTC

r9295 wrote: post #3

Correct me if I'm wrong but SLSA would only prevent artifact tampering (eg. Account takeover on pypi) instead of typo squatting or build script abuse for example?

You've got it right, SLSA build provenance in particular only tells you that the artifact you have came from X software repo, at Y commit/tag, built using Z workflow. SLSA doesn't make any mention of what is actually in the artifact (but you can now safely verify the correct commit knowing it was used as input).

Typosquatting is an interesting one, because if you've made a typo in one place but not the other (ie installing package name "requestss", but repo is "psf/requests") then SLSA would "save" you by erroring on the mismatch. But that doesn't stop you from typoing in /both/ parameters.

Re: Python 3.12.0 from a supply chain security perspective

#18

Post by SethMLarson » Thu, Oct 05, 2023, 12:52 PM UTC

carreau wrote: post #4

As a curiosity, what would it entail to make the two tgz byte-for-byte identical ? There was/is some discussion in setuptools about how to normalize the tarball ( https://github.com/pypa/setuptools/issues/2133#issuecomment-... ) coudl something similar be applied to Building Python itself ?

I believe the only differences were uid/gid and username/groupname values between the two tarballs. One had the information of Thomas Wouters, the release manager of 3.12, and the other had generic GitHub Action usernames/groups.

Normalizing these values to something known like 0/0 would have done the trick.

Re: Python 3.12.0 from a supply chain security perspective

#19

Post by carreau » Thu, Oct 05, 2023, 1:27 PM UTC

stabbles wrote: post #15

carreau wrote: post #4

As a curiosity, what would it entail to make the two tgz byte-for-byte identical ? There was/is some discussion in setuptools about how to normalize the tarball ( https://github.com/pypa/setuptools/issues/2133#issuecomment-... ) coudl something similar be applied to Building Python itself ?

The suggestion there (uid = gid = 1000; uname = user; gname = users) isn't great. Just use uid = gid = 0, and omit uname/gname. If you're distributing software via a tarball, the uid/gid bits are meaningless. They only make sense when you archive / backup a directory and plan to extract on the same system. If you set them to anything other than 0, it may happen that when the tarball is extracted as root user, ownersh…

Thanks for advice, and I assume you are the one who commented on the upstream issue. This show it is not trivial, and it would be nice to be done automatically by default.

Re: Python 3.12.0 from a supply chain security perspective

#20

Post by carreau » Thu, Oct 05, 2023, 1:27 PM UTC

SethMLarson wrote: post #18

carreau wrote: post #4

As a curiosity, what would it entail to make the two tgz byte-for-byte identical ? There was/is some discussion in setuptools about how to normalize the tarball ( https://github.com/pypa/setuptools/issues/2133#issuecomment-... ) coudl something similar be applied to Building Python itself ?

I believe the only differences were uid/gid and username/groupname values between the two tarballs. One had the information of Thomas Wouters, the release manager of 3.12, and the other had generic GitHub Action usernames/groups. Normalizing these values to something known like 0/0 would have done the trick.

Thanks for the article and taking the time to reply here.