by woodruffw 2 days ago

There's a discussion thread for email attestations here[1]. Those would support local builds.

But to take a step back: if you want attested builds because you think attestation is itself a gold star, then you're working from the wrong priors. Attestations do not imply that a package is secure any more than a hash does; what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.

I think PyPI's documentation does a good job of conveying this nuance[2], and that much of the opprobrium of attestations comes from people either not understanding that nuance or imputing conspiracies into it.

[1]: https://github.com/pypi/warehouse/issues/18007

[2]: https://docs.pypi.org/attestations/security-model/#trustwort...

pdonis 2 days ago

> what they are is a cryptographic signature over a set of claims, which allows for authenticated auditing of a project's state over time.

Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges". You could do this just as easily by, for example, letting me upload to my PyPI account a public key (the same as forges like Github do to allow SSH access), and then letting me sign builds that I upload using the corresponding private key. But that kind of workflow is not being supported. Why not?

  • woodruffw 2 days ago

    > Which is great--but does not in any way require that the attestation can only be produced by working through one of a small set of recognized "forges"

    That's true, there's some context elsewhere in these threads about how PyPI can enable email attestations. The short version is that PyPI wanted to make this work by default, which was trivial with Trusted Publishing (since it already provides a strong source of verifiable machine identity).

    "Why not a public key?" is a good question, and the very short answer is that greenfield signing designs should not assume that all users are able to manage the secrecy of long-lived key materials, much less force them to make parameter decisions when (re)generating materials. This is the same mistake that the Web PKI made for decades, and has spent the last 15 years slowly untangling. The ideal setup is one where users demonstrate control over an identity, and then the signing scheme binds that identity to a short-lived signing key. That's how email attestations work.

    • pdonis 2 days ago

      > email attestations

      From what I can gather from the github issue that was linked to, those would still require the email identity to be vouched for by a provider. (Though if whoever already is hosting my email could be a "provider", that would at least greatly expand the list of providers from the current small number of forges.) There doesn't seem to be any way for me to, for example, link a public key I generate myself to an email address that PyPI already has on file for me.

    • zahlman 1 day ago

      > "Why not a public key?" is a good question, and the very short answer is that greenfield signing designs should not assume that all users are able to manage the secrecy of long-lived key materials, much less force them to make parameter decisions when (re)generating materials.

      How is offering the public-key option in any way making such an assumption or applying any such force?

      • woodruffw 1 day ago

        As can be seen in every sub-thread of this HN post, users are constantly buffeted by bad advice around key management and generation. It’s the reason applied cryptographers still have to even say “don’t use PGP,” despite it not being a sensible thing to even seek out in 2026 (much less 2016).

        (Other large-scale PKI initiatives, like the Web PKI, learned this a long time ago. It’s why we have the CABF BRs instead of asking end users to pinky promise that they’re going to securely generate -‘d store a key and keep it around for 10+ years.)

        • pdonis 1 day ago

          So basically, you are viewing an attestation as making some kind of promise about how carefully the attester handles their private keys. And you're willing to trust a small number of large tech companies to do that, but you're not willing to trust individuals like me to do it. Am I reading that right?

          • woodruffw 1 day ago

            No, I wouldn't say that's right. Attestations are signatures from user-held keys under the hood; they're just short-lived keys bound to an identity rather than long-lived keys.

            The nuance is that the party that does that binding is typically someone who is doing more than keeping their private keys on disk, which is why they're trusted to handle more than short-lived key materials. As mentioned elsewhere that can include "big tech companies" (is ActiveState big? I have no idea) but PyPI also supports self-hosted GitLab instances for organizations. Key management is what matters to us, and knowing that we can trust the parties that we still have to trust.

            • pdonis 1 day ago

              > the party that does that binding

              In other words, you only trust parties to do the binding whose key management you find to be acceptable, and "I keep my private key on my local machine that nobody else ever has access to" is not acceptable. Correct?

              > PyPI also supports self-hosted GitLab instances for organizations

              How does PyPI verify that those organizations are doing key management in a way you find acceptable?

        • pdonis 1 day ago

          > the Web PKI

          The Web allows self-signed certificates. I can put one on my website and people can use HTTPS to connect to it that way. And then it's up to each person who goes to my website to decide how secure my self-signed certificate is, as opposed to one that comes from a CA.

          But of course the browser vendors don't look at it that way, they'll put serious warnings up to the user if they see a self-signed certificate, which they won't for a CA-signed certificate. That's basically your thought process as well, yes? Except that since you control the equivalent of the "website" and its certificate here (namely PyPI and its upload process), you won't even allow the equivalent of a self-signed certificate at all.

          • woodruffw 1 day ago

            The Web PKI does not allow self-signed certificates. You can self-sign a certificate and badger your users into trusting it, but as you note the browsers won't help you do that because it's a security pit of despair.

            > you won't even allow the equivalent of a self-signed certificate at all.

            You can embed a signature into a Python package if you want. There's even a spec for it[1], although nobody has ever really adopted it. And, of course, you can always host a signature somewhere else (and even link to it via standard packaging metadata). But PyPI will no longer allow you to upload a PGP sidecar (which were never standardized, either) because we have strong evidence that (1) nobody ever really bothered to verify them, and (2) even if they had, the quality of the key material was poor.

            [1]: https://peps.python.org/pep-0427/#signed-wheel-files

            • pdonis 1 day ago

              > The Web PKI does not allow self-signed certificates.

              That's not what the rest of your post says. The rest of your post says that you can use one for your website--which of course is true since no one can stop you if you insist on doing it--but that the rest of the Web infrastructure will treat your certificate poorly.

              I am simply pointing out that it looks to me like PyPI's attestation mechanism is taking a similar viewpoint to the Web infrastructure in this respect.

              > You can embed a signature into a Python package if you want.

              I'm talking about the attestation mechanism. That mechanism, unless I'm missing something, does not support self-signed certificates.

eesmith 2 days ago

Have you considered that perhaps you are working from the wrong priors?

I believe your prior is that you expect people who read the PyPI page to understand what attestation means. I believe your prior is they will see a notice about "not from a trusted publisher" and not treat it like demerit or warning. I believe your prior is that seeing this message will get people to do more due diligence than they currently do, and that the due diligence will be relevant and appropriate.

Based on my experience with my users, I don't have the same confidence.

I don't understand what "authenticated auditing" means, so why should they?

What project state do people think is important to audit? My code repo is private, hosted on Sourcehut. My build system is private, done locally. The only thing I release is the pre-built binary package. There's very little state to examine, and I can't figure out why anyone cares.

I also have customers who buy a source code license. They have never asked for an audit of the project's state over time.

I read the "Security model and considerations" document. I found it opaque, no doubt because the entire topic is foreign to me, but it doesn't do a good job of explaining. Take the second paragraph:

"The most basic type of attestation (the type enabled by default in pypa/gh-action-pypi-publish) asserts that the project was published by an authorized publisher such as a particular CI provider in a particular code repository.".

I know pypa/gh-action-pypi-publish is some sort of GitHub thing. I don't use GH or third-party CI so that clarification does not help.

It doesn't describe what "authorized publisher" means, other than a generic example of "particular CI provider in a particular code repository". That phrase exists only once in the document. Is it different from a "Trusted Publisher"?

Why are the terms "authorized" and "Trusted" used, instead of something like "affiliated"? If someone is not a trusted publisher or an authorized publisher it's is too easily misinterpreted as "untrusted publisher" or "unauthorized publisher", which seems scary. If someone is not an affiliated publisher, "unaffiliated publisher" sounds less scary. (I'm not saying "affiliated" is the right word, only an example of a roughly appropriate word where the negation of it is not so scary.)

Then later, "It protects against modification of a project after it was built".

I think that's supposed to be "modification of a release after it was built"?

My use of "project" matches the one at https://docs.pypi.org/using-pypi/pypi-basics/ "A project is a collection of releases and files, and information about them. ... A "release" on PyPI is a specific version of a project." I want to modify the project with updated metadata as things change, and to make new releases.

Oh, and I can't find a description of what "build" means in that document. It's used as if everyone expects to know what it means, but a few years ago I learned that modern Python packaging uses "build" different than I do.

When I compile my Python/C extension I "build" it. When I make a wheel I "package" it.

I believe that modern packaging calls the whole thing "build"? Or just the latter step?

Incidentally, "python -m build --sdist" on my project does not work as a replacement for "python setup.py sdist" because my setup.py includes a code generation step to get from a clean repo to the Python/C extension files included in the distribution. I haven't figured out the modern way to handle that. I think it's because of the differences in philosophy of what "build" means, and what actions are allowed during different stages of the distribution process, which is why I think "build" needs a specific definition, just like "project" and "release" have specific definitions.

  • woodruffw 2 days ago

    I mean, my actual prior here is that nobody should care or even think about this stuff. Authenticity and minimally-scoped credentials should be ambient properties of a system, not something that users even need to read docs for, much less enable. But if we're engaging with the premise that users are looking at this stuff at all, then I think it's facially reasonable to expect them to read the docs.

    For context: attestations aren't something you even really enable. They just happen, by default, if you use Trusted Publishing from a forge like GitHub or GitLab (and hopefully, more in the future).

    FWIW, your feedback on the docs is very helpful. It's genuinely useful to hear that the phrasing in it is imprecise, and I'll try and improve some of it this week. The main confusion that I think you're having (which is IMO reasonable) is that the doc was written to be theoretically splittable from the "Trusted Publishing" concept, i.e. so that "publishers" (principals who can publish) who aren't tied to a Trusted Publisher can attest to their published files.

    • pdonis 2 days ago

      > nobody should care or even think about this stuff.

      Does that mean I, as someone who has projects on PyPI, shouldn't care if I am unable to provide any attestations because I don't want to have to go through one of the recognized forges?

      > Authenticity and minimally-scoped credentials should be ambient properties of a system

      But if the only way to get them is to go through one of the recognized forges, then they're not "ambient properties" of just PyPI alone, because there's no way for me to use them just with PyPI alone.

    • eesmith 2 days ago

      Okay, but if the information is on the project page then it's pretty likely that users will look at this stuff without reading the docs. My feeling was the doc was written for people "Publishing Projects to PyPI" and not for people "Using PyPI", quoting the relevant sections of the documentation.

      Of course, in real life 99.9% of people aren't looking at the page at all, and either following a documentation step which says "install by doing XYZ" or transitively from some other project they install.

      I'll compare "attestations aren't something you even really enable" with this from the email attestation proposal: "Another possibility here would be to have it be opt-in: maintainers can mark their already-verified emails as being eligible for attestations on PyPI, e.g. via a checkbox in the user settings." ;)

      I also can't help but wonder who reads https://docs.pypi.org/publishing/publishing-to-pypi/ where it says "For instructions on how to package your Python project, please refer to the Python Packaging User Guide.", follows those steps including the one about using twine and TestPyPI, does all that, and then gets to "Publishing package distribution releases using GitHub Actions CI/CD workflows" and gives up, since twine works.

      Or goes back to the pypi documentation and sees "To use an API token: Set your username to __token__" when I can't find anything like that at https://packaging.python.org/en/latest/guides/publishing-pac... , Those seem like describing two very different systems, speaking as someone with no experience with them at all, who learned to use twine because of the Python Packaging User Guide.

    • eesmith 17 hours ago

      Could you specifically address this comment of mine?

      > Why are the terms "authorized" and "Trusted" used, instead of something like "affiliated"? If someone is not a trusted publisher or an authorized publisher it's is too easily misinterpreted as "untrusted publisher" or "unauthorized publisher", which seems scary. If someone is not an affiliated publisher, "unaffiliated publisher" sounds less scary. (I'm not saying "affiliated" is the right word, only an example of a roughly appropriate word where the negation of it is not so scary.)

      This is information presented to someone who does not know the term of art, and will therefore not read it that way.

      I have headaches still from dealing with people who insisted that my "well-formed" XML needed to be "valid", even though they didn't use schemas, nor know what a schema was, but simply because they didn't want the XML to be "invalid".