Skip to content

PEP 458: Add clarification #1284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 29, 2020
Merged
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 51 additions & 47 deletions pep-0458.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,32 +23,6 @@ Post-History: 06-Jan-2019
Abstract
========

Attacks on software repositories are common, even in organizations with very
good security practices__. The resulting repository compromise allows an
attacker to edit all files stored on the repository and sign these files using
any keys stored on the repository (online keys). In many signing schemes (like
TLS), this access allows the attacker to replace files on the repository and
make it look like these files are coming from PyPI. Without a way to revoke and
replace the trusted private key, it is very challenging to recover from a
repository compromise. In addition to the dangers of repository compromise,
software repositories are vulnerable to an attacker on the network (MITM)
intercepting and changing files. These and other attacks on software
repositories are detailed here__. This PEP aims to protect users of PyPI from
compromises of the integrity, consistency and freshness properties of PyPI
packages, and enhances compromise resilience, by mitigating key risk and
providing mechanisms to recover from a compromise of PyPI or its signing keys.
In addition to protecting direct users of PyPI, this PEP aims to provide similar
protection for users of PyPI mirrors.

To provide compromise resilient protection of PyPI, this PEP proposes the use of
The Update Framework [2]_ (TUF). TUF provides protection from a variety of
attacks on software update systems, while also providing mechanisms to recover
from a repository compromise. TUF has been used in production by a number of
organizations, including use in Cloud Native Computing Foundation’s Notary
service, which provides the infrastructure for container image signing in Docker
Registry. The TUF specification has been the subject of three independent
security audits__.

This PEP describes changes to the PyPI infrastructure that are needed to ensure
that users get valid packages from PyPI. These changes should have minimal
impact on other parts of the ecosystem. The PEP focuses on communication between
Expand All @@ -62,15 +36,12 @@ mechanism. There is documentation for how to consume TUF metadata in the TUF
repository. However, changes to PyPI consumers are not required, and can be done
according to the timelines and priorities of individual projects.

__ https://github.com/theupdateframework/pip/wiki/Attacks-on-software-repositories
__ https://theupdateframework.github.io/security.html
__ https://theupdateframework.github.io/audits.html


Proposed TUF Integration
========================

This PEP proposes how The Update Framework [2]_ (TUF) should be integrated with the
This PEP proposes how The Update Framework [2]_ (TUF)__ should be integrated with the
Python Package Index (PyPI [1]_). TUF was designed to be a flexible
security add-on to a software updater or package manager. The framework
integrates best security practices, such as separating role responsibilities,
Expand All @@ -82,8 +53,10 @@ files. Or, alternatively, a role
responsible for indicating the latest snapshot of the repository may also have to be
compromised.

__ https://www.linuxfoundation.org/cloud-containers-virtualization/2017/10/cncf-host-two-security-projects-notary-tuf-specification/

The proposed integration will allow modern package managers, such as pip [3]_ to
be more secure against security attacks on PyPI, and to better protect
be more secure against attacks on PyPI, and to better protect
users from such attacks. Specifically, this PEP describes how PyPI processes
should be adapted to generate and incorporate TUF metadata (i.e., the minimum
security model). The minimum security model supports verification of PyPI
Expand Down Expand Up @@ -123,12 +96,34 @@ PEP Status
Due to the amount of work required to implement this PEP, in early
2019 it was deferred until appropriate funding could be secured to
implement the PEP. The Python Software Foundation secured this funding
[22]_ and new PEP coauthors restarted PEP discussion.
[22]_ and new PEP coauthors restarted PEP discussion__.

__ https://discuss.python.org/t/pep-458-secure-pypi-downloads-with-package-signing/2648/


Motivation
==========

Attacks on software repositories are common, even in organizations with very
good security practices__. The resulting repository compromise allows an
attacker to edit all files stored on the repository and sign these files using
any keys stored on the repository (online keys). In many signing schemes (like
TLS), this access allows the attacker to replace files on the repository and
make it look like these files are coming from PyPI. Without a way to revoke and
replace the trusted private key, it is very challenging to recover from a
repository compromise. In addition to the dangers of repository compromise,
software repositories are vulnerable to an attacker on the network (MITM)
intercepting and changing files. These and other attacks on software
repositories are detailed here__. This PEP aims to protect users of PyPI from
compromises of the integrity, consistency, and freshness properties of PyPI
packages, and enhances compromise resilience by mitigating key risk and
providing mechanisms to recover from a compromise of PyPI or its signing keys.
In addition to protecting direct users of PyPI, this PEP aims to provide similar
protection for users of PyPI mirrors.

__ https://github.com/theupdateframework/pip/wiki/Attacks-on-software-repositories
__ https://theupdateframework.github.io/security.html

On January 5, 2013, the Python Software Foundation (PSF) announced that [4]_ a security
breach had occurred on the
python.org wikis for Python and Jython. As a result, all of the wiki data was destroyed.
Expand Down Expand Up @@ -174,11 +169,16 @@ compromised.

__ https://mail.python.org/pipermail/distutils-sig/2013-September/022755.html

With the intent to protect PyPI against infrastructure compromises, this PEP
proposes integrating PyPI with The Update Framework [2]_ (TUF). TUF helps
secure new or existing software update systems that can result in clients
being compromised or crashed. It solves these problems by providing a flexible
security framework that can be added to software updaters.
To provide compromise resilient protection of PyPI, this PEP proposes the use of
The Update Framework [2]_ (TUF). TUF provides protection from a variety of
attacks on software update systems, while also providing mechanisms to recover
from a repository compromise. TUF has been used in production by a number of
organizations, including use in Cloud Native Computing Foundation’s Notary
service, which provides the infrastructure for container image signing in Docker
Registry. The TUF specification has been the subject of three independent
security audits__.

__ https://theupdateframework.github.io/audits.html


Threat Model
Expand All @@ -188,8 +188,7 @@ The threat model assumes the following:

* Offline keys are safe and securely stored.

* Attackers can compromise at least one of PyPI's trusted keys stored online,
and may do so at once or over a period of time.
* Attackers cannot compromise PyPI's trusted keys stored online.

* Attackers can respond to client requests.

Expand All @@ -198,6 +197,10 @@ leave installed) something other than the most up-to-date version of a
software distribution file. If the attacker is preventing the installation
of updates, they do not want clients to realize there is anything wrong.

This threat model describes the minimum security model. The maximum security
model described in PEP 480 also assumes that attackers can compromise PyPI's
online keys.


Definitions
===========
Expand Down Expand Up @@ -401,7 +404,7 @@ roles used in TUF.
+-----------------------------------------------------------------------------+
| Roles and Responsibilities |
+-----------------+-----------------------------------------------------------+
| root | The root roles is the locus of trust for the entire |
| root | The root role is the locus of trust for the entire |
| | repository. The root role signs the root.json metadata |
| | file. This file indicates which keys are authorized for |
| | each of the top-level roles, including for the root role |
Expand Down Expand Up @@ -580,8 +583,8 @@ This is crucial for the implementation of PEP 480 [21]_.
Metadata Expiry Times
---------------------

The metadata for the *root*, *targets*, and *bins* roles SHOULD each expire in one year, because these
two metadata files are expected to change very rarely.
The metadata for the *root*, *targets*, and *bins* roles SHOULD each expire in
one year, because these metadata files are expected to change very rarely.

The *timestamp*, *snapshot*, and *bin-n* metadata SHOULD each expire in one day
because a CDN or mirror SHOULD synchronize itself with PyPI every day.
Expand Down Expand Up @@ -750,13 +753,14 @@ It is primarily used for key revocation, and it is the locus of trust for all
of PyPI. The *root* role signs for the keys that are authorized for each of
the top-level roles (including its own). Keys belonging to the *root* role are
intended to be very well-protected and used with the least frequency of all
keys. It is RECOMMENDED that every PSF board member own a (strong) root key.
keys. It is RECOMMENDED that the PSF board determine the current set of trusted
root key holders, each of whom will own a (strong) root key.
A majority of them can then constitute a quorum to revoke or endow trust in all
top-level keys. Alternatively, the system administrators of PyPI could be
given responsibility for signing for the *root* role. Therefore, the *root*
role SHOULD require (t, n) keys, where n is the number of either all PyPI
administrators or all PSF board members, and t > 1 (so that at least two
members must sign the *root* role).
role SHOULD require (t, n) keys, where n is the number of key holders determined
by the PSF board, and t > 1 (so that at least two members must sign the *root*
role).

The *targets* role will be used only to sign for the static delegation of all
targets to the *bins* role. Since these target delegations must be secured
Expand Down Expand Up @@ -942,7 +946,7 @@ written to disk, include its hash in its filename:
where HASH is the `hex digest`__ of the hash of the file contents and
FILENAME is the original filename.

This means that there are multiple copies of every target file, one for each
This means that there MAY be multiple copies of every target file, one for each
of the cryptographic hash functions specified above.

__ https://docs.python.org/3.7/library/hashlib.html#hashlib.hash.hexdigest
Expand Down