From 4ef5558912c12c09ffb791becd36d17ebaf42907 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 10 Jul 2026 14:52:09 +0200 Subject: [PATCH 01/28] PEP 825: add a non-normative section on multiple sources MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 69 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 62 insertions(+), 7 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index a89f110b747..0a36ed50041 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -799,6 +799,63 @@ To generate the ``{name}-{version}-variants.json`` file: ``default-priorities.namespace`` value +Installing wheels from multiple sources (non-normative) +------------------------------------------------------- + +As of the time of writing, there are no accepted standards addressing +the support for installing packages from multiple sources, and the +existing tools (such as ``pip`` and ``uv``) disagree on the exact +behavior. This problem is described in more detail in the informational +:pep:`766`. Variant wheels expand the problem scope, as the variant +metadata that influences the selection among multiple wheels is scoped +at index level, and therefore the metadata acquired from different +indexes cannot be compared or combined in a meaningful way. For these +reasons, the specification does not attempt to standardize a behavior, +but instead considers it implementation-defined and provides a few +non-normative suggestions on the possible solutions. + +It is entirely valid for tools not to support this behavior, either by +not providing support for using multiple sources at all, or by rejecting +to proceed if more than one of the sources includes variant wheels. + +When processing variant wheels from different sources, it is recommended +to consider their variant metadata in isolation. Matching variant labels +or namespaces do not establish that they have the same meaning across +sources. + +A tool that searches sources in priority order (for example, the "index +priority" in :pep:`766`) can order variants from one source at a time +using the `variant ordering`_ algorithm, and proceed to the next source +only if the current one has no viable candidates. No cross-source +metadata merge is necessary. + +A tool that collates candidates from all sources before selecting among +them (for example, "version priority" in :pep:`766`) must compare wheels +from different sources directly. For a package version, if exactly one +source provides compatible variant wheels, those variants can be +selected ahead of non-variant wheels, as generally variant wheels are +preferred over non-variant wheels. If multiple sources provide +compatible variants, their metadata can be combined unambiguously only +when the metadata is consistent as specified in the `index-level +metadata`_ section. + +When metadata cannot be combined unambiguously, there is no uniquely +correct global ordering. Valid tool-specific choices include: + +- refusing to install with an error +- printing a warning and ignoring variant wheels, falling back to + selection among non-variant wheels +- using a deterministic criterion for selecting among multiple + compatible variant wheels, for example by preferring the wheel from + the index that came earlier in the option arguments +- requesting additional input from the user + +These choices can produce a valid installation without guaranteeing a +globally optimal selection. Silently fabricating a combined order or +resolving conflicting label mappings is technically possible, but +discouraged: it fabricates semantics that no source declared. + + Rationale ========= @@ -1071,13 +1128,6 @@ The following problems are deferred to subsequent PEPs in the series: - overriding the compatibility detection using static data - building variant wheels -In addition to that, the following matters are left -implementation-defined: - -- Selecting variant wheels from multiple sources. Currently, there is no - standard defined behavior for regular wheels, nor consensus across - different packaging tools on how to handle that. - Acknowledgements ================ @@ -1097,6 +1147,11 @@ and Zanie Blue. Change History ============== +- 03-Jul-2026 + + - Added a non-normative guidance for installing variant wheels from + multiple sources. + - 06-Apr-2026 - Added a formal requirement that Python tags must not start with From 0d1d980a6645649843d81157b3db4d2e1375feb2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 15 Jul 2026 15:22:29 +0200 Subject: [PATCH 02/28] PEP 825: Add "removing ordering information" to rejected ideas MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny Co-authored-by: Ralf Gommers --- peps/pep-0825.rst | 52 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 0a36ed50041..260d1d47035 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -1118,6 +1118,58 @@ justifies the benefit of more compact, and slightly more consistent naming. +Removing ordering information from wheel files +---------------------------------------------- + +The specification proposes that all the data needed to order wheel +variants is stored within the variant metadata (the `default +priorities`_ dictionary). This data is expected to originate from a +common source per project (a subsequent PEP will propose an integration +within ``pyproject.toml`` file) and to be copied into every variant +wheel at build time, and afterwards into the `index-level metadata`_ +file. + +On the discussion thread concerning this PEP, `an extensive argument +regarding the data model was made +`__. +Essentially, the argument was that this data becomes a wheel-level +property that, when inserted in a particular wheel, is saying something +about "other wheels". We do not believe this is the case. Rather, the +data is project-level metadata which is copied into the wheel. +Similarly to other project metadata, there is no reference to other +wheels. For example, the ordering data can specify namespaces for which +no variant wheels exist in a particular release; nothing in the format +depends on the presence of specific other wheels. + +Furthermore, this design specifically ensures that the index-level +metadata file is a cache rather than a first-order data source. Since +all the data is stored in the wheels: + +1. It is possible to generate the index-level metadata file based on + available wheels alone, without any additional input data. Notably, + this empowers indexes to generate it automatically without requiring + any changes to the tools or workflows used to upload wheels. + +2. It is possible to select across multiple wheels even in absence of + index-level metadata file, for example when installing from a local + directory. + +There is indeed a real risk that two wheels built at different times +and with different tooling may end up having inconsistent metadata. +However, the PEP specifically requires consistency, and expects the +tools to verify it. Furthermore, the problem is more general; such a +build scenario could lead to any other duplicated part of Core Metadata +being inconsistent across wheels and causing unexpected behavior in +tools. + +The post also makes a claim that detaching the ordering data from +variant metadata makes it possible for tools to accept an override of +the ordering data in a standard format. However, there is no relation +between that and the presence of ordering data in variant metadata; such +a format can be introduced either way (for example, using the subset of +variant metadata). + + Out of scope ------------ From cea39e7edd7d2fa926b1585d506ca05a74f73ae7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 15 Jul 2026 15:23:15 +0200 Subject: [PATCH 03/28] PEP 825: clarify variant metadata scopes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 260d1d47035..3b3e7d0a2d5 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -194,6 +194,10 @@ draft will be promoted to version ``1.0.0``. The top-level keys are described in the subsequent sections. +Except where noted otherwise, variant metadata is project-scoped. It is +copied into individual variant wheels, but it MUST be consistent across +different variant wheels created for a particular project version. + Schema '''''' @@ -242,8 +246,9 @@ Variants '''''''' The ``variants`` dictionary provides a mapping from variant labels -to variant properties. In the variant wheel, it MUST contain the label -present in that wheel's filename. +to variant properties. In an individual variant wheel, it is scoped to +that wheel and it MUST contain the label present in that wheel's +filename. It has 3 levels. The first level keys are variant labels, the second level keys are namespaces, the third level are feature names, and the @@ -321,14 +326,14 @@ as long as the values do not prevent correct operation. Tools MAY either use or ignore these values. This file uses the same structure as `variant metadata`_, except that -the ``variants`` object MUST list all variants available on the package -index for the package version in question. The tools MUST ensure that -the variant metadata across multiple variant wheels of the same package -version and the index-level metadata file is consistent. They MAY -require that keys other than ``variants`` have exactly the same values, -or they may carefully merge their values, provided that no conflicting -information is introduced, and the result is the same irrespective of -the order in which the wheels are processed. +the ``variants`` object is index-scope and it MUST list all variants +available on the package index for the package version in question. The +tools MUST ensure that the variant metadata across multiple variant +wheels of the same package version and the index-level metadata file is +consistent. They MAY require that keys other than ``variants`` have +exactly the same values, or they may carefully merge their values, +provided that no conflicting information is introduced, and the result +is the same irrespective of the order in which the wheels are processed. This file SHOULD NOT be considered immutable and MAY be updated in a backward compatible way at any point (e.g. when adding a new variant). From 9a8d23f228b4ac402ff50fb93a559f381cf84055 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 24 Jul 2026 04:12:37 +0200 Subject: [PATCH 04/28] PEP 825: add explicit "implementation requirements" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny Co-authored-by: konsti-openai --- peps/pep-0825.rst | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 3b3e7d0a2d5..5733c215dac 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -70,6 +70,26 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", document are to be interpreted as described in :rfc:`2119`. +Implementation requirements +--------------------------- + +This specification is written from the perspective of file format producers. +In the context of format definitions, the word "MUST" specifically +indicates that the requirement must be satisfied for the data to be +considered valid according to this specifications. + +Tools that produce the data formats according to this specification +(variant wheels, the index-level metadata file) MUST always produce +files that meet the requirements of this specification. + +Tools that consume the data formats according to this specification +SHOULD refuse to process the files that do not meet these requirements. +In this case, it is recommended to follow the same behavior as for +invalid Core Metadata in wheel files. For example, a wheel index may +reject the upload, while a package manager may skip the version +entirely. + + Variant wheel ------------- From f60039371bea1d9438190a9aefc0587ed41cdba8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jul 2026 15:38:20 +0200 Subject: [PATCH 05/28] PEP 825: Remove `feature` and `value` dicts from `default-priorities` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We do not really have a good use case for these, I have been leaning backwards to even provide example, the justification is limited to "completeness" and they're just adding complexity. I think it's better to remove them before we put more effort into adding consistency requirements. Signed-off-by: Michał Górny --- peps/pep-0825.rst | 118 ++++-------------- .../pep-0825/appendix-variant-json-schema.rst | 2 +- ...a-0.1.0.json => variant-schema-0.1.1.json} | 48 +------ 3 files changed, 30 insertions(+), 138 deletions(-) rename peps/pep-0825/{variant-schema-0.1.0.json => variant-schema-0.1.1.json} (57%) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 5733c215dac..6a8cf6b2be6 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -193,17 +193,12 @@ PEP defines the following structure: +- $schema +- default-priorities | +- namespace : list[str] - | +- feature - | +- {namespace} : list[str] = [] - | +- property - | +- {namespace} - | +- {feature} : list[str] = [] +- variants +- {variant_label} +- {namespace} +- {feature} : list[str] = [] -This structure corresponds to the version ``0.1.0`` of the format. The +This structure corresponds to the version ``0.1.1`` of the format. The version number is stored as part of the schema_ URL. Whenever the format changes, the version number must be incremented. Tools MUST assume that all variant wheels using an unknown format version are unsupported. @@ -239,8 +234,9 @@ include updated versions of the schema. The schema is available in Default priorities '''''''''''''''''' -The ``default-priorities`` dictionary defines the ordering of variants. -The exact algorithm is described in the `Variant ordering`_ section. +The ``default-priorities`` dictionary defines the ordering of +namespaces which is used in variant ordering. The exact algorithm is +described in the `Variant ordering`_ section. The following key is REQUIRED: @@ -248,19 +244,6 @@ The following key is REQUIRED: wheels for a given package version, ordered in decreasing priority. This list MUST contain all namespaces used in variant properties. -It MAY have the following OPTIONAL keys: - -- ``feature: dict[str, list[str]]``: A dictionary with namespaces as - keys, and ordered list of corresponding feature names as values. - The feature names are ordered in decreasing priority. It is used to - override the default feature ordering. - -- ``property: dict[str, dict[str, list[str]]]``: A nested dictionary - with namespaces as first-level keys, feature names as second-level - keys and ordered lists of corresponding property values as - second-level values. The feature values are ordered in decreasing - priority. It is used to override the default value ordering. - Variants '''''''' @@ -283,7 +266,7 @@ Example { // The schema URL will be replaced with the final URL on packaging.python.org - "$schema": "https://variants-schema.wheelnext.dev/peps/825/v0.1.0.json", + "$schema": "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json", "default-priorities": { // REQUIRED: specifies that x86_64 CPU properties are more important than @@ -291,20 +274,6 @@ Example // does not matter), and both are more important than specific BLAS/LAPACK // library: "namespace": ["x86_64", "aarch64", "blas_lapack"], - - // OPTIONAL: makes "library" the most important feature in "blas_lapack" - // namespace - "feature": { - "blas_lapack": ["library"] - }, - - // OPTIONAL: makes ["mkl", "openblas"] the most important values of - // "blas_lapack :: library" feature - "property": { - "blas_lapack": { - "library": ["mkl", "openblas"] - } - } }, "variants": { @@ -370,7 +339,7 @@ like: { // The schema URL will be replaced with the final URL on packaging.python.org - "$schema": "https://variants-schema.wheelnext.dev/peps/825/v0.1.0.json", + "$schema": "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json", "default-priorities": { // identical to above }, @@ -430,32 +399,11 @@ algorithm: ``default-priorities.namespace`` key from `index-level metadata`_. This is ``namespace_order`` in the example. -2. For every namespace: +2. For every namespace, take the ordered list of compatible feature + names obtained previously. This is ``feature_order`` in the example. - i. Construct the initial ordered list of feature names by copying the - value of the respective ``default-priorities.feature.{namespace}`` - key. - - ii. Take the ordered list of compatible feature names obtained - previously and iterate over it, in order. For every feature name - that is not present in the constructed list, append it to the - end. - - After this step, a list of ordered feature names is available for - every namespace. This is ``feature_order`` in the example. - -3. For every feature: - - i. Construct the initial ordered list of values by copying the value - of the respective - ``default-priorities.property.{namespace}.{feature_name}`` key. - - ii. Take the ordered list of compatible feature values obtained - previously and iterate over it, in order. For every value that is - not present in the constructed list, append it to the end. - - After this step, a list of ordered property values is available for - every feature. This is ``value_order`` in the example. +3. For every feature, take the ordered list of compatible values obtained + previously. This is ``value_order`` in the example. 4. For every group, determine the most preferred value corresponding to every variant feature present in the variant properties corresponding @@ -522,31 +470,22 @@ compatibility tags. # default-priorities dict from index-level metadata default_priorities = { "namespace": [...], # : list[str] - "feature": {...}, # : dict[str, list[str]] - "property": {...}, # : dict[str, dict[str, list[str]]] } - # 1. Construct the ordered list of namespaces. + # 1. Obtain the ordered list of namespaces from the variant metadata. namespace_order = default_priorities["namespace"] - feature_order = {} - value_order = {} - - for namespace in namespace_order: - # 2. Construct the ordered lists of feature names. - feature_order[namespace] = default_priorities["feature"].get(namespace, []) - for feature_name in get_compatible_feature_names(namespace): - if feature_name not in feature_order[namespace]: - feature_order[namespace].append(feature_name) - - value_order[namespace] = {} - for feature_name in feature_order[namespace]: - # 3. Construct the ordered lists of feature values. - value_order[namespace][feature_name] = ( - default_priorities["property"].get(namespace, {}).get(feature_name, []) - ) - for feature_value in get_compatible_feature_values(namespace, feature_name): - if feature_value not in value_order[namespace][feature_name]: - value_order[namespace][feature_name].append(feature_value) + # 2. Obtain the ordered lists of features. + feature_order = { + namespace: get_compatible_feature_names(namespace) + for namespace in namespace_order + } + # 3. Obtain the ordered lists of feature values. + value_order = { + namespace: { + feature_name: get_compatible_feature_values(namespace, feature_name) + for feature_name in feature_order[namespace] + } for namespace in namespace_order + } def best_value_property(namespace: str, feature_name: str, feature_values: str) -> str: @@ -726,7 +665,7 @@ Example ] [packages.variant_json] - "$schema" = "https://variants-schema.wheelnext.dev/peps/825/v0.1.0.json" + "$schema" = "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json" [packages.variant_json.default-priorities] namespace = [ "x86_64", "aarch64", "blas_lapack" ] @@ -819,9 +758,6 @@ To generate the ``{name}-{version}-variants.json`` file: - common keys of these dictionaries must have exactly the same value - ``default-priorities.namespace`` list can be replaced if the new value starts with the old value - - ``default-priorities.feature`` and ``default-priorities.value`` - keys can be added if they were not present in the previous - ``default-priorities.namespace`` value Installing wheels from multiple sources (non-normative) @@ -966,9 +902,7 @@ provided. However, namespaces are governed independently and considered on equal footing, and therefore there will be no standard ordering for them. Instead, the ordering of namespaces will be explicitly stated in the variants metadata, which in turn will be provided by the package -maintainer as part of the build process. For completeness, it will also -be possible to provide overrides for the ordering of features and values -via the same mechanism. +maintainer as part of the build process. In the vast majority of real use cases, ordering based on properties will suffice. However, in a pathological case two different variant @@ -1228,6 +1162,8 @@ Change History - Added a non-normative guidance for installing variant wheels from multiple sources. + - Removed ``default-priorities.feature`` + and ``default-priorities.value``. - 06-Apr-2026 diff --git a/peps/pep-0825/appendix-variant-json-schema.rst b/peps/pep-0825/appendix-variant-json-schema.rst index 6505ba74357..082374e286b 100644 --- a/peps/pep-0825/appendix-variant-json-schema.rst +++ b/peps/pep-0825/appendix-variant-json-schema.rst @@ -5,7 +5,7 @@ Appendix: JSON Schema for Variant Metadata ========================================== -.. literalinclude:: variant-schema-0.1.0.json +.. literalinclude:: variant-schema-0.1.1.json :language: json :linenos: :name: variant-json-schema diff --git a/peps/pep-0825/variant-schema-0.1.0.json b/peps/pep-0825/variant-schema-0.1.1.json similarity index 57% rename from peps/pep-0825/variant-schema-0.1.0.json rename to peps/pep-0825/variant-schema-0.1.1.json index 2b792fd1a8f..28266d43986 100644 --- a/peps/pep-0825/variant-schema-0.1.0.json +++ b/peps/pep-0825/variant-schema-0.1.1.json @@ -1,7 +1,7 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://variants-schema.wheelnext.dev/peps/825/v0.1.0.json", - "title": "Variant metadata, v0.1.0", + "$id": "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json", + "title": "Variant metadata, v0.1.1", "description": "The format for variant metadata (variant.json) and index-level metadata ({name}-{version}-variants.json)", "type": "object", "properties": { @@ -22,50 +22,6 @@ }, "minItems": 1, "uniqueItems": true - }, - "feature": { - "description": "Default feature priorities (by namespace)", - "type": "object", - "patternProperties": { - "^[a-z0-9_]+$": { - "description": "The most preferred features", - "type": "array", - "items": { - "type": "string", - "pattern": "^[a-z0-9_]+$" - }, - "minItems": 0, - "uniqueItems": true - } - }, - "additionalProperties": false, - "uniqueItems": true - }, - "property": { - "description": "Default property priorities (by namespace)", - "type": "object", - "patternProperties": { - "^[a-z0-9_]+$": { - "description": "Default property priorities (by feature name)", - "type": "object", - "patternProperties": { - "^[a-z0-9_]+$": { - "description": "The most preferred feature values", - "type": "array", - "items": { - "type": "string", - "pattern": "^[a-z0-9_.]+$" - }, - "minItems": 0, - "uniqueItems": true - } - }, - "additionalProperties": false, - "uniqueItems": true - } - }, - "additionalProperties": false, - "uniqueItems": true } }, "additionalProperties": false, From 2c5217ace1efed949ce5df8cad1cc3fdcc25159e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 28 Jul 2026 21:03:42 +0200 Subject: [PATCH 06/28] PEP 825: use semantic versioning for the schema version MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expand the schema versioning to strictly follow semantic versioning. That is, major versions represent breaking changes and require tools to reject incompatible versions, whereas minor versions represent backwards compatible changes and permit tools to accept them. Signed-off-by: Michał Górny Co-authored-by: Ralf Gommers --- peps/pep-0825.rst | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 6a8cf6b2be6..82663fa7899 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -199,13 +199,26 @@ PEP defines the following structure: +- {feature} : list[str] = [] This structure corresponds to the version ``0.1.1`` of the format. The -version number is stored as part of the schema_ URL. Whenever the format -changes, the version number must be incremented. Tools MUST assume that -all variant wheels using an unknown format version are unsupported. - -The version numbers starting with zero are reserved for drafts and MUST -NOT be used in production. Once the proposal is complete, the latest -draft will be promoted to version ``1.0.0``. +version number is stored as part of the schema_ URL. The version numbers +follow semantic versioning. + +The numbers starting with zero are reserved for drafts and MUST NOT be +used in production. Tools MUST NOT make any compatibility assumptions +over these versions. Once the proposal is complete, the latest draft +will be promoted to version ``1.0.0``. + +If a backwards incompatible change is done to the specification, +the major version number MUST be incremented, and the remaining version +components MUST be zeroed. Tools MUST reject metadata with a major +version number that they do not support. + +If a backwards compatible change is done to the specification, the minor +version number MUST be incremented, and the patch number MUST be zeroed. +Installers and other tools that only consume variant metadata SHOULD +accept metadata with a newer minor version number than the latest +supported, provided that the major version is supported. Tools +outputting variant metadata MUST NOT output a version that they do not +explicitly support. The top-level keys are described in the subsequent sections. @@ -849,6 +862,15 @@ issues because of their length, the property lists are stored inside the wheel and mapped to a short label that is chosen by the package maintainer and intended to be human-readable. +Variant metadata is stored in an additional JSON format file rather than +being added to the :doc:`packaging:specifications/core-metadata`. It is +versioned independently, and tools that are not specifically concerned +about variant metadata can ignore its compatibility rules. It follows +the same compatibility rules as those for Core Metadata. JSON provides +a more convenient format for structured data, as well as more natural +conversion from TOML (so that the data can be sourced from +``pyproject.toml``). + Wheel filenames alone do not provide sufficient metadata to drive variant wheel selection. To avoid tools having to fetch the variant metadata straight from multiple wheel files, the metadata from wheels @@ -1164,6 +1186,8 @@ Change History multiple sources. - Removed ``default-priorities.feature`` and ``default-priorities.value``. + - Made schema versioning use semantic versioning, with its backwards + compatibility impliciations. - 06-Apr-2026 From 740ae19a3b6f1834ff06ce2864c98546b6e23756 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jul 2026 20:17:51 +0200 Subject: [PATCH 07/28] PEP 825: define consistency requirements more explicitly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 82663fa7899..81c9329d23e 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -220,11 +220,8 @@ supported, provided that the major version is supported. Tools outputting variant metadata MUST NOT output a version that they do not explicitly support. -The top-level keys are described in the subsequent sections. - -Except where noted otherwise, variant metadata is project-scoped. It is -copied into individual variant wheels, but it MUST be consistent across -different variant wheels created for a particular project version. +The top-level keys, as well as their scope and consistency requirements, +are described in the subsequent sections. Schema @@ -257,6 +254,12 @@ The following key is REQUIRED: wheels for a given package version, ordered in decreasing priority. This list MUST contain all namespaces used in variant properties. +Default priorities are defined at the project scope. Different wheels +SHOULD use the same value. However, appending additional namespaces is +permitted. The metadata is considered consistent if the longer list +starts with the elements of the shorter list, in order. In this case, +combining the metadata MUST result in the longer list being used. + Variants '''''''' @@ -271,6 +274,10 @@ level keys are namespaces, the third level are feature names, and the third level values are sets of feature values, converted to lists, sorted lexically. +For the metadata to be consistent, the same keys MUST always correspond +to the same values. When combining metadata, the resulting ``variants`` +dictionary MUST be an union of all the input dictionaries. + Example ''''''' @@ -332,10 +339,7 @@ the ``variants`` object is index-scope and it MUST list all variants available on the package index for the package version in question. The tools MUST ensure that the variant metadata across multiple variant wheels of the same package version and the index-level metadata file is -consistent. They MAY require that keys other than ``variants`` have -exactly the same values, or they may carefully merge their values, -provided that no conflicting information is introduced, and the result -is the same irrespective of the order in which the wheels are processed. +consistent, as defined in sections corresponding to the specific keys. This file SHOULD NOT be considered immutable and MAY be updated in a backward compatible way at any point (e.g. when adding a new variant). @@ -810,8 +814,8 @@ source provides compatible variant wheels, those variants can be selected ahead of non-variant wheels, as generally variant wheels are preferred over non-variant wheels. If multiple sources provide compatible variants, their metadata can be combined unambiguously only -when the metadata is consistent as specified in the `index-level -metadata`_ section. +when the metadata is consistent as specified in the sections +corresponding to the individual metadata keys. When metadata cannot be combined unambiguously, there is no uniquely correct global ordering. Valid tool-specific choices include: From 340b79e5622d819555a4c0683efc58e55834d92c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 4 Aug 2026 14:35:33 +0200 Subject: [PATCH 08/28] PEP 825: add a specific installation example MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 118 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 81c9329d23e..807ec4ce347 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -777,6 +777,124 @@ To generate the ``{name}-{version}-variants.json`` file: value starts with the old value +Installation example (non-normative) +------------------------------------ + +Let's say that PyTorch publishes a number of variant wheels: + +.. code-block:: text + + torch-2.13.0-{py}-{abi}-{platform}-cuda12.6.whl + ^^^^^^^^ + torch-2.13.0-{py}-{abi}-{platform}-cuda13.0.whl + ^^^^^^^^ + torch-2.13.0-{py}-{abi}-{platform}-cuda13.2.whl + ^^^^^^^^ + torch-2.13.0-{py}-{abi}-{platform}-rocm7.2.whl + ^^^^^^^ + torch-2.13.0-{py}-{abi}-{platform}-null.whl # CPU-only + ^^^^ + +The highlighted filename parts are the variant label. + +Each of these wheels carries a variant metadata file that contains: + +.. code:: json5 + + { + "$schema": "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json", + "default-priorities": { + "namespace": ["nvidia", "amd"] + }, + "variants": { + // ... + } + } + +In every wheel, the ``variants`` dictionary contains a single key that +is the variant label and whose value lists all properties corresponding +to that label. For example, the ``cuda*`` variant wheels contain +properties expressing the compatibility with NVIDIA GPUs, whereas +``rocm*`` variant wheels the compatibility with AMD GPUs. The ``null`` +variant has no properties. + +In addition to these variant wheels, a ``torch-2.13.0-variants.json`` +file is published with the variant metadata merged from individual +wheels. It has the same contents as the example, except that the +``variants`` dictionary includes all variant labels and their +properties. + +When a package manager is requested to install ``torch``, in order: + +1. The index is queried to determine available wheels. It is + established that 2.13.0 is the newest version and it is selected. + +2. The available 2.13.0 wheels are filtered by Platform compatibility + tags. Only wheels that are compatible with the current system remain. + +3. The ``torch-2.13.0-variants.json`` file is found in the index + response and it is downloaded. Its contents are read to determine the + mapping from variant labels to sets of variant properties, as well as + the namespace preference order. + +4. The lists of compatible features and feature values are obtained for + all namespaces that are used in the ``variants`` dictionary. Wheels + with variant labels corresponding to properties that aren't on these + lists are incompatible and are filtered out. + + For example: + + - The ``cuda*`` labels map to a + ``nvidia :: cuda_version_lower_bound`` property whose value + specifies the minimum CUDA driver version, and + ``nvidia :: sm_arch`` properties whose values list supported GPUs. + The installer queries the driver (if available) to determine + whether a compatible runtime and one of the compatible GPUs are + available. If it cannot find the driver, a compatible runtime + version or a compatible GPU, the wheels are removed from the list. + + - Similarly, the ``rocm*`` labels map to a ``amd :: rocm_version`` + property that specifies the supported ROCm version and + ``amd :: gfx_arch`` properties that list supported GPUs. The + installer queries the appropriate driver in a similar manner as + before. If it cannot find the driver, a compatible runtime version + or a compatible GPU, the wheels are removed from the list. + + - The ``null`` label always corresponds to an empty property set, + and it is therefore always compatible. + + On a system with a compatible NVIDIA GPU and a CUDA runtime, at least + one of the ``cuda*`` wheels and the ``null`` wheels will remain on + the list. + +5. If variant wheels with multiple different labels remain on the list, + the results are sorted per the algorithm in `variant ordering`_. + The most preferred label is selected. + + For example, in this case the ``cuda*`` wheels are ordered by their + properties, using the lists obtained in step 4. The wheels for CUDA + 13 will sort before the ones for CUDA 12, as newer CUDA versions are + preferred. The ``null`` wheel always sorts last, so it would be + selected only if none of the GPU wheels were compatible. + +6. If at this point multiple wheels with the same label remain on the + list, the final selection is performed based on Platform + compatibility tags. This is a rare occurrence and it does not apply + here. + + It could happen, for example, if the same wheel variant was + provided both with ``abi3`` and ``cp315`` tags. When installing for + Python 3.15, the installer would choose between these two based on + the tag. + +7. The metadata for the selected wheel is processed. The variant + properties corresponding to the wheel (as stored in the ``variants`` + dictionary) are used to process `environment markers`_. This results + in additional CUDA-related dependencies being selected. + +8. The wheel is downloaded and installed. + + Installing wheels from multiple sources (non-normative) ------------------------------------------------------- From 3b4ba37024b85b41fc18f88a25f0f92ae47924b6 Mon Sep 17 00:00:00 2001 From: rgommers Date: Mon, 3 Aug 2026 18:13:57 +0000 Subject: [PATCH 09/28] PEP 825: fix typos and grammar Use the current Discourse thread slug in the header URLs; the old pep-817 slug still redirects, but no longer names the PEP. --- peps/pep-0825.rst | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 807ec4ce347..60e535b3e60 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -12,12 +12,12 @@ Author: Jonathan Dekhtiar , Donald Stufft , Andy R. Terrel PEP-Delegate: Paul Moore -Discussions-To: https://discuss.python.org/t/pep-817-split-wheel-variants-package-format/106196 +Discussions-To: https://discuss.python.org/t/pep-825-wheel-variants-package-format-split-from-pep-817/106196 Status: Draft Type: Standards Track Topic: Packaging Created: 17-Feb-2026 -Post-History: `17-Feb-2026 `__ +Post-History: `17-Feb-2026 `__ Abstract ======== @@ -76,7 +76,7 @@ Implementation requirements This specification is written from the perspective of file format producers. In the context of format definitions, the word "MUST" specifically indicates that the requirement must be satisfied for the data to be -considered valid according to this specifications. +considered valid according to this specification. Tools that produce the data formats according to this specification (variant wheels, the index-level metadata file) MUST always produce @@ -254,11 +254,12 @@ The following key is REQUIRED: wheels for a given package version, ordered in decreasing priority. This list MUST contain all namespaces used in variant properties. -Default priorities are defined at the project scope. Different wheels -SHOULD use the same value. However, appending additional namespaces is -permitted. The metadata is considered consistent if the longer list -starts with the elements of the shorter list, in order. In this case, -combining the metadata MUST result in the longer list being used. +Default priorities are defined at the project scope. The value in +different wheels SHOULD be identical, or one an extension of the other: +additional namespaces MAY be appended. The metadata is considered +consistent if the longer list starts with the elements of the shorter +list, in the same order. In this case, combining the metadata MUST result +in the longer list being used. Variants @@ -270,13 +271,13 @@ that wheel and it MUST contain the label present in that wheel's filename. It has 3 levels. The first level keys are variant labels, the second -level keys are namespaces, the third level are feature names, and the -third level values are sets of feature values, converted to lists, -sorted lexically. +level keys are namespaces, and the third level keys are feature names. +The third level values are sets of feature values, converted to lists +and sorted lexically. For the metadata to be consistent, the same keys MUST always correspond to the same values. When combining metadata, the resulting ``variants`` -dictionary MUST be an union of all the input dictionaries. +dictionary MUST be a union of all the input dictionaries. Example @@ -335,7 +336,7 @@ as long as the values do not prevent correct operation. Tools MAY either use or ignore these values. This file uses the same structure as `variant metadata`_, except that -the ``variants`` object is index-scope and it MUST list all variants +the ``variants`` object is index-scoped and it MUST list all variants available on the package index for the package version in question. The tools MUST ensure that the variant metadata across multiple variant wheels of the same package version and the index-level metadata file is @@ -610,7 +611,7 @@ The ``variant_label`` marker is a plain string: # satisfied by the variant "foobar" dep4; variant_label == "foobar" - # satisfied by any wheel other other than the null variant + # satisfied by any wheel other than the null variant # (including the non-variant wheel) dep5; variant_label != "null" # satisfied by the non-variant wheel From 7136689340e420da71a265c6013554b2087af6c1 Mon Sep 17 00:00:00 2001 From: rgommers Date: Mon, 3 Aug 2026 18:14:27 +0000 Subject: [PATCH 10/28] PEP 825: correct and complete the Change History - The removed key was `default-priorities.property`, not `default-priorities.value`; the latter never existed. The error came from the publishing section, which had the same mistake. - List the four remaining changes made in this round: the "Implementation requirements" section, the per-key metadata scopes, the per-key consistency requirements, and the new rejected idea. - Move the 11-May-2026 entry into place; the list is otherwise in reverse chronological order. --- peps/pep-0825.rst | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 60e535b3e60..dac6ca8b76d 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -1305,12 +1305,23 @@ Change History - 03-Jul-2026 - - Added a non-normative guidance for installing variant wheels from + - Added non-normative guidance for installing variant wheels from multiple sources. + - Added an explicit "Implementation requirements" section. + - Clarified the scope of the individual variant metadata keys, and + stated the consistency requirements for each of them alongside. + - Added "Removing ordering information from wheel files" to rejected + ideas. - Removed ``default-priorities.feature`` - and ``default-priorities.value``. + and ``default-priorities.property``. - Made schema versioning use semantic versioning, with its backwards - compatibility impliciations. + compatibility implications. + +- 11-May-2026 + + - Added replacing platform compatibility tags entirely to rejected + ideas. + - Clarified interpretation of sorting algorithm and index support. - 06-Apr-2026 @@ -1338,12 +1349,6 @@ Change History - Changed ``pylock.toml`` integration to inline variant metadata rather than storing a URL and a hash. -- 11-May-2026 - - - Added replacing platform compatibility tags entirely to rejected - ideas. - - Clarified interpretation of sorting algorithm and index support. - Appendices ========== From ed3ba6d8db871d1408a9ce694b8eff40f5030d0a Mon Sep 17 00:00:00 2001 From: rgommers Date: Mon, 3 Aug 2026 18:14:48 +0000 Subject: [PATCH 11/28] PEP 825: settle on `[packages.variants-json]` in pylock.toml The prose introduced the subtable as `[packages.variants-json]`, while the proposed specification text and the example used `[packages.variant_json]`. Use the spelling with a dash throughout, that is consistent with PEP 751. --- peps/pep-0825.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index dac6ca8b76d..c1048bf2596 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -647,8 +647,8 @@ Proposed specification update The proposed text for :doc:`packaging:specifications/pylock-toml` follows: -``[packages.variant_json]`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +``[packages.variants-json]`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **Type**: table - **Required?**: no @@ -682,13 +682,13 @@ Example { url = "https://pypi.anaconda.org/mgorny/simple/numpy/2.3.4/numpy-2.3.4-cp314-cp314-macosx_13_0_x86_64-openblas.whl", hashes = {} }, ] - [packages.variant_json] + [packages.variants-json] "$schema" = "https://variants-schema.wheelnext.dev/peps/825/v0.1.1.json" - [packages.variant_json.default-priorities] + [packages.variants-json.default-priorities] namespace = [ "x86_64", "aarch64", "blas_lapack" ] - [packages.variant_json.variants] + [packages.variants-json.variants] null = { } x86_64_v3_openblas = { "blas_lapack" = { "library" = ["openblas"]}, "x86_64" = { "level" = ["v3"]} } x86_64_v4_mkl = { "blas_lapack" = { "library" = ["mkl"]}, "x86_64" = { "level" = ["v4"]} } From 90d3e123275bad15a036b2818eb119c759c251c3 Mon Sep 17 00:00:00 2001 From: rgommers Date: Mon, 3 Aug 2026 18:15:59 +0000 Subject: [PATCH 12/28] PEP 825: make the JSON schema match the specification Four of the schema's constraints disagreed with the normative text: - The variant label pattern still carried a 16-character cap (`^[a-z0-9_.]{1,16}$`), left over from before the label length limit was removed on 17-Feb-2026. The specification requires only `^[0-9a-z_.]+$`. The cap rejected this PEP's own example label, `x86_64_v3_openblas` (18 characters). - The namespace and feature name patterns permitted `.`, which the specification does not: both MUST match `^[a-z0-9_]+$`. The feature *value* pattern correctly keeps `.` and is unchanged. - The schema required `default-priorities.namespace` to be non-empty (`minItems: 1`), but the prose only required the list to contain all namespaces used in variant properties. State the requirement in the text as well: a package version providing variant wheels MUST use at least one variant namespace. - The subschema for a namespace's value declared no type, so an array there satisfied the schema; `patternProperties` and `additionalProperties` only constrain objects. The specification requires a mapping of feature names, so declare `"type": "object"`. Also drop the `uniqueItems` keywords whose instance is an object; `uniqueItems` only constrains arrays, so those never applied. The two on arrays are kept. These are bugs in the schema relative to the prose, so it is corrected in place rather than versioned: 0.x is reserved for drafts and the PEP states that tools must not make compatibility assumptions across them. --- peps/pep-0825.rst | 4 +++- peps/pep-0825/variant-schema-0.1.1.json | 16 ++++++---------- 2 files changed, 9 insertions(+), 11 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index c1048bf2596..b8b5c2095a8 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -252,7 +252,9 @@ The following key is REQUIRED: - ``namespace: list[str]``: All variant namespaces used in variant wheels for a given package version, ordered in decreasing priority. - This list MUST contain all namespaces used in variant properties. + This list MUST contain all namespaces used in variant properties, and + it MUST NOT be empty: a package version providing variant wheels MUST + use at least one variant namespace. Default priorities are defined at the project scope. The value in different wheels SHOULD be identical, or one an extension of the other: diff --git a/peps/pep-0825/variant-schema-0.1.1.json b/peps/pep-0825/variant-schema-0.1.1.json index 28266d43986..82288a313df 100644 --- a/peps/pep-0825/variant-schema-0.1.1.json +++ b/peps/pep-0825/variant-schema-0.1.1.json @@ -25,7 +25,6 @@ } }, "additionalProperties": false, - "uniqueItems": true, "required": [ "namespace" ] @@ -34,14 +33,15 @@ "description": "Mapping of variant labels to properties", "type": "object", "patternProperties": { - "^[a-z0-9_.]{1,16}$": { + "^[a-z0-9_.]+$": { "type": "object", "description": "Mapping of namespaces in a variant", "patternProperties": { - "^[a-z0-9_.]+$": { + "^[a-z0-9_]+$": { "description": "Mapping of feature names in a namespace", + "type": "object", "patternProperties": { - "^[a-z0-9_.]+$": { + "^[a-z0-9_]+$": { "description": "List of values for this variant feature", "type": "array", "items": { @@ -52,16 +52,13 @@ "uniqueItems": true } }, - "uniqueItems": true, "additionalProperties": false } }, - "uniqueItems": true, "additionalProperties": false } }, - "additionalProperties": false, - "uniqueItems": true + "additionalProperties": false } }, "required": [ @@ -69,6 +66,5 @@ "default-priorities", "variants" ], - "additionalProperties": false, - "uniqueItems": true + "additionalProperties": false } From a6177ef21570a496e54770b8528694e5b606bb90 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 5 Aug 2026 21:19:17 +0200 Subject: [PATCH 13/28] PEP 825: update the abstract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The abstract was largely unchanged from PEP 817, so update it to make it clearer what the scope of this PEP is. Avoid introducing unnecessary vocabulary while at it. Signed-off-by: Michał Górny --- peps/pep-0825.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index b8b5c2095a8..51a2f06b647 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -22,16 +22,17 @@ Post-History: `17-Feb-2026 Date: Thu, 6 Aug 2026 12:24:03 +0200 Subject: [PATCH 14/28] PEP 825: make it clear that index-level metadata is an optimization MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 51a2f06b647..8e1070bf7c9 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -321,12 +321,15 @@ Example Index-level metadata -------------------- -For every package version that includes at least one variant wheel, -there MUST exist a corresponding ``{name}-{version}-variants.json`` -file. The ``{name}`` and ``{version}`` placeholders correspond to the -package name and version, normalized according to the same rules as -wheel files, as found in the :ref:`packaging:wheel-file-name-spec` of -the Binary Distribution Format specification. +When a package version that includes at least one variant wheel is +hosted on an index, a corresponding ``{name}-{version}-variants.json`` +file MUST be hosted as well. The purpose of the file is to optimize +variant metadata lookups and remove the necessity of fetching multiple +variant wheels during dependency resolution. The ``{name}`` and +``{version}`` placeholders correspond to the package name and version, +normalized according to the same rules as wheel files, as found in the +:ref:`packaging:wheel-file-name-spec` of the Binary Distribution Format +specification. The exact URL where the file is hosted is insignificant, but it MUST be provided in all the responses where the variant wheels are included. From 8554dfecc72c8390273852ba5b2f42aad4511ec2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 6 Aug 2026 12:34:26 +0200 Subject: [PATCH 15/28] PEP 825: decouple variant ordering from index MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decouple the variant ordering algorithm from index, as requested on DPO. While most of the time it will be done against wheels from an index, the algorithm itself is generic, so replace the references to "index-level metadata" with more generic "combined variant metadata", indicating that it can either be obtained from an index or from wheels. Signed-off-by: Michał Górny --- peps/pep-0825.rst | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 8e1070bf7c9..6c33e2b7083 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -406,7 +406,13 @@ Variant ordering This specification defines an ordering between different wheels based on the presence of variant metadata. -For the purpose of ordering, variant properties are grouped into +For the purpose of ordering, the combined variant metadata for all +candidate variant wheels MUST be obtained. When installing from an +index, this data MUST be obtained from the `index-level metadata`_ file. +Otherwise, it MUST be obtained from the wheels directly and combined in +the way specified in `variant metadata`_ section. + +Variant properties from all the eligible variant wheels are grouped into features, and features into namespaces. For every namespace, the tool MUST obtain a list of compatible features, and for every feature, a list of compatible values. The method of obtaining these lists will be @@ -420,8 +426,8 @@ groups of variant wheels MUST then be ordered according to the following algorithm: 1. Construct the ordered list of namespaces by copying the value of the - ``default-priorities.namespace`` key from `index-level metadata`_. - This is ``namespace_order`` in the example. + ``default-priorities.namespace`` key from the combined variant + metadata. This is ``namespace_order`` in the example. 2. For every namespace, take the ordered list of compatible feature names obtained previously. This is ``feature_order`` in the example. @@ -491,7 +497,7 @@ compatibility tags. ... - # default-priorities dict from index-level metadata + # default-priorities dict from combined variant metadata default_priorities = { "namespace": [...], # : list[str] } From e81e012ce01cd983b92385a119af4e4cc254e9ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 6 Aug 2026 12:37:19 +0200 Subject: [PATCH 16/28] PEP 825: decouple `pylock.toml` from indexes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 6c33e2b7083..b488eaae42b 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -644,10 +644,11 @@ as wheels with different Platform compatibility tags: either all variant (and non-variant) wheels can be listed, or a subset of them. A new ``[packages.variants-json]`` subtable is added to the file. It -MUST inline the contents of the `index-level metadata`_ file, converting -the JSON structure into the respective TOML types. The ``$schema`` key -MUST be preserved to facilitate versioning. The tools MAY remove keys -that are not relevant to variant wheels present in ``pylock.toml``. +MUST inline combined variant metadata, following the same format as the +`index-level metadata`_ file, converting the JSON structure into the +respective TOML types. The ``$schema`` key MUST be preserved to +facilitate versioning. The tools MAY remove keys that are not relevant +to variant wheels present in ``pylock.toml``. If variant wheels are listed, the tool SHOULD resolve variants to select the best wheel file. @@ -1317,6 +1318,9 @@ Change History - 03-Jul-2026 + - Decoupled most of the specification from `index-level metadata`_, + clarifying that it is only an optimization for scenarios where + wheels are published on an index. - Added non-normative guidance for installing variant wheels from multiple sources. - Added an explicit "Implementation requirements" section. From 2060e5c733cd47d8cacb26d33bb4bf75575ebc21 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 7 Aug 2026 17:49:14 +0200 Subject: [PATCH 17/28] PEP 825: environment marker improvements MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The additions to Specification are as discussed on DPO. The Rationale additions aim to answer all the concerns and misunderstandings we've seen in discussions of the design of variant environment markers. Signed-off-by: Michał Górny Co-authored-by: Ralf Gommers --- peps/pep-0825.rst | 205 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 174 insertions(+), 31 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index b488eaae42b..6fcded2fd6e 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -590,50 +590,107 @@ Environment markers ------------------- Four new :ref:`environment markers -` are introduced in -dependency specifications: - -1. ``variant_namespaces`` corresponding to the set of namespaces of all - the variant properties that the wheel variant was built for. -2. ``variant_features`` corresponding to the set of - ``namespace :: feature`` pairs of all the variant properties that the - wheel variant was built for. -3. ``variant_properties`` corresponding to the set of - ``namespace :: feature :: value`` tuples of all the variant - properties that the wheel variant was built for. -4. ``variant_label`` corresponding to the exact variant label that the - wheel was built with. For the non-variant wheel, it is an empty +` are introduced in +dependency specifications. Unlike the markers defined by the +:ref:`packaging:dependency-specifiers` specification, their values are +not the same for every wheel: they are scoped to the variant properties +that the wheel being processed was built for. They MUST be obtained as +described in `evaluating variant markers`_. These are: + +1. ``variant_label``: a string, expressing the exact variant label of + the wheel being processed. For non-variant wheels, it is an empty string. +2. ``variant_properties``: a set of all + ``namespace :: feature :: value`` tuples of the properties + corresponding to the variant label that are compatible with the + target system. For non-variant wheels, it is an empty set. +3. ``variant_features``: a set of all ``namespace :: feature`` pairs + corresponding to the properties in ``variant_properties``. +4. ``variant_namespaces``: a set of all namespaces of all the properties + in ``variant_properties``. -The markers evaluating to sets of strings MUST be matched via the ``in`` -or ``not in`` operator, e.g.: +``variant_label`` is a ``String`` field, while ``variant_properties``, +``variant_features`` and ``variant_namespaces`` are ``Set of String`` +fields. The operators available for each, and their semantics, are those +defined for the respective field types in +:ref:`packaging:dependency-specifiers`. -.. code:: +Implementations MUST ignore differences in whitespace around the ``::`` +separators when matching features and properties. - # satisfied by any "foo :: * :: *" property - dep1; "foo" in variant_namespaces - # satisfied by any "foo :: bar :: *" property - dep2; "foo :: bar" in variant_features - # satisfied only by "foo :: bar :: baz" property - dep3; "foo :: bar :: baz" in variant_properties -The ``variant_label`` marker is a plain string: +Evaluating variant markers +'''''''''''''''''''''''''' + +The variant markers MUST only be used in dependency specifiers, and they +MUST NOT take part in selecting a wheel: they gate the individual +dependency specifiers of a wheel that has already been selected. They +MUST be evaluated only once variant wheel selection, as described in +`variant ordering`_, has taken place. + +Their values MUST be determined as follows: + +1. ``variant_label`` is the variant label of the selected wheel, as + found in its filename. + +2. The properties that the ``variants`` dictionary of the `variant + metadata`_ maps to that label are taken, and expanded into + ``namespace :: feature :: value`` triples for every listed feature + value. Either the `variant metadata`_ contained in the wheel itself + or the `index-level metadata`_ MAY be used for this, as the + consistency requirements guarantee that the two agree. + +3. That set is filtered to the variant properties that the target system + supports, as already determined during variant wheel selection. + ``variant_properties`` is the result. + +4. ``variant_features`` and ``variant_namespaces`` are derived from + ``variant_properties``. + +For non-variant wheels, ``variant_label`` is an empty string and the +three set-valued markers are empty sets. No variant metadata is needed +in order to evaluate variant markers for such wheels. + + +Example +''''''' + +The following dependency specifiers illustrate the four markers and the +operators permitted with each of them: .. code:: # satisfied by the variant "foobar" - dep4; variant_label == "foobar" + dep1; variant_label == "foobar" # satisfied by any wheel other than the null variant # (including the non-variant wheel) - dep5; variant_label != "null" + dep2; variant_label != "null" # satisfied by the non-variant wheel - dep6; variant_label == "" + dep3; variant_label == "" + # satisfied by any "foo :: * :: *" property + dep4; "foo" in variant_namespaces + # satisfied by any "foo :: bar :: *" property + dep5; "foo :: bar" in variant_features + # satisfied only by "foo :: bar :: baz" property + dep6; "foo :: bar :: baz" in variant_properties + # equivalent + dep7; "foo::bar::baz" in variant_properties + +The filtering in step 3. is only observable where a variant feature +lists multiple values, of which the target system needs to support just +one (see `variant properties`_). Consider a wheel whose properties +include both of: + +.. code:: text -Implementations MUST ignore differences in whitespace while matching the -features and properties. + nvidia :: sm_arch :: 120_real + nvidia :: sm_arch :: 110_real -Variant marker expressions MUST be evaluated against the variant -properties stored in the wheel being installed. +On a system that provides both architectures, ``variant_properties`` +contains both properties. On a system that provides only the former, it +contains ``nvidia :: sm_arch :: 120_real`` alone, and a dependency +specifier testing for ``nvidia :: sm_arch :: 110_real`` is therefore not +satisfied. Integration with pylock.toml @@ -739,10 +796,15 @@ behavior would be to: 9. If multiple wheels for a given version share the same variant label, order them by Platform compatibility tags and build number, and select the best wheel. +10. Read the dependencies of the selected wheel, and evaluate the + `environment markers`_ occurring in them, using the label of the + selected wheel and those of its variant properties that the target + system supports, as described in `evaluating variant markers`_. Note that steps 4. through 8. are introduced specifically for variant wheels. The remaining steps correspond to the current installer -behavior. +behavior. Step 10. is modified through the presence of new environment +markers. When installing from a source that does not provide an `index-level metadata`_, the same algorithm can be used, except that the variant @@ -763,6 +825,10 @@ to: 4. Obtain the ordered lists of compatible variant properties. The mechanism for this will be specified in a subsequent PEP. 5. Verify the wheel compatibility via compatible properties. +6. Read the dependencies of the wheel, and evaluate the `environment + markers`_ occurring in them, using the label of the wheel and those + of its variant properties that the target system supports, as + described in `evaluating variant markers`_. Publishing variant wheels on an index @@ -1084,6 +1150,64 @@ due to the variant metadata being updated or being generated in a way that does not guarantee stable bytewise output. +Variant environment markers +--------------------------- + +Variant properties take part in installing a variant wheel at two +distinct points, and only the second of them concerns markers. An +installer first selects a package version, filters the wheels for that +version by Platform compatibility tags, and then filters and orders the +remaining variant wheels using their variant properties, checked against +the properties that the target system supports; tools may override the +resulting choice. Only afterwards are the dependencies of the selected +wheel read, and the variant markers occurring in them evaluated. + +Filtering and ordering wheels is therefore driven by variant properties, +not by markers. Markers never gate the selection of a wheel; they only +gate individual dependency specifiers, and they are evaluated only once +a wheel has been selected. Were it otherwise, a marker would have to be +evaluated before the wheel whose properties it refers to was known. + +The values of the three set-valued markers are filtered to the variant +properties that the target system supports. Without that step, a wheel +would pull in the dependencies of every value a variant feature lists, +rather than those of the value that is actually in use. + +Because of this filtering, the variant markers do not depart from the +established meaning of an environment marker. Every property in +``variant_properties`` is by construction supported by the target +system, so the three set-valued markers describe the environment, just +as the markers defined in :ref:`packaging:dependency-specifiers` do. +What is specific to variant wheels is the vocabulary rather than the +semantics. The existing markers expose a fixed set of environment +attributes to every wheel alike, whereas a variant property has to be +declared by the wheel before it can be observed at all: the wheel's +`variant metadata`_ determines which part of the environment its +dependency specifiers are able to see. ``variant_label`` is the +exception, as it names the selected variant and says nothing about the +environment beyond the fact that this variant was deemed compatible. + +Three consequences of this design are worth noting: + +- Filtering never removes an entire feature or namespace from a wheel + that the target system supports, since such a wheel has at least one + supported value for every feature it declares (see `variant + properties`_). For those wheels, ``variant_features`` and + ``variant_namespaces`` list every feature and namespace that the wheel + was built for. + +- For the null variant, ``variant_label`` is ``"null"`` and the three + set-valued markers are empty sets, as the null variant has zero + properties. ``variant_label`` is consequently the only marker that + distinguishes a null variant from a non-variant wheel. + +- A marker referring to a property, feature or namespace that the wheel + does not declare cannot be satisfied in any environment, since + filtering only ever removes properties. Such markers can therefore be + resolved at build time, which is what makes the partial evaluation + described in `Backwards Compatibility`_ possible. + + Backwards Compatibility ======================= @@ -1298,6 +1422,22 @@ The following problems are deferred to subsequent PEPs in the series: - building variant wheels +Open Issues +=========== + +These questions must be resolved before this PEP can be accepted. + +Use of variant environment markers +---------------------------------- + +The design of the variant `environment markers`_ is not yet settled. The same +effect can be achieved through Dynamic dependencies; the open question is +whether markers are the right mechanism for obtaining it while keeping +dependency metadata static across a release. This is under discussion in +`this thread `__. +The specification reflects the current design. + + Acknowledgements ================ @@ -1332,6 +1472,9 @@ Change History and ``default-priorities.property``. - Made schema versioning use semantic versioning, with its backwards compatibility implications. + - Made ``variant_properties`` environment marker use variant + properties compatible with the system rather than all the properties + specified in the metadata. - 11-May-2026 From 88d79b211b39ce144dc9fcfdc740054d881ae0a7 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Fri, 7 Aug 2026 21:21:44 +0200 Subject: [PATCH 18/28] PEP 825: make the rejected idea read standalone Two changes to "Removing ordering information from wheel files": - Avoid a link to a specific post (this was my suggestion, changed my mind - apologies) - Reduce the length of that section; acknowledge there is a risk. We will address this by the overall argument for metadata consistency that is still to come. --- peps/pep-0825.rst | 39 ++++++++++++++++----------------------- 1 file changed, 16 insertions(+), 23 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 6fcded2fd6e..741dbc0bb1c 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -1370,17 +1370,14 @@ within ``pyproject.toml`` file) and to be copied into every variant wheel at build time, and afterwards into the `index-level metadata`_ file. -On the discussion thread concerning this PEP, `an extensive argument -regarding the data model was made -`__. -Essentially, the argument was that this data becomes a wheel-level -property that, when inserted in a particular wheel, is saying something -about "other wheels". We do not believe this is the case. Rather, the -data is project-level metadata which is copied into the wheel. -Similarly to other project metadata, there is no reference to other -wheels. For example, the ordering data can specify namespaces for which -no variant wheels exist in a particular release; nothing in the format -depends on the presence of specific other wheels. +It has been argued that this makes the ordering data a wheel-level +property which, once inserted into a particular wheel, says something +about other wheels. That is not the case. The data is project-level +metadata that is copied into the wheel, and like the other project +metadata carried there, it does not reference other wheels. The ordering +data can specify namespaces for which no variant wheels exist in a +particular release, and nothing in the format depends on the presence of +any specific other wheel. Furthermore, this design specifically ensures that the index-level metadata file is a cache rather than a first-order data source. Since @@ -1397,18 +1394,14 @@ all the data is stored in the wheels: There is indeed a real risk that two wheels built at different times and with different tooling may end up having inconsistent metadata. -However, the PEP specifically requires consistency, and expects the -tools to verify it. Furthermore, the problem is more general; such a -build scenario could lead to any other duplicated part of Core Metadata -being inconsistent across wheels and causing unexpected behavior in -tools. - -The post also makes a claim that detaching the ordering data from -variant metadata makes it possible for tools to accept an override of -the ordering data in a standard format. However, there is no relation -between that and the presence of ordering data in variant metadata; such -a format can be introduced either way (for example, using the subset of -variant metadata). +However, the specification requires consistency, and states how tools +are expected to respond where it does not hold. + +It has also been suggested that detaching the ordering data from variant +metadata would make it possible for tools to accept an override of that +data in a standard format. However, there is no relation between the +two; such a format can be introduced either way, for example using a +subset of variant metadata. Out of scope From 2d203274a2b5a57161e2de6c637b69f966143e92 Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Fri, 7 Aug 2026 21:48:09 +0200 Subject: [PATCH 19/28] PEP 825: gather the consistency requirements into one section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Paul asked for this new section, and it's actually helpful to have a section titled "Metadata consistency" I think. It shows there aren't that many requirements, and explains it's auto-fulfilled when building from a single source. Also one small fix in the "Index-level metadata" section. Co-authored-by: Michał Górny --- peps/pep-0825.rst | 50 +++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 44 insertions(+), 6 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 741dbc0bb1c..d0dc6fb4df0 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -343,10 +343,9 @@ use or ignore these values. This file uses the same structure as `variant metadata`_, except that the ``variants`` object is index-scoped and it MUST list all variants -available on the package index for the package version in question. The -tools MUST ensure that the variant metadata across multiple variant -wheels of the same package version and the index-level metadata file is -consistent, as defined in sections corresponding to the specific keys. +available on the package index for the package version in question. It +MUST be consistent with the `variant metadata`_ of the individual +wheels, as specified in `metadata consistency`_. This file SHOULD NOT be considered immutable and MAY be updated in a backward compatible way at any point (e.g. when adding a new variant). @@ -400,6 +399,45 @@ like: } +Metadata consistency +-------------------- + +The `variant metadata`_ carried by the individual variant wheels of a +package version, and the `index-level metadata`_ file where one is +published, all describe the same release, and are required to agree with +one another. Gathered in one place, and stated in full in the sections +defining the respective keys, the requirements are: + +- ``default-priorities.namespace``: the lists MUST either be identical, + or the longer MUST start with the elements of the shorter one, in the + same order. Combining them MUST yield the longer list. + +- ``variants``: the same variant label MUST always map to the same set + of properties. Combining them MUST yield the union of the dictionaries. + +Both rules are symmetric, so combining metadata that satisfies them +gives the same result regardless of the order in which the inputs are +processed. + +Meeting these requirements is the responsibility of the publisher of the +package version. Since the data originates from a single source per +project and is copied into the wheels at build time, they are satisfied +by construction unless the wheels of one release are built from +different inputs. + +Tools consuming variant metadata MAY assume that these requirements are +met. This specification does not require them to verify it, and does not +prescribe what they do if they detect that it does not hold. + +Where a user draws wheels for the same package from more than one +source, no publisher is in a position to guarantee consistency with the +others. Ensuring that the sources being combined are consistent is then +the responsibility of the user. Tools are not required to detect or +resolve inconsistencies between sources; `installing wheels from +multiple sources (non-normative)`_ discusses what they can reasonably do +instead. + + Variant ordering ---------------- @@ -811,8 +849,8 @@ metadata`_, the same algorithm can be used, except that the variant metadata needs to be read directly from the wheels. -Installing a local wheel -'''''''''''''''''''''''' +Installing a specific local wheel +''''''''''''''''''''''''''''''''' When asked to install a local wheel file, the proposed behavior would be to: From d310406bdb3cc9becdea88d13ebb47d3870ec2c6 Mon Sep 17 00:00:00 2001 From: rgommers Date: Fri, 7 Aug 2026 10:11:49 +0000 Subject: [PATCH 20/28] PEP 825: finish decoupling the spec from the index-level file Follow-up to PR 72, which describes the index-level metadata file as an optimization. Several sections still assumed it was the authoritative source. Variant ordering no longer requires the index-level file. The combined metadata may be sourced from it or from the wheels; both are required to agree, and tools SHOULD prefer the file where available because it is cheaper. This also removes the need to say what "installing from an index" means, which was never defined and which the Rationale straddles by counting a webserver directory listing as an index. The suggested implementation logic gave two answers for a missing file: step 5 treated the variant wheels as incompatible, while the paragraph below it said to read the metadata from the wheels. Keep the latter, and make it clear that the cost is not as high as one might think at first (because previous filtering). Also make declining an option. The section is non-normative, and a tool that finds the remaining cost unacceptable may treat the variant wheels as incompatible instead. The case the Rejected Ideas argument rests on is installing from a local directory, where the wheels are already present and the fallback costs nothing. Publishing no longer restates the merge rules (the description had drifted). The multi-source section justified leaving behavior undefined on the grounds that the metadata is scoped at index level - and we decoupled that. Finally, split the response to invalid data by the role of the tool. Requiring consumers to refuse it meant an installer skipping a version outright, so one malformed wheel could act as a de facto yank for every user. Indexes should reject at upload; installers should degrade. --- peps/pep-0825.rst | 83 ++++++++++++++++++++++++++--------------------- 1 file changed, 46 insertions(+), 37 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index d0dc6fb4df0..6cdae67817b 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -84,11 +84,15 @@ Tools that produce the data formats according to this specification files that meet the requirements of this specification. Tools that consume the data formats according to this specification -SHOULD refuse to process the files that do not meet these requirements. -In this case, it is recommended to follow the same behavior as for -invalid Core Metadata in wheel files. For example, a wheel index may -reject the upload, while a package manager may skip the version -entirely. +SHOULD NOT rely on data that does not meet these requirements. The +appropriate response depends on the role of the tool, and follows the +same pattern as for invalid Core Metadata in wheel files. Tools that are +in a position to reject invalid data at the point it enters the +ecosystem, such as a package index accepting an upload, SHOULD do so. +Tools that encounter it later, such as an installer resolving a +dependency, SHOULD prefer to degrade gracefully rather than fail +outright, for example by ignoring the variant wheels and selecting among +the remaining ones. Variant wheel @@ -445,10 +449,12 @@ This specification defines an ordering between different wheels based on the presence of variant metadata. For the purpose of ordering, the combined variant metadata for all -candidate variant wheels MUST be obtained. When installing from an -index, this data MUST be obtained from the `index-level metadata`_ file. -Otherwise, it MUST be obtained from the wheels directly and combined in -the way specified in `variant metadata`_ section. +candidate variant wheels MUST be obtained. It can be sourced either from +the `index-level metadata`_ file, or from the individual wheels, +combined as specified in the `variant metadata`_ section. Both sources +yield the same result, since they are required to be consistent, and +tools SHOULD prefer the index-level metadata file where it is available, +as obtaining the data from it is considerably cheaper. Variant properties from all the eligible variant wheels are grouped into features, and features into namespaces. For every namespace, the tool @@ -817,14 +823,19 @@ behavior would be to: 3. Filter available wheels based on Platform Compatibility Tags. 4. Determine if any of the remaining wheels are variant wheels. If not, proceed as with non-variant wheels. -5. If any wheels feature a `variant label`_, download the `index-level - metadata`_ file, ``{name}-{version}-variants.json``. If this - file is missing, assume all variant wheels are incompatible and - proceed as with non-variant wheels. +5. If any wheels feature a `variant label`_, obtain the combined variant + metadata. Normally this means downloading the `index-level + metadata`_ file, ``{name}-{version}-variants.json``. If the source + does not provide that file, the combined metadata can instead be read + from the candidate wheels. Only one wheel per distinct variant label + needs to be inspected, and only the `variant metadata`_ file within + it, rather than the whole wheel. Where this is still too expensive, a + tool may treat the variant wheels as incompatible and proceed with + the non-variant wheels instead. 6. Map the variant labels into sets of variant properties using the - index-level variant metadata file. If any of the labels present in - wheel filenames are missing in the file, assume that the respective - wheels are incompatible. + combined variant metadata. If any of the labels present in wheel + filenames are missing from it, assume that the respective wheels are + incompatible. 7. Obtain the ordered lists of compatible variant properties. The mechanism for this will be specified in a subsequent PEP. 8. Filter and order variants based on the lists of compatible @@ -844,9 +855,8 @@ wheels. The remaining steps correspond to the current installer behavior. Step 10. is modified through the presence of new environment markers. -When installing from a source that does not provide an `index-level -metadata`_, the same algorithm can be used, except that the variant -metadata needs to be read directly from the wheels. +The same algorithm applies to sources other than an index, such as a +local directory of wheels. Installing a specific local wheel @@ -882,17 +892,11 @@ If the index is responsible for generating the file, it should use some mechanism to defer publishing it until the release is fully uploaded (for example, :pep:`694`). -To generate the ``{name}-{version}-variants.json`` file: - -1. For the first variant wheel for a given package version, copy the - data from its ``*.dist-info/variant.json`` file. -2. For subsequent wheels, merge the data from their - ``*.dist-info/variant.json`` files into the existing data: - - - disjoint keys of ``variants`` dictionary are merged together - - common keys of these dictionaries must have exactly the same value - - ``default-priorities.namespace`` list can be replaced if the new - value starts with the old value +To generate the ``{name}-{version}-variants.json`` file, take the +``*.dist-info/variant.json`` files of all the variant wheels for a given +package version and combine them, as specified for the individual keys +in the `default priorities`_ and `variants`_ sections. The result does +not depend on the order in which the wheels are processed. Installation example (non-normative) @@ -1020,13 +1024,17 @@ As of the time of writing, there are no accepted standards addressing the support for installing packages from multiple sources, and the existing tools (such as ``pip`` and ``uv``) disagree on the exact behavior. This problem is described in more detail in the informational -:pep:`766`. Variant wheels expand the problem scope, as the variant -metadata that influences the selection among multiple wheels is scoped -at index level, and therefore the metadata acquired from different -indexes cannot be compared or combined in a meaningful way. For these -reasons, the specification does not attempt to standardize a behavior, -but instead considers it implementation-defined and provides a few -non-normative suggestions on the possible solutions. +:pep:`766`. Variant wheels expand the problem scope. The consistency +requirements that make variant metadata combinable hold within a single +project, and nothing obliges independent publishers to meet them with +respect to one another: the same variant label may map to different +properties, and namespace orderings chosen independently need not be +extensions of one another. Establishing whether the metadata from two +sources happens to be combinable is possible, but the cost of doing so +in the general case is prohibitive, and there is no correct answer when +it is not. For these reasons, the specification does not attempt to +standardize a behavior, but instead considers it implementation-defined +and provides a few non-normative suggestions on the possible solutions. It is entirely valid for tools not to support this behavior, either by not providing support for using multiple sources at all, or by rejecting @@ -1506,6 +1514,7 @@ Change History - Made ``variant_properties`` environment marker use variant properties compatible with the system rather than all the properties specified in the metadata. + - Various smaller fixes for language and design consistency. - 11-May-2026 From 613c69c0241fd04238414362f987c6b729d7bcdc Mon Sep 17 00:00:00 2001 From: rgommers Date: Fri, 7 Aug 2026 20:51:04 +0000 Subject: [PATCH 21/28] PEP 825: assorted wording fixes - The ``variants`` dictionary of an individual wheel is now required to contain exactly one entry rather than merely to contain the wheel's own label. Both the JSON example and the installation example already described it that way, so this settles the two in favour of the narrower reading. - Soften the claim that variant metadata "follows the same compatibility rules as those for Core Metadata". It does not: the versioning rules stated a few paragraphs earlier are semantic versioning, which is not Core Metadata's scheme. Say the versioning is similar in spirit. --- peps/pep-0825.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 6cdae67817b..d6774cb928b 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -214,7 +214,7 @@ will be promoted to version ``1.0.0``. If a backwards incompatible change is done to the specification, the major version number MUST be incremented, and the remaining version -components MUST be zeroed. Tools MUST reject metadata with a major +components MUST be zeroed. Tools MUST reject metadata with a major version number that they do not support. If a backwards compatible change is done to the specification, the minor @@ -274,8 +274,8 @@ Variants The ``variants`` dictionary provides a mapping from variant labels to variant properties. In an individual variant wheel, it is scoped to -that wheel and it MUST contain the label present in that wheel's -filename. +that wheel and it MUST contain exactly one entry, whose key is the +variant label present in that wheel's filename. It has 3 levels. The first level keys are variant labels, the second level keys are namespaces, and the third level keys are feature names. @@ -1113,8 +1113,8 @@ maintainer and intended to be human-readable. Variant metadata is stored in an additional JSON format file rather than being added to the :doc:`packaging:specifications/core-metadata`. It is versioned independently, and tools that are not specifically concerned -about variant metadata can ignore its compatibility rules. It follows -the same compatibility rules as those for Core Metadata. JSON provides +about variant metadata can ignore its compatibility rules. Its +versioning is similar in spirit to that of Core Metadata. JSON provides a more convenient format for structured data, as well as more natural conversion from TOML (so that the data can be sourced from ``pyproject.toml``). @@ -1495,7 +1495,7 @@ and Zanie Blue. Change History ============== -- 03-Jul-2026 +- 08-Aug-2026 - Decoupled most of the specification from `index-level metadata`_, clarifying that it is only an optimization for scenarios where From 0c9f1062edefd69f11549444a09db3293d38966f Mon Sep 17 00:00:00 2001 From: rgommers Date: Fri, 7 Aug 2026 20:53:01 +0000 Subject: [PATCH 22/28] PEP 825: give one answer on inconsistent metadata Three sections said different things about what a tool owes when the variant metadata is not consistent. Settle on one position: tools are not required to verify conformance; where they have established that the data does not conform, the response in "Implementation requirements" applies, split by role as before. --- peps/pep-0825.rst | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index d6774cb928b..8a1f4498265 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -83,11 +83,12 @@ Tools that produce the data formats according to this specification (variant wheels, the index-level metadata file) MUST always produce files that meet the requirements of this specification. -Tools that consume the data formats according to this specification -SHOULD NOT rely on data that does not meet these requirements. The -appropriate response depends on the role of the tool, and follows the -same pattern as for invalid Core Metadata in wheel files. Tools that are -in a position to reject invalid data at the point it enters the +Tools that consume the data formats according to this specification are +not required to verify that the data meets these requirements, but +SHOULD NOT rely on data that they have established does not meet them. +The appropriate response depends on the role of the tool, and follows +the same pattern as for invalid Core Metadata in wheel files. Tools that +are in a position to reject invalid data at the point it enters the ecosystem, such as a package index accepting an upload, SHOULD do so. Tools that encounter it later, such as an installer resolving a dependency, SHOULD prefer to degrade gracefully rather than fail @@ -430,8 +431,9 @@ by construction unless the wheels of one release are built from different inputs. Tools consuming variant metadata MAY assume that these requirements are -met. This specification does not require them to verify it, and does not -prescribe what they do if they detect that it does not hold. +met, and are not required to verify it. Where a tool does establish that +they are not met, the response described in `implementation +requirements`_ applies. Where a user draws wheels for the same package from more than one source, no publisher is in a position to guarantee consistency with the @@ -1440,8 +1442,9 @@ all the data is stored in the wheels: There is indeed a real risk that two wheels built at different times and with different tooling may end up having inconsistent metadata. -However, the specification requires consistency, and states how tools -are expected to respond where it does not hold. +However, the specification requires consistency and makes the publisher +of the package version responsible for it, as described in `metadata +consistency`_. It has also been suggested that detaching the ordering data from variant metadata would make it possible for tools to accept an override of that From a070403bcf10a745a63a04dc772f2f232dac46a7 Mon Sep 17 00:00:00 2001 From: rgommers Date: Fri, 7 Aug 2026 21:02:55 +0000 Subject: [PATCH 23/28] PEP 825: say how variant markers work in pylock.toml The pylock.toml section said nothing about markers, leaving a reader to work out whether a marker scoped to a selected wheel can be expressed in a lock file at all (discussed on DPO). The two places that have no such context follow from the existing rule that variant markers may only appear in dependency specifiers, so they are noted as a consequence. Also tighten the allowance to drop metadata from the inlined table - it was a bit too vague before. --- peps/pep-0825.rst | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 8a1f4498265..c754452542b 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -750,12 +750,26 @@ A new ``[packages.variants-json]`` subtable is added to the file. It MUST inline combined variant metadata, following the same format as the `index-level metadata`_ file, converting the JSON structure into the respective TOML types. The ``$schema`` key MUST be preserved to -facilitate versioning. The tools MAY remove keys that are not relevant -to variant wheels present in ``pylock.toml``. +facilitate versioning. The tools MAY remove the entries of the +``variants`` dictionary whose labels do not occur among the wheels +listed for the package, along with any namespaces that are thereby left +unused in ``default-priorities.namespace``. The entry for every label +that does occur MUST be retained in full, as `evaluating variant +markers`_ requires the complete set of properties corresponding to the +selected label. If variant wheels are listed, the tool SHOULD resolve variants to select the best wheel file. +Variant `environment markers`_ occurring in the dependency specifiers of +a locked package are evaluated in the context of the wheel selected for +that package's entry, as described in `evaluating variant markers`_. No +further context is needed in the lock file: the markers are evaluated +only once that wheel has been selected, and every package entry resolves +its own wheel. Since variant markers may only be used in dependency +specifiers, they cannot occur in ``packages.marker`` or in the top-level +``environments`` key, neither of which is scoped to a selected wheel. + Proposed specification update ''''''''''''''''''''''''''''' @@ -1514,9 +1528,10 @@ Change History and ``default-priorities.property``. - Made schema versioning use semantic versioning, with its backwards compatibility implications. - - Made ``variant_properties`` environment marker use variant - properties compatible with the system rather than all the properties - specified in the metadata. + - Improve environment marker content. Make ``variant_properties`` + marker use variant properties compatible with the system rather + than all the properties specified in the metadata. + - Update pylock.toml section to explain environment marker usage. - Various smaller fixes for language and design consistency. - 11-May-2026 From 882f8948395cf1903866acaa04410abc4266b7f0 Mon Sep 17 00:00:00 2001 From: rgommers Date: Fri, 7 Aug 2026 21:16:43 +0000 Subject: [PATCH 24/28] PEP 825: keep selection overrides inside the compatibility filter Filtering the marker values against what the target system supports assumes the selected wheel is one the system supports. Selection overrides could break that assumption: a tool offering "install this variant regardless of my hardware" would select a wheel whose properties are unsupported, the filter would reduce `variant_properties` to the empty set, and every dependency gated on those properties would disappear. The install would then be missing exactly the dependencies the chosen wheel needs, silently. Note that this is not a matter of ordering the pipeline differently. Filtering is an intersection with what the system supports, so it comes out empty whenever the override runs. Constrain the overrides instead: they act on the wheels already found compatible, and may reorder or narrow that set but not reach past it. Installing for a different system than the one being installed to is a question of what counts as supported, which is already deferred to a later PEP, so point there for that case. Also correct step 7. of the installation example, which described the markers as evaluated against the properties stored in the `variants` dictionary, without the filtering that the marker section requires. --- peps/pep-0825.rst | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index c754452542b..e24b89bfee4 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -521,7 +521,13 @@ sorted from the most preferred to the least preferred. The tools MAY provide options to override the default ordering, for example by specifying a preference for specific namespaces, features or properties. The tools MAY also provide options to exclude specific -variants, or to select a particular variant. +variants, or to select a particular variant. These options operate on +the wheels that were found compatible, so they MAY reorder or narrow +that set, but MUST NOT cause a wheel to be selected whose properties +the target system does not support. Installing a variant wheel for a +system other than the one being installed to is instead a matter of +overriding which properties are considered supported, which is `out of +scope`_ for this PEP. Alternatively, the sort algorithm for variant wheels could be described using the following pseudocode. For simplicity, this code does not @@ -688,7 +694,10 @@ Their values MUST be determined as follows: 3. That set is filtered to the variant properties that the target system supports, as already determined during variant wheel selection. - ``variant_properties`` is the result. + ``variant_properties`` is the result. Since a wheel can only be + selected if the target system supports it, and that requires at least + one supported value for every feature the wheel declares, this step + never removes a feature entirely. 4. ``variant_features`` and ``variant_namespaces`` are derived from ``variant_properties``. @@ -1026,9 +1035,10 @@ When a package manager is requested to install ``torch``, in order: the tag. 7. The metadata for the selected wheel is processed. The variant - properties corresponding to the wheel (as stored in the ``variants`` - dictionary) are used to process `environment markers`_. This results - in additional CUDA-related dependencies being selected. + properties corresponding to the wheel, narrowed to those the system + was found to support in step 4, are used to process `environment + markers`_. This results in additional CUDA-related dependencies being + selected. 8. The wheel is downloaded and installed. @@ -1251,12 +1261,15 @@ environment beyond the fact that this variant was deemed compatible. Three consequences of this design are worth noting: -- Filtering never removes an entire feature or namespace from a wheel - that the target system supports, since such a wheel has at least one - supported value for every feature it declares (see `variant - properties`_). For those wheels, ``variant_features`` and - ``variant_namespaces`` list every feature and namespace that the wheel - was built for. +- Filtering never removes an entire feature or namespace, since a wheel + can only be selected if the target system supports at least one value + for every feature it declares (see `variant properties`_). + ``variant_features`` and ``variant_namespaces`` therefore always list + every feature and namespace the wheel was built for. This is why the + overrides permitted in `variant ordering`_ may not reach past the + compatibility filter: a wheel selected in spite of being unsupported + could have its properties filtered away, and the dependencies gated on + them would silently disappear. - For the null variant, ``variant_label`` is ``"null"`` and the three set-valued markers are empty sets, as the null variant has zero From b75049fc1324165fedc8b998ad915876244ee25c Mon Sep 17 00:00:00 2001 From: Ralf Gommers Date: Sat, 8 Aug 2026 20:53:02 +0200 Subject: [PATCH 25/28] PEP 825: show why the marker values are filtered/narrowed The Rationale said only that without narrowing a wheel "would pull in the dependencies of every value a variant feature lists", which states the mechanism without giving a reader anything to picture, and the installation example illustrated a case where narrowing changes nothing observable. Between them the step looked like complexity with no purpose. Address that by: - Explicitly naming the main use case it exists for. - Note also when the alternative (the dependency also publishing variants) applies. There may be other reasons, like "my dependency has a bug for specific hardware". Those reasons also occur for other environment markers. However, that would make the text even longer - one example should suffice. In the end, the ability to treat GPU hardware like CPU families (selection-wise) is what matters. --- peps/pep-0825.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index e24b89bfee4..3f714a5b398 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -1040,6 +1040,17 @@ When a package manager is requested to install ``torch``, in order: markers`_. This results in additional CUDA-related dependencies being selected. + Suppose the wheel also depends on a library that supports only one + of the architectures the wheel supports: + + .. code:: text + + fast-gemm; "nvidia :: sm_arch :: 120_real" in variant_properties + + On a system with an older GPU, ``120_real`` is not among the + supported properties found in step 4, so the narrowing removes it and + this dependency is not selected. + 8. The wheel is downloaded and installed. @@ -1241,9 +1252,15 @@ a wheel has been selected. Were it otherwise, a marker would have to be evaluated before the wheel whose properties it refers to was known. The values of the three set-valued markers are filtered to the variant -properties that the target system supports. Without that step, a wheel -would pull in the dependencies of every value a variant feature lists, -rather than those of the value that is actually in use. +properties that the target system supports. A wheel lists every value it +runs on, so without that step a dependency gated on one of them would be +installed wherever the wheel runs. A wheel built for GPU architectures +from ``80_real`` to ``120_real`` may depend on a library that supports +only ``120_real``, much as a dependency that supports a single CPU +architecture is gated on ``platform_machine``. Where a dependency does +exist for every value of a feature, it can instead be published as a +variant package and depended upon unconditionally, leaving the choice to +`variant ordering`_. Because of this filtering, the variant markers do not depart from the established meaning of an environment marker. Every property in From 2d5bcf293119d55b3116128d0a034f80d340118b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 10 Aug 2026 08:04:18 +0200 Subject: [PATCH 26/28] PEP 825: update the date in change history MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index 3f714a5b398..a5587a8eed7 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -1542,7 +1542,7 @@ and Zanie Blue. Change History ============== -- 08-Aug-2026 +- 10-Aug-2026 - Decoupled most of the specification from `index-level metadata`_, clarifying that it is only an optimization for scenarios where From 2c8b3287cc6992968f106efd481681398c3b65a6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 13 Aug 2026 13:13:08 +0200 Subject: [PATCH 27/28] Apply suggestions from code review Co-authored-by: Carol Willing --- peps/pep-0825.rst | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index a5587a8eed7..a831c4a84ce 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -26,7 +26,9 @@ This PEP provides the data format for variant wheels, an extension to :doc:`packaging:specifications/binary-distribution-format` that permits building multiple variants of the same package while embedding additional compatibility data. This data is stored inside the wheel, and -expressed via a human-readable variant label in the filename. When +expressed via a human-readable variant label in the filename. + +When wheels are hosted on an index, it is additionally exposed in a separate JSON file as an optimization. It will be followed by additional PEPs defining the remaining aspects of variant wheels. The final aim of the @@ -83,9 +85,9 @@ Tools that produce the data formats according to this specification (variant wheels, the index-level metadata file) MUST always produce files that meet the requirements of this specification. -Tools that consume the data formats according to this specification are -not required to verify that the data meets these requirements, but -SHOULD NOT rely on data that they have established does not meet them. +Tools that consume this specification's data formats are +not required to verify that the data meets these requirements. A tool +SHOULD NOT rely on data that the tool has established does not meet the specification format. The appropriate response depends on the role of the tool, and follows the same pattern as for invalid Core Metadata in wheel files. Tools that are in a position to reject invalid data at the point it enters the @@ -523,7 +525,7 @@ example by specifying a preference for specific namespaces, features or properties. The tools MAY also provide options to exclude specific variants, or to select a particular variant. These options operate on the wheels that were found compatible, so they MAY reorder or narrow -that set, but MUST NOT cause a wheel to be selected whose properties +that set, but MUST NOT cause selection of a wheel with properties the target system does not support. Installing a variant wheel for a system other than the one being installed to is instead a matter of overriding which properties are considered supported, which is `out of @@ -731,7 +733,7 @@ operators permitted with each of them: # equivalent dep7; "foo::bar::baz" in variant_properties -The filtering in step 3. is only observable where a variant feature +The filtering in step 3 is only observable where a variant feature lists multiple values, of which the target system needs to support just one (see `variant properties`_). Consider a wheel whose properties include both of: @@ -1059,9 +1061,9 @@ Installing wheels from multiple sources (non-normative) As of the time of writing, there are no accepted standards addressing the support for installing packages from multiple sources, and the -existing tools (such as ``pip`` and ``uv``) disagree on the exact +existing tools differ on the exact behavior. This problem is described in more detail in the informational -:pep:`766`. Variant wheels expand the problem scope. The consistency +:pep:`766`. Variant wheels extend these differences. The consistency requirements that make variant metadata combinable hold within a single project, and nothing obliges independent publishers to meet them with respect to one another: the same variant label may map to different @@ -1071,7 +1073,7 @@ sources happens to be combinable is possible, but the cost of doing so in the general case is prohibitive, and there is no correct answer when it is not. For these reasons, the specification does not attempt to standardize a behavior, but instead considers it implementation-defined -and provides a few non-normative suggestions on the possible solutions. +and provides a few non-normative, suggested solutions, including using non-variant wheel. It is entirely valid for tools not to support this behavior, either by not providing support for using multiple sources at all, or by rejecting From c7f7b0ff0ff2af55cc16998d56624457ee9f3a28 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 13 Aug 2026 13:30:55 +0200 Subject: [PATCH 28/28] Follow the remaining suggestions and reflow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Michał Górny --- peps/pep-0825.rst | 154 +++++++++++++++++++++++----------------------- 1 file changed, 78 insertions(+), 76 deletions(-) diff --git a/peps/pep-0825.rst b/peps/pep-0825.rst index a831c4a84ce..3de55594b80 100644 --- a/peps/pep-0825.rst +++ b/peps/pep-0825.rst @@ -28,11 +28,10 @@ building multiple variants of the same package while embedding additional compatibility data. This data is stored inside the wheel, and expressed via a human-readable variant label in the filename. -When -wheels are hosted on an index, it is additionally exposed in a separate -JSON file as an optimization. It will be followed by additional PEPs -defining the remaining aspects of variant wheels. The final aim of the -specification is to make ``{tool} install {package}`` capable of +When wheels are hosted on an index, it is additionally exposed in a +separate JSON file as an optimization. It will be followed by additional +PEPs defining the remaining aspects of variant wheels. The final aim of +the specification is to make ``{tool} install {package}`` capable of selecting the most appropriate variant of packages where additional compatibility dimensions such as GPU support need to be accounted for. @@ -76,26 +75,27 @@ document are to be interpreted as described in :rfc:`2119`. Implementation requirements --------------------------- -This specification is written from the perspective of file format producers. +This specification is written from the perspective of tools producing +the various file formats defined or changed in this specification, such +as variant wheels, variant metadata files or ``pylock.toml`` files. In the context of format definitions, the word "MUST" specifically indicates that the requirement must be satisfied for the data to be considered valid according to this specification. -Tools that produce the data formats according to this specification -(variant wheels, the index-level metadata file) MUST always produce -files that meet the requirements of this specification. +Tools that produce the data formats according to this specification MUST +always produce files that meet the requirements of this specification. -Tools that consume this specification's data formats are -not required to verify that the data meets these requirements. A tool -SHOULD NOT rely on data that the tool has established does not meet the specification format. -The appropriate response depends on the role of the tool, and follows -the same pattern as for invalid Core Metadata in wheel files. Tools that -are in a position to reject invalid data at the point it enters the -ecosystem, such as a package index accepting an upload, SHOULD do so. -Tools that encounter it later, such as an installer resolving a -dependency, SHOULD prefer to degrade gracefully rather than fail -outright, for example by ignoring the variant wheels and selecting among -the remaining ones. +Tools that consume this specification's data formats are not required to +verify that the data meets these requirements. A tool SHOULD NOT rely on +data that the tool has established does not meet the specification +format. The appropriate response depends on the role of the tool, and +follows the same pattern as for invalid Core Metadata in wheel files. +Tools that are in a position to reject invalid data at the point it +enters the ecosystem, such as a package index accepting an upload, +SHOULD do so. Tools that encounter it later, such as an installer +resolving a dependency, SHOULD prefer to degrade gracefully rather than +fail outright, for example by ignoring the variant wheels and selecting +among the remaining ones. Variant wheel @@ -268,8 +268,8 @@ Default priorities are defined at the project scope. The value in different wheels SHOULD be identical, or one an extension of the other: additional namespaces MAY be appended. The metadata is considered consistent if the longer list starts with the elements of the shorter -list, in the same order. In this case, combining the metadata MUST result -in the longer list being used. +list, in the same order. In this case, combining the metadata MUST +result in the longer list being used. Variants @@ -331,10 +331,10 @@ Index-level metadata When a package version that includes at least one variant wheel is hosted on an index, a corresponding ``{name}-{version}-variants.json`` file MUST be hosted as well. The purpose of the file is to optimize -variant metadata lookups and remove the necessity of fetching multiple -variant wheels during dependency resolution. The ``{name}`` and -``{version}`` placeholders correspond to the package name and version, -normalized according to the same rules as wheel files, as found in the +variant metadata lookups by removing the need to fetch multiple variant +wheels during dependency resolution. The ``{name}`` and ``{version}`` +placeholders correspond to the package name and version, normalized +according to the same rules as wheel files, as found in the :ref:`packaging:wheel-file-name-spec` of the Binary Distribution Format specification. @@ -420,7 +420,8 @@ defining the respective keys, the requirements are: same order. Combining them MUST yield the longer list. - ``variants``: the same variant label MUST always map to the same set - of properties. Combining them MUST yield the union of the dictionaries. + of properties. Combining them MUST yield the union of the + dictionaries. Both rules are symmetric, so combining metadata that satisfies them gives the same result regardless of the order in which the inputs are @@ -438,10 +439,10 @@ they are not met, the response described in `implementation requirements`_ applies. Where a user draws wheels for the same package from more than one -source, no publisher is in a position to guarantee consistency with the -others. Ensuring that the sources being combined are consistent is then -the responsibility of the user. Tools are not required to detect or -resolve inconsistencies between sources; `installing wheels from +published source, no publisher is in a position to guarantee consistency +with the others. Ensuring that the sources being combined are consistent +is then the responsibility of the user. Tools are not required to detect +or resolve inconsistencies between sources; `installing wheels from multiple sources (non-normative)`_ discusses what they can reasonably do instead. @@ -480,8 +481,8 @@ algorithm: 2. For every namespace, take the ordered list of compatible feature names obtained previously. This is ``feature_order`` in the example. -3. For every feature, take the ordered list of compatible values obtained - previously. This is ``value_order`` in the example. +3. For every feature, take the ordered list of compatible values + obtained previously. This is ``value_order`` in the example. 4. For every group, determine the most preferred value corresponding to every variant feature present in the variant properties corresponding @@ -525,11 +526,11 @@ example by specifying a preference for specific namespaces, features or properties. The tools MAY also provide options to exclude specific variants, or to select a particular variant. These options operate on the wheels that were found compatible, so they MAY reorder or narrow -that set, but MUST NOT cause selection of a wheel with properties -the target system does not support. Installing a variant wheel for a -system other than the one being installed to is instead a matter of -overriding which properties are considered supported, which is `out of -scope`_ for this PEP. +that set, but MUST NOT cause selection of a wheel with properties the +target system does not support. Installing a variant wheel for a system +other than the one being installed to is instead a matter of overriding +which properties are considered supported, which is `out of scope`_ for +this PEP. Alternatively, the sort algorithm for variant wheels could be described using the following pseudocode. For simplicity, this code does not @@ -581,7 +582,7 @@ compatibility tags. def property_key(prop: tuple[str, str, str]) -> tuple[int, int, int]: - """Construct a sort key for variant property (akin to step 5.)""" + """Construct a sort key for variant property (akin to step 5)""" namespace, feature_name, feature_value = prop return ( namespace_order.index(namespace), @@ -598,7 +599,7 @@ compatibility tags. properties: dict[str, dict[str, list[str]]] def best_value_properties(self: Self) -> list[tuple[str, str, str]]: - """Determine the most preferred values for every feature, step 4.""" + """Determine the most preferred values for every feature, step 4""" return [ ( namespace, @@ -610,11 +611,11 @@ compatibility tags. ] def sorted_properties(self: Self) -> list[tuple[str, str, str]]: - """Sort the list of features with their best values (step 6.)""" + """Sort the list of features with their best values (step 6)""" return sorted(self.best_value_properties(), key=property_key) def __lt__(self: Self, other: Self) -> bool: - """Variant comparison function for sorting (part of step 7.)""" + """Variant comparison function for sorting (part of step 7)""" self_properties = self.sorted_properties() other_properties = other.sorted_properties() # Proceed from the first to the last common sort best-value property. @@ -676,8 +677,8 @@ separators when matching features and properties. Evaluating variant markers '''''''''''''''''''''''''' -The variant markers MUST only be used in dependency specifiers, and they -MUST NOT take part in selecting a wheel: they gate the individual +The variant markers MUST only be used in dependency specifiers and MUST +NOT take part in selecting a wheel. These markers gate the individual dependency specifiers of a wheel that has already been selected. They MUST be evaluated only once variant wheel selection, as described in `variant ordering`_, has taken place. @@ -694,12 +695,12 @@ Their values MUST be determined as follows: or the `index-level metadata`_ MAY be used for this, as the consistency requirements guarantee that the two agree. -3. That set is filtered to the variant properties that the target system - supports, as already determined during variant wheel selection. - ``variant_properties`` is the result. Since a wheel can only be - selected if the target system supports it, and that requires at least - one supported value for every feature the wheel declares, this step - never removes a feature entirely. +3. The set of expanded triples is filtered to the variant properties + that the target system supports, as already determined during variant + wheel selection. ``variant_properties`` is the result. Since a wheel + can only be selected if the target system supports it, and that + requires at least one supported value for every feature the wheel + declares, this step never removes a feature entirely. 4. ``variant_features`` and ``variant_namespaces`` are derived from ``variant_properties``. @@ -733,10 +734,10 @@ operators permitted with each of them: # equivalent dep7; "foo::bar::baz" in variant_properties -The filtering in step 3 is only observable where a variant feature -lists multiple values, of which the target system needs to support just -one (see `variant properties`_). Consider a wheel whose properties -include both of: +The filtering in step 3 is only observable where a variant feature lists +multiple values, of which the target system needs to support just one +(see `variant properties`_). Consider a wheel whose properties include +both of: .. code:: text @@ -799,8 +800,8 @@ follows: - The structure of this table MUST conform to the JSON Schema. - Tools that support variant-aware resolution MUST validate this table against the referenced schema. - - Tools that do not support variant-aware resolution MAY ignore this table - but SHOULD preserve it when rewriting the lock file. + - Tools that do not support variant-aware resolution MAY ignore this + table but SHOULD preserve it when rewriting the lock file. Example @@ -877,9 +878,9 @@ behavior would be to: selected wheel and those of its variant properties that the target system supports, as described in `evaluating variant markers`_. -Note that steps 4. through 8. are introduced specifically for variant +Note that steps 4 through 8 are introduced specifically for variant wheels. The remaining steps correspond to the current installer -behavior. Step 10. is modified through the presence of new environment +behavior. Step 10 is modified through the presence of new environment markers. The same algorithm applies to sources other than an index, such as a @@ -1061,19 +1062,19 @@ Installing wheels from multiple sources (non-normative) As of the time of writing, there are no accepted standards addressing the support for installing packages from multiple sources, and the -existing tools differ on the exact -behavior. This problem is described in more detail in the informational -:pep:`766`. Variant wheels extend these differences. The consistency -requirements that make variant metadata combinable hold within a single -project, and nothing obliges independent publishers to meet them with -respect to one another: the same variant label may map to different -properties, and namespace orderings chosen independently need not be -extensions of one another. Establishing whether the metadata from two -sources happens to be combinable is possible, but the cost of doing so -in the general case is prohibitive, and there is no correct answer when -it is not. For these reasons, the specification does not attempt to -standardize a behavior, but instead considers it implementation-defined -and provides a few non-normative, suggested solutions, including using non-variant wheel. +existing tools differ on the exact behavior. This problem is described +in more detail in the informational :pep:`766`. Variant wheels extend +these differences. The consistency requirements that make variant +metadata combinable hold within a single source tree, and nothing +obliges independent publishers to meet them with respect to one another: +the same variant label may map to different properties, and namespace +orderings chosen independently need not be extensions of one another. +Establishing whether the metadata from two sources happens to be +combinable is possible, but the cost of doing so in the general case is +prohibitive, and there is no correct answer when it is not. For these +reasons, the specification does not attempt to standardize a behavior, +but instead considers it implementation-defined and provides a few +non-normative, suggested solutions, including using non-variant wheel. It is entirely valid for tools not to support this behavior, either by not providing support for using multiple sources at all, or by rejecting @@ -1518,11 +1519,12 @@ These questions must be resolved before this PEP can be accepted. Use of variant environment markers ---------------------------------- -The design of the variant `environment markers`_ is not yet settled. The same -effect can be achieved through Dynamic dependencies; the open question is -whether markers are the right mechanism for obtaining it while keeping -dependency metadata static across a release. This is under discussion in -`this thread `__. +The design of the variant `environment markers`_ is not yet settled. The +same effect can be achieved through Dynamic dependencies; the open +question is whether markers are the right mechanism for obtaining it +while keeping dependency metadata static across a release. This is under +discussion in `this thread +`__. The specification reflects the current design.