| |
@@ -1,5 +1,5 @@
|
| |
= Python Packaging Guidelines
|
| |
- :last-reviewed: 2020-01-01
|
| |
+ :last-reviewed: 2020-08-05
|
| |
:toc:
|
| |
|
| |
== Python Version Support
|
| |
@@ -17,11 +17,12 @@
|
| |
|
| |
== Multiple Python Runtimes
|
| |
|
| |
- Since Fedora 31 `/usr/bin/python` is, if it is installed, a symbolic link to `/usr/bin/python3`.
|
| |
+ On Fedora `/usr/bin/python` is, if it is installed, a symbolic link to `/usr/bin/python3`.
|
| |
It was a symbolic link to `/usr/bin/python2` on previous releases.
|
| |
|
| |
Packages in Fedora MUST NOT use `/usr/bin/python`. Instead packages for Python 3 MUST use `/usr/bin/python3` (even if upstream supports both Python 2 and 3). As a result of that `/usr/bin/python` (as well as `/usr/bin/env python` and similar) MUST NOT be used in shebang lines or as a dependency of a package. All uses of unversioned python executables in shebang lines will fail the build.
|
| |
- These shebangs MUST be fixed. If it is necessary to disable the checks, please see the information in xref:index#_shebang_lines[Shebang lines].
|
| |
+ These shebangs MUST be fixed (for example by using the `+%py3_shebang_fix+` macro in the spec file).
|
| |
+ If it is necessary to disable the checks, please see the information in xref:index#_shebang_lines[Shebang lines].
|
| |
|
| |
All Python runtimes have a virtual provide for `+python(abi) = $MAJOR.$MINOR+`. For example, the Python 3.7 runtime package has:
|
| |
|
| |
@@ -40,6 +41,13 @@
|
| |
|
| |
This rule does not apply to applications.
|
| |
|
| |
+ The character `+` in names of built packages (i.e. non-SRPM)
|
| |
+ that include `.dist-info` or `.egg-info` directories is reserved for <<Python Extras>> and MUST NOT be used for any other purpose.
|
| |
+ The `+` character triggers the automatic dependency generator for extras.
|
| |
+ Replace any `+` signs in the upstream name with `-`,
|
| |
+ or omit them when at the beginning of the name.
|
| |
+ As an exception, `+` characters are permitted at the _end_ of the name.
|
| |
+
|
| |
== Dependencies
|
| |
|
| |
Packages building for Python 3 will need `BuildRequires: python3-devel`.
|
| |
@@ -95,7 +103,7 @@
|
| |
python3.7dist(traitlets) >= 4.2.1
|
| |
....
|
| |
|
| |
- Note that since Fedora 31, `+.0+` suffixes are removed from version numbers to match the behavior of Python tools.
|
| |
+ Note that any `+.0+` suffixes are removed from version numbers to match the behavior of Python tools.
|
| |
(https://www.python.org/dev/peps/pep-0440/#final-releases[PEP 440] specifies that `+X.Y+` and `+X.Y.0+` are treated as equal.)
|
| |
|
| |
This generator is enabled by default in Fedora. If a packager wishes to explicitly opt out of the generator because the upstream metadata are not applicable, a packager SHOULD opt out explicitly by adding:
|
| |
@@ -104,36 +112,160 @@
|
| |
|
| |
Although this statement can be used anywhere in the spec, we recommend putting it just before the main package's `+%description+` declaration.
|
| |
|
| |
+ === Python Extras
|
| |
+
|
| |
+ https://www.python.org/dev/peps/pep-0508/#extras[Python extras] are a way for Python projects to declare that extra dependencies are required for additional functionality.
|
| |
+
|
| |
+ For example, `requests` has several standard dependencies (e.g. `urllib3`).
|
| |
+ But it also declares an _extra_ named `+requests[security]+`,
|
| |
+ which lists additional dependencies (e.g. `cryptography`).
|
| |
+ Unlike RPM subpackages,
|
| |
+ extras can only specify additional dependencies,
|
| |
+ not additional files.
|
| |
+ The main package will work if the optional dependency is not installed,
|
| |
+ but it might have limited functionality.
|
| |
+
|
| |
+ Python tools treat extras as virtual packages.
|
| |
+ For example,
|
| |
+ if a user runs `+pip install requests[security]+`,
|
| |
+ or installs a project that depends on `+requests[security]+`,
|
| |
+ both `requests` and `cryptography` will be installed.
|
| |
+
|
| |
+ Starting with Fedora 33,
|
| |
+ extras are usually provided by packages with no files.
|
| |
+ Instead of square brackets,
|
| |
+ Fedora package names conventionally use the `+` character to separate the package name and the _extra_ name,
|
| |
+ e.g. the package would be named `python3-requests+security`.
|
| |
+ The plus sign is valid in RPM package names,
|
| |
+ but not in Python canonical project names nor in extras identifiers.
|
| |
+
|
| |
+ Python packages SHOULD have `Provides` for all extras the upstream project specifies,
|
| |
+ except those that are not useful for other packages
|
| |
+ (for example build/development requirements,
|
| |
+ commonly named `dev`, `doc` or `test`).
|
| |
+
|
| |
+ A package that provides a Python extra MUST provide `+python3dist(…[…])+` and `+python3.Xdist(…[…])+`,
|
| |
+ for example, `+python3.9dist(requests[security])+`.
|
| |
+ These requirements SHOULD be generated using the automatic dependency generator.
|
| |
+
|
| |
+ A package that provides a Python extra MUST require the extra's main package with exact NEVR.
|
| |
+
|
| |
+ A subpackage that primarily provides one Python extra SHOULD be named by appending `+` and the extra name to the main package name.
|
| |
+ For example, `++python3-requests+security++`.
|
| |
+
|
| |
+ The most straightforward way to provide an extra is with a dedicated subpackage containing no files (a "metapackage").
|
| |
+ This case can be automated with the `+%python_extras_subpkg+` macro.
|
| |
+
|
| |
+ Alternative approach:
|
| |
+ when some extra is always useful in a distro,
|
| |
+ it can be provided by the main package;
|
| |
+ when several extras are related,
|
| |
+ they may be provided by a single subpackage.
|
| |
+ However,
|
| |
+ having one dedicated subpackage per extra
|
| |
+ allows you to use the automatic dependency generator to ensure that the extras' requirements will stay in sync with upstream.
|
| |
+ If you create a dedicated subpackage and want it to be always/usually installed, you MAY Require/Recommend/Suggest it from the main package.
|
| |
+
|
| |
+ The dependency generator for extras activates if the following holds:
|
| |
+
|
| |
+ - The package must contain the `.egg-info`/`.dist-info` directory, usually as `+%ghost+`.
|
| |
+ - The package name must end with `+EXTRA` (where `EXTRA` is the extra name).
|
| |
+
|
| |
+ As an example,
|
| |
+ the extra subpackage for `+requests[security]+` can be specified using the `+%python_extras_subpkg+` convenience macro as follows.
|
| |
+ The macro takes the main package name and name(s) of the extra(s) as well as path to the `.egg-info` or `.dist-info` directory:
|
| |
+
|
| |
+ ----
|
| |
+ %{?python_extras_subpkg:%python_extras_subpkg -n python3-requests -i %{python3_sitelib}/*.egg-info security}
|
| |
+ ----
|
| |
+
|
| |
+ For this case,
|
| |
+ the extras dependency generator will read upstream metadata from the `.egg-info` directory.
|
| |
+ If it finds that the `security` extra has a dependency on `cryptography`,
|
| |
+ it will generate `+Requires: python3.Xdist(cryptography)+`,
|
| |
+ `+Provides: python3dist(requests[security])+`
|
| |
+ (and the corresponding `+python3.Xdist+` variant).
|
| |
+
|
| |
+ If you need additional features that the `+%python_extras_subpkg+` macro does not cover,
|
| |
+ you will need to write the subpackage sections manually.
|
| |
+ Such features can be, for example:
|
| |
+
|
| |
+ - Obsoleting/providing other names (e.g. obsoleted extras packages)
|
| |
+ - Manual strong or weak dependencies on other (possibly non-Python) packages
|
| |
+ - Including files excluded from the main package (if such files only make sense with the extra and the base package does not fail without them)
|
| |
+
|
| |
+ As an example of what you need to write in these cases,
|
| |
+ the `+%python_extras_subpkg+` macro invocation above expands to the following:
|
| |
+
|
| |
+ ----
|
| |
+ %package -n python3-requests+security
|
| |
+ Summary: Metapackage for python3-requests: security extras
|
| |
+ Requires: python3-requests = %{?epoch:%{epoch}:}%{version}-%{release}
|
| |
+ %description -n python3-requests+security
|
| |
+ This is a metapackage bringing in security extras requires for python3-requests.
|
| |
+ It contains no code, just makes sure the dependencies are installed.
|
| |
+
|
| |
+ %files -n python3-requests+security
|
| |
+ %ghost %{python3_sitelib}/*.egg-info
|
| |
+ ----
|
| |
+
|
| |
+ Note that the dependency generator does not add a dependency on the main package
|
| |
+ (the `+Requires: python3-setuptools_scm = ...+` above).
|
| |
+ If you are not using the `+%python_extras_subpkg+` macro,
|
| |
+ you need to add it manually.
|
| |
+
|
| |
+ NOTE: The `+%python_extras_subpkg+` can take multiple extras names to generate multiple packages.
|
| |
+ For more options, see the https://fedoraproject.org/wiki/Changes/PythonExtras[change proposal which introduced this].
|
| |
+
|
| |
== Provides
|
| |
|
| |
- Using a fictional module named "example", the subpackage containing the Python 3 version *must* provide `python3-example`. This is of course always the case if the subpackage is named `python3-example` (as in the examples below). If the subpackage has some other name then then `Provides: python3-example` must be added explicitly (but see the `+%python_provide+` macro below).
|
| |
+ For any module `foo` intended to be used in Python 3 with `import foo`,
|
| |
+ the package that includes it *should* provide `python3-foo`.
|
| |
+ This is of course always the case if the subpackage is named `python3-foo`
|
| |
+ (as in the examples below).
|
| |
+ If the subpackage has some other name,
|
| |
+ then `Provides: python3-foo` should be added explicitly
|
| |
+ (via `+%py_provides python3-foo+`, see below).
|
| |
+
|
| |
+ === The %py_provides macro
|
| |
|
| |
- === The %python_provide macro
|
| |
+ All packages that provide `+python3-...+` (for any `+...+`) SHOULD also provide `+python-...+` and `+python3.X-...+`.
|
| |
+ Starting from Fedora 33,
|
| |
+ most of the Python packages named `+python3-...+` will provide such names automatically via the dependency generator in `/usr/lib/rpm/fileattrs/pythonname.attr`.
|
| |
|
| |
- All packages that provide `+python3-...+` (for any `+...+`) SHOULD use the `+%python_provide+` macro with the provide, for example:
|
| |
+ Any manually added virtual provides of `+python3-...+` SHOULD be done via the `+%py_provides+` macro.
|
| |
+
|
| |
+ Instead of:
|
| |
|
| |
----
|
| |
Provides: python3-pkg_resources = %{version}-%{release}
|
| |
- %{?python_provide:%python_provide python3-pkg_resources}
|
| |
----
|
| |
|
| |
- In Fedora 31 and further, it adds a virtual provide of `+python-...+` for Python 3 libraries.
|
| |
- For previous releases, it used to add the virtual provide for Python 2 libraries.
|
| |
- Since Fedora 33, it also adds virtual provide of `+python3X-...+` (for example `+python39-...+`).
|
| |
+ Do:
|
| |
+
|
| |
+ ----
|
| |
+ %py_provides python3-pkg_resources
|
| |
+ ----
|
| |
+
|
| |
+ Optionally,
|
| |
+ supply a custom `epoch-version-release` as a second argument to `+%py_provides+`.
|
| |
|
| |
- On Fedora 33 or newer it is no longer needed to use `+%python_provide+` for package names:
|
| |
+ On releases older than Fedora 33,
|
| |
+ or (for technical limitations) for packages without files,
|
| |
+ it is necessary to use `+%py_provides+` even for package names:
|
| |
|
| |
----
|
| |
%package -n python3-%{srcname}
|
| |
Summary: %{summary}
|
| |
- %{?python_provide:%python_provide python3-%{srcname}}
|
| |
+ %py_provides python3-%{srcname}
|
| |
----
|
| |
|
| |
- Such provides are now added automatically.
|
| |
- (The only exception are meta-packages without files, for technical limitations.)
|
| |
- Packagers SHOULD try to remove explicit `+%python_provide+` calls for package names,
|
| |
- but MAY preserve them if they aim for compatibility with older releases.
|
| |
+ Packagers SHOULD try to remove explicit `+%py_provides+` calls for package names,
|
| |
+ but MAY preserve them if they aim for compatibility with older releases or packages without files.
|
| |
|
| |
+ NOTE: Historically, there was `+%python_provide+` macro with similar but different semantics.
|
| |
+ It still works for compatibility reasons but it is deprecated and SHOULD NOT be used
|
| |
+ and packagers SHOULD replace is with appropriate `+%py_provides+` call.
|
| |
|
| |
== Automatic Provides with a standardized name
|
| |
|
| |
@@ -170,7 +302,7 @@
|
| |
When packaging software which is available from PyPI, you can make use of the `+%pypi_source+` macro. This macro accepts from zero to three arguments and evaluates to an appropriate URL for the source file on PyPI. The arguments are:
|
| |
|
| |
1. The name of the PyPI project. Defaults to `+%srcname+` if defined, or to `+%pypi_name+` if defined, or to `+%name+` (the package name).
|
| |
- 2. The version of the PyPI project. Defaults to `+%version+` (the package version).
|
| |
+ 2. The version of the PyPI project. Defaults to `+%version+` (the package version) with any `~` characters removed (used for alpha/beta/dev versions in RPM version but not in Python package version).
|
| |
3. The file extension to use. Defaults to `tar.gz`.
|
| |
|
| |
In most cases it is not necessary to specify any arguments.
|
| |
@@ -186,7 +318,7 @@
|
| |
|Notes
|
| |
|
| |
|`+%{__python}+`
|
| |
- |`+/usr/bin/python+` (for backwards compatibility)
|
| |
+ |(Error)
|
| |
|Don't use this macro without redefining it. Defining it changes the meaning of other "unversioned" Python macros such as `+%{python}+` or `+%{python_sitelib}+`.
|
| |
|
| |
|
| |
@@ -198,9 +330,9 @@
|
| |
|`+%{__python3}+`
|
| |
|Python 3 interpreter. Use this macro in spec files.
|
| |
|
| |
- |`+%{python_provide}+`
|
| |
+ |`+%py_provides+`
|
| |
|(Lua script)
|
| |
- |See <<The %python_provide macro>> for detailed explanation.
|
| |
+ |See <<The %py_provides macro>> for detailed explanation.
|
| |
|
| |
|`+%{python3_sitelib}+`
|
| |
|`+/usr/lib/python3.X/site-packages+`
|
| |
@@ -234,6 +366,14 @@
|
| |
|`+%{__python3} setup.py install --skip-build …+`
|
| |
|Various flags are passed to `+setup.py install+`, see `/usr/lib/rpm/macros.d/macros.python3` for details and similar macros. To add extra flags/arguments to `+setup.py install+`, separate them with `+--+`, for example: `+%py3_install -- --install-scripts %{_libexecdir}+`. To pass custom command line arguments directly to `+setup.py+`, define `+%py_setup_args+`.
|
| |
|
| |
+ |`+%__pytest+`
|
| |
+ |`+/usr/bin/pytest+`
|
| |
+ |The `pytest` command used in `+%pytest+`. Don't use this macro directly, but feel free to redefine it for usage in `+%pytest+` if desired.
|
| |
+
|
| |
+ |`+%pytest+`
|
| |
+ |`+PATH=… PYTHONPATH=… … %{__pytest}+`
|
| |
+ |Various environment variables are set to ensure the packaged version is tested. Use this macro instead of direct `pytest` calls in `+%check+`. Pass additional argument as if passed to `pytest`, e.g. `+%pytest -m "not network"+` to deselect tests marked as `network`.
|
| |
+
|
| |
|`+%{py_dist_name}+`
|
| |
|(Lua script)
|
| |
|Given a standardized name (i.e. dist name, name on PyPI) of Python software, it will convert it to a canonical format. See <<Automatic Provides with a standardized name>> for more information.
|
| |
@@ -250,6 +390,14 @@
|
| |
|(Lua script)
|
| |
|Given a Python file, lists the file and the files with its bytecode cache. See <<Byte compiling>> for more information.
|
| |
|
| |
+ |`+%{py3_shebang_flags}+`
|
| |
+ |`s`
|
| |
+ |The default set of flags for Python shebangs. Redefine or undefine this to change the set. Used by `+%py3_shebang_fix+`.
|
| |
+
|
| |
+ |`+%py3_shebang_fix …+`
|
| |
+ |(Python script)
|
| |
+ |Given paths for Python files or directories with them, it changes Python shebangs to `+#! %{__python3}+`, preserves any existing flags (if found) and adds flags defined in `+%{py3_shebang_flags}+` (if not already present).
|
| |
+
|
| |
|===
|
| |
|
| |
During `+%install+` or when listing `+%files+` you can use the `+%{python3_sitearch}+` and `+%{python3_sitelib}+` macros to specify where the installed modules are to be found. For instance:
|
| |
@@ -337,6 +485,13 @@
|
| |
(such as 3.4 in EPEL 6 or 7),
|
| |
you need to list the files manually.
|
| |
|
| |
+ NOTE: In case you need to use other macros with the `+%pycached+` macro,
|
| |
+ such as `+%exclude+` or `+%ghost+`,
|
| |
+ pass the other macro as part of the argument to `+%pycached+`.
|
| |
+ For example:
|
| |
+ `+%pycached %exclude /path/to/foo.py+`
|
| |
+ Using the macros in wrong order would only apply `+%exclude+` to the first entry that `+%pycached+` generates.
|
| |
+
|
| |
=== Manual byte compilation
|
| |
|
| |
For more details on the internals of byte compilation, please see xref:Python_Appendix.adoc#manual-bytecompilation[the appendix].
|
| |
@@ -363,5 +518,5 @@
|
| |
* *Must*: Python modules must not download any dependencies during the build process.
|
| |
* *Must*: When building a compat package, it must install using easy_install -m so it won't conflict with the main package.
|
| |
* *Must*: When building multiple versions (for a compat package) one of the packages must contain a default version that is usable via "import MODULE" with no prior setup.
|
| |
- * *Should*: Additional `+python3-...+` provides should be accompanied by a `+%python_provide+` call.
|
| |
+ * *Should*: Additional `+python3-...+` provides should be accomplished via a `+%py_provides+` call.
|
| |
* *Should*: A package which is used by another package via an egg interface should provide egg info.
|
| |
See individual commits.