| |
@@ -1,5 +1,5 @@
|
| |
= Python Packaging Guidelines
|
| |
- :last-reviewed: 2019-05-20
|
| |
+ :last-reviewed: 2020-01-01
|
| |
:toc:
|
| |
|
| |
== Python Version Support
|
| |
@@ -11,9 +11,7 @@
|
| |
Upstream support for the python2 interpreter officially ends in 2020.
|
| |
If a piece of software supports python3,
|
| |
it MUST be packaged for python3.
|
| |
- Software using python2 MUST NOT be newly packaged into Fedora 30 or newer without FESCo exception.
|
| |
- Packaging new python2 based packages into older Fedoras is discouraged,
|
| |
- but doesn't need an exception.
|
| |
+ Software using python2 MUST NOT be newly packaged into Fedora without FESCo exception.
|
| |
|
| |
For guidelines on maintaining already existing python2 packages, see the xref:Python_Appendix.adoc[appendix].
|
| |
|
| |
@@ -22,17 +20,17 @@
|
| |
Since Fedora 31 `/usr/bin/python` is, if it is installed, a symbolic link to `/usr/bin/python3`.
|
| |
It was a symbolic link to `/usr/bin/python2` on previous releases.
|
| |
|
| |
- Packages in Fedora MUST NOT use `/usr/bin/python`. Instead packages for Python 3 MUST use `/usr/bin/python3` (even if upstream supports both Python 2 and 3). As a result of that `/usr/bin/python` (as well as `/usr/bin/env python` and similar) MUST NOT be used in shebang lines or as a dependency of a package. As of Fedora 30, all uses of unversioned python executables in shebang lines will fail the build.
|
| |
+ Packages in Fedora MUST NOT use `/usr/bin/python`. Instead packages for Python 3 MUST use `/usr/bin/python3` (even if upstream supports both Python 2 and 3). As a result of that `/usr/bin/python` (as well as `/usr/bin/env python` and similar) MUST NOT be used in shebang lines or as a dependency of a package. All uses of unversioned python executables in shebang lines will fail the build.
|
| |
These shebangs MUST be fixed. If it is necessary to disable the checks, please see the information in xref:index#_shebang_lines[Shebang lines].
|
| |
|
| |
- All python runtimes have a virtual provide for `+python(abi) = $MAJOR-$MINOR+`. For example, the Python 3.7 runtime package has:
|
| |
+ All Python runtimes have a virtual provide for `+python(abi) = $MAJOR.$MINOR+`. For example, the Python 3.7 runtime package has:
|
| |
|
| |
....
|
| |
$ rpm -q --provides python3 | grep abi
|
| |
python(abi) = 3.7
|
| |
....
|
| |
|
| |
- Python modules using these runtimes should have a corresponding "Requires" line on the python runtime that they are used with. This is done automatically for files below `+/usr/lib[^/]*/python${PYVER}+`
|
| |
+ Python modules using these runtimes should have a corresponding "Requires" line on the Python runtime that they are used with. This is done automatically for files below `+/usr/lib[^/]*/python${PYVER}+`
|
| |
|
| |
Mirroring the policy for regular packages, the Python-version-specific subpackages of your package MUST NOT be removed in a release branch of Fedora.
|
| |
|
| |
@@ -52,15 +50,6 @@
|
| |
|
| |
Packages MAY use the automatic Python dependency generator. This generator uses upstream egg/dist metadata (such as https://python-packaging.readthedocs.io/en/latest/dependencies.html[setuptool's install_requires]) to determine what the package should depend on. The generator parses the installed metadata from `+/usr/lib(64)?/pythonX.Y/site-packages/[^/]+++.(egg|dist)-info/requires.txt+`, so it will not work with software that uses plain https://docs.python.org/3/distutils/[distutils].
|
| |
|
| |
- To enable this feature (F28 and F29), add:
|
| |
-
|
| |
- %{?python_enable_dependency_generator}
|
| |
-
|
| |
- To disable this feature (F30 and newer), add:
|
| |
-
|
| |
- %{?python_disable_dependency_generator}
|
| |
-
|
| |
- Although this statement can be used anywhere in the spec, we recommend putting it just before the main package's `+%description+` declaration.
|
| |
This generates run time requires in the form of `+pythonX.Ydist(foo)+`. If the generated dependencies are not accurate, additional ones can still be added manually. To remove some, a packager MAY modify upstream-provided metadata (usually specified in the `setup.py` file) in the `+%prep+` section of the specfile or fall back to xref:AutoProvidesAndRequiresFiltering.adoc[filtering] those dependencies.
|
| |
|
| |
The packager MUST inspect the generated requires for correctness. All dependencies MUST be resolvable within the targeted Fedora version.
|
| |
@@ -107,10 +96,12 @@
|
| |
Note that since Fedora 31, `+.0+` suffixes are removed from version numbers to match the behavior of Python tools.
|
| |
(https://www.python.org/dev/peps/pep-0440/#final-releases[PEP 440] specifies that `+X.Y+` and `+X.Y.0+` are treated as equal.)
|
| |
|
| |
- This generator is enabled by default in Fedora 30 and later. If a packager wishes to explicitly opt out of the generator because the upstream metadata are not applicable, a packager SHOULD opt out explicitly by adding:
|
| |
+ This generator is enabled by default in Fedora. If a packager wishes to explicitly opt out of the generator because the upstream metadata are not applicable, a packager SHOULD opt out explicitly by adding:
|
| |
|
| |
`+%{?python_disable_dependency_generator}+`
|
| |
|
| |
+ Although this statement can be used anywhere in the spec, we recommend putting it just before the main package's `+%description+` declaration.
|
| |
+
|
| |
== Provides
|
| |
|
| |
Using a fictional module named "example", the subpackage containing the Python 3 version *must* provide `python3-example`. This is of course always the case if the subpackage is named `python3-example` (as in the examples below). If the subpackage has some other name then then `Provides: python3-example` must be added explicitly (but see the `+%python_provide+` macro below).
|
| |
@@ -178,15 +169,20 @@
|
| |
|
| |
|`+%{__python}+`
|
| |
|`+/usr/bin/python+` (for backwards compatibility)
|
| |
- |Don't use this macro without redefining it. Defining is changes the meaning of other "unversioned" Python macros such as `+%{python_sitelib}+`.
|
| |
+ |Don't use this macro without redefining it. Defining it changes the meaning of other "unversioned" Python macros such as `+%{python}+` or `+%{python_sitelib}+`.
|
| |
+
|
| |
|
| |
|`+%{__python3}+`
|
| |
|`+/usr/bin/python3+`
|
| |
- |Python 3 interpreter.
|
| |
+ |Python 3 interpreter. Redefining this macro changes all the `+%{python3...}+` macros.
|
| |
+
|
| |
+ |`+%{python3}+`
|
| |
+ |`+%{__python3}+`
|
| |
+ |Python 3 interpreter. Use this macro in spec files.
|
| |
|
| |
|`+%{python_provide}+`
|
| |
|(Lua script)
|
| |
- |Given a package name, evaluates to either `Provides: python-$name` or nothing at all depending on the Pythonversion. See <<The %python_provide macro>> for an example.
|
| |
+ |Given a package name, evaluates to either `Provides: python-$name` or nothing at all depending on the Python version. See <<The %python_provide macro>> for an example.
|
| |
|
| |
|`+%{python3_sitelib}+`
|
| |
|`+/usr/lib/python3.X/site-packages+`
|
| |
@@ -206,7 +202,11 @@
|
| |
|
| |
|`+%{python3_version_nodots}+`
|
| |
|`+3X+`
|
| |
- |Python 3 version without dots. Useful when listing files explicitly in %files section, such as `+%{python3_sitelib}/foo/*.cpython-%{python3_version_nodots}.pyo+`
|
| |
+ |Python 3 version without dots. Useful when listing files explicitly in %files section, such as `+%{python3_sitearch}/foo/_speedups.cpython-%{python3_version_nodots}*.so+`
|
| |
+
|
| |
+ |`+%{python3_platform}+`
|
| |
+ |`+linux-x86_64+` on x86_64
|
| |
+ |The platform name used in Python, useful for specifying `+$PYTHONPATH+`, such as `+PYTHONPATH=build/lib.%{python3_platform}-%{python3_version}+`
|
| |
|
| |
|`+%py3_build+`
|
| |
|`+%{__python3} setup.py build …+`
|
| |
@@ -228,9 +228,13 @@
|
| |
|(Lua script)
|
| |
|Evaluates to the appropriate URL for the package. See above for more information.
|
| |
|
| |
+ |`+%pycached ….py+`
|
| |
+ |(Lua script)
|
| |
+ |Given a Python file, lists the file and the files with its bytecode cache. See <<Byte compiling>> for more information.
|
| |
+
|
| |
|===
|
| |
|
| |
- During `+%install+` or when listing `+%files+` you can use the `+python3_sitearch+` and `+python3_sitelib+` macros to specify where the installed modules are to be found. For instance:
|
| |
+ During `+%install+` or when listing `+%files+` you can use the `+%{python3_sitearch}+` and `+%{python3_sitelib}+` macros to specify where the installed modules are to be found. For instance:
|
| |
|
| |
....
|
| |
%files
|
| |
@@ -243,7 +247,7 @@
|
| |
Use of the macros has several benefits:
|
| |
|
| |
* It ensures that the packages are installed correctly on multilib architectures.
|
| |
- * Using these macros instead of hardcoding the directory in the specfile ensures your spec remains compatible with the installed python version even if the directory structure changes radically (for instance, if `+python3_sitelib+` moves into `+%{_datadir}+`).
|
| |
+ * Using these macros instead of hardcoding the directory in the specfile ensures your spec remains compatible with the installed Python version even if the directory structure changes radically (for instance, if `+python3_sitelib+` moves into `+%{_datadir}+`).
|
| |
|
| |
== Packages using Cython
|
| |
|
| |
@@ -261,13 +265,13 @@
|
| |
|
| |
== Files to include
|
| |
|
| |
- When packaging python modules, several types of files are included:
|
| |
+ When packaging Python modules, several types of files are included:
|
| |
|
| |
- * *.py source files because they are used when generating tracebacks.
|
| |
- * *.pyc byte compiled files.
|
| |
+ * +*.py+ source files because they are used when generating tracebacks.
|
| |
+ * +*.pyc+ byte compiled files.
|
| |
** Python will try to create them at runtime if they don't exist which leads to spurious SELinux AVC denials in the logs.
|
| |
- ** If the system administrator invokes python with -OO, they will be created with no docstrings. This can break some programs.
|
| |
- * *.egg-info files or directories. If these are generated by the module's build scripts they must be included in the package because they might be needed by other applications and modules at runtime.
|
| |
+ ** If the system administrator invokes Python with -OO, they will be created with no docstrings. This can break some programs.
|
| |
+ * +*.egg-info+ or +*.dist-info+ files or directories. If these are generated by the module's build scripts they must be included in the package because they might be needed by other applications and modules at runtime.
|
| |
|
| |
The source files MUST be included in the same package as the byte compiled versions.
|
| |
|
| |
@@ -282,9 +286,9 @@
|
| |
|
| |
== Byte compiling
|
| |
|
| |
- Python will automatically try to byte compile files when it runs in order to speed up startup the next time it is run. These files are saved in files with the extension of .pyc (compiled python). These files will be located inside a directory named `+__pycache__+`.
|
| |
+ Python will automatically try to byte compile files when it runs in order to speed up startup the next time it is run. These files are saved in files with the extension of .pyc (compiled Python). These files will be located inside a directory named `+__pycache__+`.
|
| |
|
| |
- The .pyc files contain byte code that is portable across OSes. If you do not include them in your packages, python will try (and generally fail) to create them when the user runs the program. If the system administrator runs the program, then the files will be successfully written, causing stray .pyc files which will not be removed when the package is removed. To prevent that the byte compiled files need to be compiled and included in the `+%files+` section. Normally, byte compilation is done for you by the `+brp-python-bytecompile+` script. This script runs after the `+%install+` section of the spec file has been processed and byte compiles any .py files that it finds in `+%{python3_sitelib}+` or `+%{python3_sitearch}+` (this recompilation puts the proper filesystem paths into the modules otherwise tracebacks would include the `+%{buildroot}+` in them).
|
| |
+ The .pyc files contain byte code that is portable across OSes. If you do not include them in your packages, Python will try (and generally fail) to create them when the user runs the program. If the system administrator runs the program, then the files will be successfully written, causing stray .pyc files which will not be removed when the package is removed. To prevent that the byte compiled files need to be compiled and included in the `+%files+` section. Normally, byte compilation is done for you by the `+brp-python-bytecompile+` script. This script runs after the `+%install+` section of the spec file has been processed and byte compiles any .py files that it finds in `+%{python3_sitelib}+` or `+%{python3_sitearch}+` (this recompilation puts the proper filesystem paths into the modules otherwise tracebacks would include the `+%{buildroot}+` in them).
|
| |
|
| |
You must include the .pyc files in your package. If the build process creates a `+__pycache__+` directory in a subdirectory of `+%{python3_sitearch}+` or `+%{python3_sitelib}+`, you must also include all items in the `+__pycache__+` directory. You MUST NOT include the directories `+%{python3_sitearch}/__pycache__+` or `+%{python3_sitelib}/__pycache__+` because they are already owned by the python3-libs package.
|
| |
|
| |
@@ -295,14 +299,26 @@
|
| |
%{python3_sitelib}/foo/
|
| |
....
|
| |
|
| |
- or, if the python code installs directly into %\{python3_sitelib}:
|
| |
+ or, if the Python code installs directly into `+%{python3_sitelib}+`, use the `+%pycached+` macro to include the bytecode cache files:
|
| |
+
|
| |
+ ....
|
| |
+ %files
|
| |
+ %pycached %{python3_sitelib}/foo.py
|
| |
+ ....
|
| |
+
|
| |
+ That evaluates roughly to
|
| |
|
| |
....
|
| |
%files
|
| |
%{python3_sitelib}/foo.py
|
| |
- %{python3_sitelib}/__pycache__/*
|
| |
+ %{python3_sitelib}/__pycache__/foo.cpython-%{python3_version_nodots}{,.opt-?}.pyc
|
| |
....
|
| |
|
| |
+ NOTE: The `+%pycached+` macro only supports Python 3.5+,
|
| |
+ so for older Python versions
|
| |
+ (such as 3.4 in EPEL 6 or 7),
|
| |
+ you need to list the files manually.
|
| |
+
|
| |
=== Manual byte compilation
|
| |
|
| |
For more details on the internals of byte compilation, please see xref:Python_Appendix.adoc#manual-bytecompilation[the appendix].
|
| |
@@ -329,5 +345,5 @@
|
| |
* *Must*: Python modules must not download any dependencies during the build process.
|
| |
* *Must*: When building a compat package, it must install using easy_install -m so it won't conflict with the main package.
|
| |
* *Must*: When building multiple versions (for a compat package) one of the packages must contain a default version that is usable via "import MODULE" with no prior setup.
|
| |
- * *Should*: If you build a python module you should use the `+%python_provide+` macro.
|
| |
+ * *Should*: If you build a Python module you should use the `+%python_provide+` macro.
|
| |
* *Should*: A package which is used by another package via an egg interface should provide egg info.
|
| |
pviktori commented 3 years ago
without a FESCo exception