#552 update python packaging guidelines
Closed: Fixed None Opened 3 years ago by zbyszek.

Current Packaging:Python guidelines are significantly tilted towards use of 2to3 and separate srpms. Most of the time neither is needed, especially for new packages that is being added nowadays. I'd like to propose the following:

  1. Make single-srpm single-dir the default option, and add an example which uses that as the standard way of packaging python modules and/or applications
  2. Update for https://fedoraproject.org/wiki/Changes/Python_3_as_Default and encourage use of python3 for binaries (i.e. change installation order to 2 first, 3 second).
  3. Make the policy symmetrical wrt. to python2 and 3, and add an explicit policy about retiring python2 or python3 subpackages: only in Branched and Rawhide.
  4. Suggest using %global _docdir_fmt %{name} to have less duplicated directories.
  5. When separate source directories must be used, use %{py3dir}.

I prepared a [https://fedoraproject.org/wiki/User:Zbyszek/PythonPackagingDraft draft] implementing this ([https://fedoraproject.org/w/index.php?title=User%3AZbyszek%2FPythonPackagingDraft&diff=418270&oldid=418203 diff]). This could be simplified even further, by removing the whole section about running 2to3 by hand.

%{py3dir} was removed from the guidelines a while ago. I missed that. Updated draft which does not re-introduce %{py3dir}: [https://fedoraproject.org/w/index.php?title=User%3AZbyszek%2FPythonPackagingDraft&diff=418368&oldid=418203 diff v2]

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-07-23/fpc.2015-07-23-16.01.txt):

OK, this was a lot of stuff, and the more I worked on it, the more annoyed I became. Plus the wiki kept eating my edits and I kept getting lost in the overhuge page. So, I've probably gone too far in cleaning up but I'll present what I have and if people yell at me then I'll try something less ambitious.

There are now three pages:

The first page incorporates the following changes:
Notes that these guidelines apply only to F22+ and EPEL7. Points at the 2nd page for old guidelines.
Includes the information about not retiring python-version-specific subpackages in stable releases.
Makes BuildRequires: section much more succinct.
Makes sure python2-foo provide is versioned, mentions %python_provide macro.
Makes the macro table collapsible and collapsed by default.
Remove EL6-specific cruft (it's in the old guidelines page).
Move the parts of the byte compilation section that nobody ever uses off to the Appendix page.
Make the single-rpm single-dir case the default. Move the entire multiple-dir case to the appendix.
Completely remove %with_python3 from the example spec. We want people to build for python3. People just paste this in without knowing why they would ever need it. If you're using the new macros you can't just conditionalize for EL6 anyway. This really makes the spec look nice. Macro-izing the summary and description would make it look even nicer once the description gets longer than a line.
Uses a generic "example" module as an example. Two different approved proposals changed it to two different things; I just punted.
Puts the example spec all together without a bunch of text and admons and whatnot in the middle.
Moves the 2to3 section to the appendix.
Simply refers to the eggs section. That whole part at the bottom was quite awkward and I don't think most people even looked down that far.
Loads and loads of typo fixes, grammar fixes, and cleanups.

The current main guideline page is now a bit over half the length it was and on the page it's less because of the collapsed macro table. I find it to be far more readable, and the sample spec no longer turns my stomach.

The old guidelines page is just as it was before writing up the new macros, except that I added a short explanatory section at the top.

The appendix page conveniently holds things which we probably should document but pretty much nobody would actually want to read without some specific reason.

Functionally I do not believe I have gone beyond anything upon which we voted but I wanted to toss this out there and see if anyone yells before I copy it into place. Been trying to get this done for three days now and I think I'm finally there.

I think the draft it very good.

Some comments:
- in https://fedoraproject.org/wiki/User:Tibbs/PythonCleanup2#Example_common_spec_file:
%{_bindir}/sample-exec-2.7 and %{_bindir}/sample-exec-3.4 should probably use %{python2_version} and %{python3_version}. Python 3.5 transition is right around the corner.

  • %global _docdir_fmt %{name} would still be good

  • "Transitioning from python2 to python3 is left to individual package maintainers except for packages in Fedora's critical path. For these, we want to port to python3 versions in the same Fedora release if possible." This could be simplified to "Transitioning from python2 to python3 is left to individual package maintainers", because the second part is already untrue and will never happen, and doesn't really matter for packaging.

Looks good, really an improvement! In the sample spec file you include, as far as I can see %{pypi_name} hasn't been defined. Perhaps you meant to replace that with %{srcname}?

Metadata Update from @zbyszek:
- Issue assigned to tibbs

2 years ago

Login to comment on this ticket.