https://fedoraproject.org/wiki/User:Mstuchli/Python3inEPEL
The modified python guidelines describe how to package Python 3 applications and modules for EPEL 7. A more detailed explanation of the whys and hows is available at [0].
This has already been approved by EPSCo, we'd like to have the amendments approved by FPC and finally we'll need to have the necessary macros added to minimal buildroot.
[0] https://fedoraproject.org/wiki/User:Bkabrda/EPEL7_Python3
After finally getting the python guidelines cleaned up, there's no way I want to add that much stuff. In fact, I'm completely dismayed that we worked to simplify the guidelines and the necessary spec line noise only to have more of this terrible stuff added.
In any case, this should all go in the EPEL guidelines. Please don't complicate the Fedora guidelines. Please work really hard to make packages which function with the current Fedora guidelines just work on EPEL7 so that the Fedora packages aren't contaminated with this stuff.
Note that EPSCo isn't a typo for FESCo, it's the people from epel-devel (https://lists.fedoraproject.org/pipermail/epel-devel/2014-August/010065.html). Which is closer to the KDE SIG than FESCo, AFAIK.
When we discussed this last someone said they were going to update the draft with regard to how the current Fedora guidelines look ... that doesn't seem to have happened yet.
Also, is it fair to say that all of the guidelines will be optional and for EPEL only? In other words there will be two options for anyone packaging py3 packages for EPEL, Fedora or EPEL? And it's completely up to the maintainer of each package? What about if you package XYX that depends on ABC and you want to do 3.x packaging but ABC maintainer doesn't?
Can you try to clarify what the downside is if py3 in EPEL doesn't have version explosion? Is it just that you have to use a copr to test each new release of py3 and then do a bunch of synchronized updates? Because atm. the downsides of doing this are much clearer.
I modified the example spec in https://fedoraproject.org/wiki/User:Bkabrda/EPEL7_Python3 to reflect the changes in the python3 packaging guidelines. I suspect that should be put in https://fedoraproject.org/wiki/User:Mstuchli/Python3inEPEL as well.
I think the main reason for maintaining two python3 stacks in epel is that it is going to take a while to migrate from one to the next and that we need to be able to issue updates for the old stack during that transition.
The big problem with copr for stuff like this at the moment is the lack of group access to a copr project - it's all one person.
My sense is that this would have to be mandatory for packages wishing to provide python3 support in EPEL, but that doing so is optional. And if the python-foo maintainer in didn't want to support EPEL, somebody else could submit and maintain python3-foo for EPEL only.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-09-03/fpc.2015-09-03-16.00.txt):
I've been playing around with trying to generate the needed python 3 packages via a macro and having limited success. The biggest hurdle I've found is that it seems impossible to avoid having spaces separate arguments to rpm macros. So you have to pass in everything as a pre-defined macro.
This is the best I could come up with so far:
{{{ %python3_package() %{lua:\ function gensubpkg(name, py3pkgver)\ py3pkgname = string.format("python%s", py3pkgver);\ py3subpkgname = string.format("%s-%s", py3pkgname, name);\ print(string.format("\n%%package -n %s\n", py3subpkgname));\ newhdr = string.gsub(hdr, "python3", py3pkgname);\ print(newhdr);\ print("\n");\ provline = string.format("%%{?python_provide:%%python_provide %s}", py3subpkgname);\ print(rpm.expand(provline));\ print(string.format("\n%%description -n %s\n", py3subpkgname));\ newdesc = string.gsub(desc, "python3", py3pkgname);\ print(newdesc);\ end\ name = rpm.expand("%{?1:%{1}}");\ hdr = rpm.expand("%{?2:%{2}}");\ desc = rpm.expand("%{?3:%{3}}");\ with_other = rpm.expand("0%{?with_python3_other}");\ vr = rpm.expand("%{?epoch:%{epoch}:}%{version}-%{release}");\ py3pkgver = rpm.expand("%{python3_pkgversion}");\ gensubpkg(name, py3pkgver);\ if tonumber(with_other) > 0 then\ py3pkgver = rpm.expand("%{python3_other_pkgversion}");\ gensubpkg(name, py3pkgver);\ end\ } }}}
Called with something like: {{{ %global pkgdescription \ A description of the package \ over several lines.
.....
%global python3_hdr \ Summary: %{sum} \ BuildRequires: python3-devel \ Requires: python3-Cython \ Requires: python3-numpy
%python3_package %{srcname} %{python3_hdr} %{pkgdescription}
%files -n python%{python3_pkgversion}-%{srcname} %license COPYING %{python3_sitelib}/%{srcname}/
%if 0%{?with_python3_other} %files -n python%{python3_other_pkgversion}-%{srcname} %license COPYING %{python3_other_sitelib}/%{srcname}/ %endif }}}
But I'm not sure if this is any better. It would be nice if we could somehow access the main package description and summary somehow from lua, but that's relatively minor.
We could conceivably generate the %files section as well, but that would require passing in the files as a macro argument as well.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-09-10/fpc.2015-09-10-16.00.txt):
So here's the generic %py_package() macro: {{{ %py_package() %{lua:\ function gensubpkg(name, pypkgver)\ pypkgname = string.format("python%s", pypkgver);\ pypkgprefix = string.format("python%s-", pypkgver);\ pysubpkgname = string.format("%s-%s", pypkgname, name);\ print(string.format("\n%%package -n %s\n", pysubpkgname));\ newhdr = string.gsub(hdr, "python%d?-", pypkgprefix);\ print(newhdr);\ print("\n");\ print(rpm.expand(string.format("%%{?python_provide:%%python_provide %s}\n", pysubpkgname)));\ print(string.format("\n%%description -n %s\n", pysubpkgname));\ newdesc = string.gsub(desc, "python%d", pypkgname);\ print(newdesc);\ print("\n");\ end\ name = rpm.expand("%{?1:%{1}}");\ hdr = rpm.expand("%{?2:%{2}}");\ desc = rpm.expand("%{?3:%{3}}");\ vr = rpm.expand("%{?epoch:%{epoch}:}%{version}-%{release}");\ print(string.format("\n%%description -n python-%s\n", name));\ print(desc);\ print("\n");\ gensubpkg(name, "2");\ if tonumber(rpm.expand("0%{?with_python3}")) > 0 then\ py3pkgver = rpm.expand("%{python3_pkgversion}");\ gensubpkg(name, py3pkgver);\ if tonumber(rpm.expand("0%{?with_python3_other}")) > 0 then\ pypkgver = rpm.expand("%{python3_other_pkgversion}");\ gensubpkg(name, pypkgver);\ end\ end\ } }}} Used like: {{{ %global pkgdesc \ A description of the package \ over several lines.
%global header \ Summary: %{sum} \ BuildRequires: python-devel \ Requires: python-Cython \ Requires: python-numpy
%py_package %{srcname} %{header} %{pkgdesc} }}} The %py_package() macro will convert all python- and python#- prefixes to the appropriate version for the given sub-package. It also outputs the main dummy package %description section.
The caveat to use is that all needed python deps must provides python2/3-foo names. This is required by the currently guidelines, but not all packages are updated yet.
There would also be a %py3_package() macro for EPEL7 python3 only packages to use.
I've updated https://fedoraproject.org/wiki/PackagingDrafts:Python with my take on the changes.
https://fedoraproject.org/w/index.php?title=PackagingDrafts%3APython&diff=cur&oldid=422445 should show the changes from the current guidelines.
I'm thinking it should be possible for %header to be generated by the macro (allowing you to override it, or just ignore the macro entirely and write things out the long way if you wish, but I just haven't had the time to figure all of this out yet. orion and tomspur seem to be the experts here.
I will find time one day.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-10-01/fpc.2015-10-01-16.00.txt):
tomspur and I have been playing with some macros at https://pagure.io/python-macros
We currently have some macros which automatically build for all of the available python versions in one macro. Not done, of course, but %py_prep, %py_build, %py_install and %py_test all work on a couple of test packages we've been converting. %py_package works as well, though it needs more hacking. We're trying to figure out if there's something we can do with file lists and build dependencies.
Just a note that the macros at https://pagure.io/python-macros have progressed significantly, though they're still not completely done. tomspur and I talked a bit on IRC yesterday about how to handle %files. Currently everything else can be done without requiring the packager to know which versions of python happen to be supported on each platform. Adding more complete python stacks only requires a single macro tweak in macros.python (or whatever it ends up being called). These macros should not conflict with any existing macros, and I don't believe they conflict with anything that happens to be defined in any current package. That means that if they were to be added to redhat-rpm-config today, nothing would change except their availability in the buildroot.
For an initial pass, not much remains to be done. I need to experiment with the best way to handle %files and then I can start doing test conversions of random python packages to see what use cases I'm missing.
Also, doing anything remotely fancy in Source: or Patch: tags breaks spectool. This is an issue now for any spec that does anything fancy, but it would also mean that any package using these new macros would also break spectool. I like spectool and use it often, but I looked at how it works and it's pretty terrible so I just rewrote it in python. Fixed a bunch of bugs that always annoyed while I was at it. I just need to polish it up, and then figure out what to do with it.
Hi,
I recently wrote a small app in python3 that I want to package for epel7, which lead me to this ticket.
Do we have any progress on this?
Thanks!
Nothing in here prevents packaging python3 things for epel7. If you insist on keeping all of the specs identical across branches then you get to enjoy some ifdef spaghetti.
We're still playing with the macros, but since nothing is blocking on them it isn't an urgent issue.
What is the current progress on python3-specific macros? Are they not in place yet?[[BR]] I'm just going to create some tickets in BZ on providing some python3 extensions in epel7, but still unsure whether everything is according the [https://fedoraproject.org/wiki/User:Bkabrda/EPEL7_Python3 guidance above].
The fancy macros we've been playing with, but they're not in place and will require a huge amount of testing before they are.
In the meantime, ifdef spaghetti is the only option.
Well, of course you can also just keep different specfiles per release and have those be free of the ifdef mess.
'''@tibbs''' well, I'm going to follow #comment:22 till the moment when fancy macros are available everywhere. FYI, I've actually filled a bug against python34-msgpack: https://bugzilla.redhat.com/show_bug.cgi?id=1290393
I'd like to revisit the buildroot recommendations in light of https://lists.fedoraproject.org/archives/list/packaging%40lists.fedoraproject.org/thread/KGHLT2HMNXCFMK4XHAZMKHGSE2SIAFWM/
I think current practice suggests creating a python-srpm-macros package that contains only the macros necessary for building srpms, which in this case is the %{python3_pkgversion} macro.
I think it still makes sense to create a separate python-rpm-macros package that contains the macros for building the RPMs, but that this doesn't go into the buildroot.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-01-07/fpc.2016-01-07-17.00.txt):
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-01-14/fpc.2016-01-14-17.01.txt):
Finally announcing things and I note that this is sitting in the "announce" state but I think there's something I'm supposed to actually write into the guidelines. Thing is, I can't seem to figure out what it actually is.
Ugh, trac workflow is weird and I can't fix it due to javascript errors....
OK, the status should be good now. Sorry for the spam.
Basically, the EPEL7 guidelines aren't ready so we couldn't link to them even if we wanted to do so (which personally I don't).
Let's wait for that, and in the meantime I'll try to revive the python macros I was working on which should hopefully eliminate the differences between EPEL7 and Fedora.
Metadata Update from @pingou: - Issue assigned to tibbs
I'd like to revive this issue.
Python 3.5 was skipped in favor of Python 3.6.
The package has been built [0][1] and is parallel installable with the current Python 3 in EPEL 7(python 3.4).
I can work on a scripted mass rebuild of the python3 packages in epel in order for those to transition to the 3.6 abi, however I'd like to know first what is it required to be done now in regards to the guidelines.
[0] https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-ddd7d1e59d [1] https://lists.fedoraproject.org/archives/list/epel-announce@lists.fedoraproject.org/thread/VLTFSZEQHXBZHHDAKMT4KCQEKPSFV5OS/
bump?
I'm also interested in Python 3 packaging for EPEL7.
Can somebody point me to the draft that is most current? Perhaps https://fedoraproject.org/wiki/PackagingDrafts:Python3EPEL?
That should mostly work. Right @cstratak? Also note that 3.6 is currently available as the "other pkg version", built most of the packages never built against it.
Metadata Update from @churchyard: - Issue close_status updated to: None
Indeed @tadej that should be the most current draft, but not necessarily up to date. Also I'm thinking that it would be a good time to prepare the ground for making the 3.6 stack the default python3 stack in epel 7.
@churchyard, @cstratak, thanks for your answers.
I would also like to revive this issue, help making the guidelines official and update the packages that I'm interested in.
One thing that I noticed while I updated the uWSGI's Python plugins to work with both Python 3 versions available in EPEL 7 is that some python3_other_* macros are missing.
python3_other_*
I ended up defining them directly in the SPEC file.
Can I create a PR and add them to macros.python3?
macros.python3
I'm not sure about other macros, e.g. %py3_build(), %py3_build_egg(), ... Would it be useful to have %py3_other_* versions of them?
%py3_build()
%py3_build_egg()
%py3_other_*
Note that EPEL guidelines aren't really in the purview of this committee. However, if there are changes needed which provide compatibility between EPEL and Fedora, and those changes don't introduce needless complication into the Fedora guidelines, then feel free to propose those changes.
Otherwise, you can edit https://fedoraproject.org/wiki/EPEL:Packaging as needed. If additional macros are needed at SRPM build time, I can help get them added to epel-rpm-macros. Just ping me, or file a bugzilla ticket.
@tadej, we have noticed this as well in https://src.fedoraproject.org/rpms/python3-setuptools/pull-request/2
Please, add the macros.
@tadej, we have noticed this as well in https://src.fedoraproject.org/rpms/python3-setuptools/pull-request/2 Please, add the macros.
@churchyard, PR opened: https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/5
I think there is nothing to do for committee, EPEL is out of our scope.
Metadata Update from @ignatenkobrain: - Issue close_status updated to: nothingtodo - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.