#534 Add provides for python2-foo to python-foo packages
Closed: Fixed None Opened 4 years ago by rkuska.

I would like to add provides for python-foo packages to provide also python2-foo.

https://fedoraproject.org/wiki/User:Rkuska/provides

Question is what to do with packages which were created in a times when there was a py* naming exception (for example pytest).
I see 2 possibilities:
1. add provides python-foo
2. rename package to python-foo and add provides/obsolete for foo

I wrote my reasoning in mail posted to python-devel
https://lists.fedoraproject.org/pipermail/python-devel/2015-April/000693.html


I think renaming the {{{python-setuptools}}} package to {{{python2-setuptools}}} and provide python-setuptools from the {{{%{?py_default_major}}}} package would be more flexible for the python modules. This could be added right now to both subpackages and based on the beginning of the right {{{python%{?py_default_major}}}} it could emit the plain {{{python-setuptools}}}.
What's your opinion towards this? Would this also help on unified packaging for epel?

For the py* naming exceptions, I'd vote for the rename too, so we get more consistency (and hopefully less amounts of python guidelines)...

The biggest problem with using python2-setuptools as the package name is that you then have users encountering bugs in python3-setuptools and having to file them against python2-setuptools in bugzilla. Similarly, applying for pkgdb acls, approving karma in bodhi, and anything else based on package source name. because of that you need to either keep the source package name python-foo or split the python2 and python3 modules into two separate source packages.

For the py* naming exceptions, renaming the package to python-pytest makes the most sense.

However, for all of this remember that someone has to do the work. In the past I've organized people to do the work for large python undertakings like this (removing dependencies on python-setuptools-devel. Moving from PIL to pillow) via a Fedora Change. But the majority of the work was performed by just a few people. So you probably need to plan on a Fedora Change for this and also marshal internal developer resources to convert all the packages over.

With that in mind, make sure you aren't taking on so many changes at one time that you don't have the free time to perform the change.

Sorry, I should be more clear that I was talking about this example of a spec file linked above:

https://fedoraproject.org/wiki/User:Rkuska/provides

The package is still named {{{python-setuptools}}} with no {{{%files}}} section but two subpackages for python2 and python3 and the "right" subpackage will emit the provides.

Right, that's the proposal. But now I'm not sure what you are wanting to change from that?

I want to forbid the usage of a unversioned python {{{%files}}} section and an automatic switch to new python runtimes.

Here is the [https://fedoraproject.org/w/index.php?title=User%3ATomspur%2FPackaging%3APython&diff=cur&oldid=413644 proposed diff] to the packaging guidelines.

When building for more than one python runtime from one spec file, the {{{%python_provide}}} macro ensures that once we flip the bit for a new python runtime, the default {{{python-$foo}}} will also switch accordingly (e.g. At least in theory {{{pypy3-setuptools}}} could provide {{{python-setuptools}}} when the macro is changed).

Here is the python_provide macro, that seems to be working over here (I'm new to lua, so don't know if it can be further simplified):
{{{

%python_provide() %{lua:\
function string.starts(String,Start)\
return string.sub(String,1,string.len(Start))==Start\
end\
str = rpm.expand("%{?1:%{1}}");\
vr = rpm.expand("%{version}-%{release}")
if (string.starts(str, "python2-")) then\
print("Provides: python-");\
print(string.sub(str,9,string.len(str)));\
elseif (string.starts(str, "python3-")) then\
--No unversioned provides as python3 is not default\
elseif (string.starts(str, "pypy-")) then\
--No unversioned provides as pypy is not default\
elseif (string.starts(str, "pypy3-")) then\
--No unversioned provides as pypy is not default\
elseif (string.starts(str, "python-")) then\
--Providing the current default python\
print("Provides: python2-");\
print(string.sub(str,9,string.len(str)));\
else\
print("ERROR: ");\
print(str);\
print("not recognized.");\
end\
print(" = ");\
print(vr);\
}

}}}

Replying to [comment:2 toshio]:

The biggest problem with using python2-setuptools as the package name is that you then have users encountering bugs in python3-setuptools and having to file them against python2-setuptools in bugzilla. Similarly, applying for pkgdb acls, approving karma in bodhi, and anything else based on package source name. because of that you need to either keep the source package name python-foo or split the python2 and python3 modules into two separate source packages.

For the py* naming exceptions, renaming the package to python-pytest makes the most sense.

However, for all of this remember that someone has to do the work. In the past I've organized people to do the work for large python undertakings like this (removing dependencies on python-setuptools-devel. Moving from PIL to pillow) via a Fedora Change. But the majority of the work was performed by just a few people. So you probably need to plan on a Fedora Change for this and also marshal internal developer resources to convert all the packages over.

With that in mind, make sure you aren't taking on so many changes at one time that you don't have the free time to perform the change.

That's a good point. I've opened this ticket to have already python2-foo packages in place before the f24 comes where I want to propose the new packaging in following format (from what I understand this is also what Tom meant by subpackaging, right?):

https://fedoraproject.org/wiki/User:Rkuska/f24packaging

But the truth is that is not that important and I can just solve both problems in one feature (in f24).

I am opposed to what Tom suggest as I am against the idea that python-foo packages will provide python3-foo package (or vice versa) until upstream decides what to do with /usr/bin/python.

I do realize that specfiles will get a bit more verbose by this and therefore it would be great to solve this verbosity by https://fedorahosted.org/fpc/ticket/281

Replying to [comment:6 rkuska]:

Replying to [comment:2 toshio]:

With that in mind, make sure you aren't taking on so many changes at one time that you don't have the free time to perform the change.

That's a good point. I've opened this ticket to have already python2-foo packages in place before the f24 comes where I want to propose the new packaging in following format (from what I understand this is also what Tom meant by subpackaging, right?):

https://fedoraproject.org/wiki/User:Rkuska/f24packaging

This would be the alternative to my proposal. I'd like to be able to install {{{python-click}}} (when looking at your example linked above), which will install the click module for the default python implementation, which would be python2 on F22 and python3 for F23+. The lua macro in comment:5 would make this possible.

But the truth is that is not that important and I can just solve both problems in one feature (in f24).

I am opposed to what Tom suggest as I am against the idea that python-foo packages will provide python3-foo package (or vice versa) until upstream decides what to do with /usr/bin/python.

You don't want to do that yet untill upstream decides what to do with {{{/usr/bin/python}}}, so the flip in the lua macro can also just happen later than the change of the default implementation. This is fine and I don't propose to change anything with that. It is just a macro to remove some verbosity and make the python packages easier to maintain (at least from my pov).

Replying to [comment:7 rkuska]:

I do realize that specfiles will get a bit more verbose by this and therefore it would be great to solve this verbosity by https://fedorahosted.org/fpc/ticket/281

Could you also try that macro over there? It doesn't work for me so far for the non-{{{setup.py}}} case.

So I guess we have two questions for the next meeting:

  • Rename pyfoo packages to python-foo and add provides/obsoletes for pyfoo or leave them as is and add provides.
    Note that the former requires re-reviews or an mass rename exception from fesco such as done here:
    https://fedorahosted.org/fesco/ticket/812
    FPC can only help with the decision, but implementation/change needs to be done outside of that.

I'd favor renames and could help with the reviews, but cannot drive that change. rkuska will try to join the meeting.

  • Add Provides python2-foo to python-foo package for F23 and forbid python-foo packages in spec files for F24 where packages for more than one python implementation is built.

Alternatively the lua macro could be directly added to the guidelines, so that new packages are already set for the final F24 result. Old packages don't need much changes (just add the %python_provide macro instead of plain Provides:), which gives more time to gratually change all python packages for F24.

I'd favor the lua macro obviously :)

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-06-04/fpc.2015-06-04-16.02.txt):

Since I don't actaully see a draft of what was padded, I've written this up by adding a short "Provides" section after "BuildRequires" and modifying the included specfile template. Hope it looks OK to everyone.

Announcement text:

Updated the Python guidelines to indicate that Python2 modules must have Provides: python2-foo.
https://fedoraproject.org/wiki/Packaging:Python
https://fedoraproject.org/w/index.php?title=Packaging%3APython&diff=416070&oldid=416009
* https://fedorahosted.org/fpc/ticket/534

Will hold off on announcing this until the macros get in which simplify the actual implementation. This will save people from thinking they have to update packages now, and then doing it again after the macros are in.

The statement that you must have Provides: python2-foo still needs to be there regardless.

Sorry for the confusion, the diff above in [comment:10 comment#10] has changed as there is {{{cur}}} in it and I used the same wiki page for further editions of ticket #281.

The diff, we voted on this ticket is [https://fedoraproject.org/w/index.php?title=User%3ATomspur%2FPackaging%3APython&diff=414855&oldid=413644 this one] and any further edits will be dealt with in #281 as [https://fedoraproject.org/w/index.php?title=User%3ATomspur%2FPackaging%3APython&diff=cur&oldid=414855 this diff].

Hi guys, just wanted to chime in with a proposal to this discussion - I think the python2-foo provides should be versioned, e.g. something like {{{Provides: python2-foo = %{version}-%{release}}}}. This should allow all the various version checks even if depending packages use the python2 provides.

Hope this makes sense.

When using the {{{python_prove}}} macro it will append the version and release automatically, see [http://pkgs.fedoraproject.org/cgit/python.git/tree/macros.python#n17 here]. So it just needs to be appened, when adding manually the requires.

Replying to [comment:16 tomspur]:

When using the {{{python_prove}}} macro it will append the version and release automatically, see [http://pkgs.fedoraproject.org/cgit/python.git/tree/macros.python#n17 here]. So it just needs to be appened, when adding manually the requires.

Ok, thanks. Two notes:
- This should perhaps be clarified in the current version of guidelines, which say nothing of version/release.
- I think the macro should also take epoch into account, something like {{{%{?epoch:%{epoch}:}%{version}-%{release}}}}.

Metadata Update from @tomspur:
- Issue assigned to tibbs

2 years ago

Login to comment on this ticket.

Metadata