#103 python38-rpm-macros brought into EPEL8.2 buildroot
Closed: Fixed 3 years ago by tdawson. Opened 3 years ago by orion.

Looks like with RHEL8.2 we're getting the following in the buildroot:

DEBUG util.py:602:   python36                               x86_64  3.6.8-2.module+el8.1.0+3334+5cb623d7 build   19 k
DEBUG util.py:602:   python38-rpm-macros                    noarch  3.8.0-6.module+el8.2.0+5978+503155c0 build   66 k

This appears to be setting %{__python3} to python3.8:

sh: /usr/bin/python3.8: No such file or directory

but we're still really building with python 3.6


Here's the rule: When building python packages in EL8, you need one of python3X-rpm-macros installed. Multiple are available, only once can be installed at a time.

IMHO EPEL macro packages should suggest python36-rpm-macros to make it take precedence.

cc @torsava

Well, it does... sort of.

rpm -q --requires epel-rpm-macros
(python-modular-rpm-macros if python3-devel)

And python-modular-rpm-macros is provides by python36-rpm-macros.

But now python38-rpm-macros also provides the unversioned python-modular-rpm-macros, even though python38 is not the default python3 that provides python3-devel.

The hard requirement must be on the general name, otherwise you could not build Python 3.8 packages on EPEL 8.

The soft requirement however, should IMHO be on 3.6.

EPEL8 gathers all the default modules from RHEL8 and put them into the repo.
The problem is, that both python36 and python38 have been marked as default, despite some conflicts,

Is this considered a bug in RHEL8?

What if we semi-hard coded it?
Change
Requires: (python-modular-rpm-macros if python3-devel)
To
Requires: (python36-rpm-macros if python36-devel)
Requires: (python38-rpm-macros if python38-devel)

Yes, we would have to update that each time RHEL8 comes out with another default python version, but at least we wouldn't have this problem.

Updated epel-rpm-macros with the above fix.
I have tested it and it is working correctly.
I currently have it in override for three weeks, and in bodhi.

https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-6e26ad6672

This won't work properly:

Requires: (python36-rpm-macros if python36-devel)
Requires: (python38-rpm-macros if python38-devel)

It means python36-devel and python38-devel cannot be installed on the same system with epel-rpm-macros installed.

This should do the right thing, as said in my first comment:

Requires: (python-modular-rpm-macros if python3-devel)
Suggests: python36-rpm-macros

Is this considered a bug in RHEL8?

No.

This should do the right thing, as said in my first comment:
Requires: (python-modular-rpm-macros if python3-devel)
Suggests: python36-rpm-macros

Ahh, I had no idea Suggests was even a thing. So I glossed over that. Now I know.
Running some tests, and then updating.

Results from testing:

Requires: (python-modular-rpm-macros if python3-devel)
Suggests: python36-rpm-macros
====
python36-devel : python36-rpm-macros installed
python38-devel : no rpm-macros  installed
python36-devel python38-devel : python36-rpm-macros installed
Requires:       (python38-rpm-macros if python38-devel)
Requires:       (python-modular-rpm-macros if python3-devel)
Suggests:       python36-rpm-macros
====
python36-devel : python36-rpm-macros installed
python38-devel : python38-rpm-macros installed
python36-devel python38-devel : python38-rpm-macros installed
Requires:       (python36-rpm-macros if python36-devel)
Requires:       (python-modular-rpm-macros if python3-devel)
Suggests:       python38-rpm-macros
====
python36-devel : python36-rpm-macros installed
python38-devel : no rpm-macros  installed
python36-devel python38-devel : python36-rpm-macros installed

The last part was odd, so a little more investigation showed that only python36-devel is providing python3-devel. Thus our Requires is only getting triggered by python36-devel.

So, yes, using Suggests only will do the job. But, then if someone does want to build a package with python38-devel only, they won't get the correct macros.

I'm thinking the best route would be the middle one

Requires:       (python38-rpm-macros if python38-devel)
Requires:       (python-modular-rpm-macros if python3-devel)
Suggests:       python36-rpm-macros

That way you always get the rpm macros no matter which you choose. You are able to install both python36-devel and python38-devel.
The only drawback is that if you have both installed, you have to have python38-rpm-macros installed and not python36.

I tried Suggests: (python38-rpm-macros if python38-devel) but that doesn't seem to work if you also have Suggests: python36-rpm-macros

So, yes, using Suggests only will do the job. But, then if someone does want to build a package with python38-devel only, they won't get the correct macros.

That is fine. The only RHEL-supported way to build RHEL 8 Python 3.8 packages is to pull in the macros as BuildRequires.

That way you always get the rpm macros no matter which you choose. You are able to install both python36-devel and python38-devel.
The only drawback is that if you have both installed, you have to have python38-rpm-macros installed and not python36.

Also, this will stop working once (if) python39-devel is added to RHEL 8.

I tried Suggests: (python38-rpm-macros if python38-devel) but that doesn't seem to work if you also have Suggests: python36-rpm-macros

What about:

Requires: (python-modular-rpm-macros if python3-devel)
Suggests: (python36-rpm-macros if python36-devel)
Suggests: (python38-rpm-macros if python38-devel)

That way you get the proper macros when you only have one python3X-devel installed, but you still can have both devels installed (in that case, the macros will be "random" again, but you can switch manually).

(If that solution works, I'll lobby RHEL 8 python3X-devel packages to add Suggests: python3X-rpm-macros.)

What about:
Requires: (python-modular-rpm-macros if python3-devel)
Suggests: (python36-rpm-macros if python36-devel)
Suggests: (python38-rpm-macros if python38-devel)

Nope, still doesn't install anything if only python38-devel is installed.
I even tried
Recommends: (python38-rpm-macros if python38-devel)
And that doesn't work.
Even by itself, the Recommends doesn't work, which I think is a bug.

I'm starting to feel that you are right about python3.8 packages should include the rpm-macros in their BuildRequires.

It is a holiday today in CZ. I suggest to wait til Monday, for @torsava who can share some RHEL point of view.

Sounds good.
I have an updated epel-rpm-macros up in the epel8 override repo's. So python builds will build for now.
But next week, let's figure out the correct solution.

Honestly, the main reason I added the provide python-modular-rpm-macros was so that so I could do this:

Provides:   python-modular-rpm-macros == 3.6
Conflicts:  python-modular-rpm-macros > 3.6

and

Provides:   python-modular-rpm-macros == 3.8
Conflicts:  python-modular-rpm-macros > 3.8

to conflict with any possible higher version, as a higher version will always override the lower one.

I don't think it makes sense at any point to require/suggest the name python-modular-rpm-macros, because I can't imagine a situation where you don't care which package you have.

I'm starting to feel that you are right about python3.8 packages should include the rpm-macros in their BuildRequires.

I agree.

I don't think it makes sense at any point to require/suggest the name python-modular-rpm-macros, because I can't imagine a situation where you don't care which package you have.

I'm starting to feel that you are right about python3.8 packages should include the rpm-macros in their BuildRequires.

I agree.

If I'm reading this right, it sounds like you think we should take the whole
Requires: (python-modular-rpm-macros if python3-devel)
out of epel-rpm-macros.

Is that correct?

I don't think it makes sense at any point to require/suggest the name python-modular-rpm-macros, because I can't imagine a situation where you don't care which package you have.
I'm starting to feel that you are right about python3.8 packages should include the rpm-macros in their BuildRequires.
I agree.

If I'm reading this right, it sounds like you think we should take the whole
Requires: (python-modular-rpm-macros if python3-devel)
out of epel-rpm-macros.
Is that correct?

Yes, I don't think that makes sense now, after a 2nd conflicting python-modular-rpm-macros package was added.

If you can make a reliable system that pulls in python36-rpm-macros to the buildroot only if python3-devel is present, and python38-rpm-macros only if python38-devel is present, that would be marvellous. But it looks like it's flaky at best, plus we would have to figure out what to do if both are present.

So barring that, I'd say BuildRequiring the specific macros from the spec file is the best solution.

Note that dozens of python3 packages in epel8 will suddenly build with platform python, if we remove the current thing.

Note that dozens of python3 packages in epel8 will suddenly build with platform python, if we remove the current thing.

That's true. The packages would need to be transitioned to the BR. At least some of them. I'm not sure if it's been settled which packages should build on which Python.
Python modules that are to be consumed by users should definitely be built with python36-rpm-macros. But tools that run on Python, they might be better off building against platform-python.

Just so we know what's in them, here are the macros from the rpms.

python36-rpm-macros

%__python3 /usr/bin/python3.6

python38-rpm-macros

%__python3 /usr/bin/python3.8
%python3_pkgversion 38

%py3_install_wheel() %{expand:\\\
  CFLAGS="%{optflags}" %{__python3} -m pip install -I dist/%{1} --root %{buildroot} --no-deps
}

If we completely remove the Requires, then neither of those get installed unless a packager specifically adds either.

<No Requires:  (python-modular-rpm-macros if python3-devel) >
====
python36-devel : no python3X-rpm-macros  installed
python38-devel : no python3X-rpm-macros  installed
python36-devel python38-devel : no python3X-rpm-macros  installed

I did a quick check (before Miro sent the above comment) and there are quite a few packages using __python3 in their spec files. If we don't have it, then many of those packages are going to want the platform python.

My recommendation is what Miro said above.

Requires: (python-modular-rpm-macros if python3-devel)
Suggests: python36-rpm-macros

I think that would keep things consistent with what is currently expected.
If people want their python packaged with python38 or higher, they will need to pull in the python38-rpm-macros.
And there is nothing preventing them from listing the python36-rpm-macros if they want to keep things consistent across the different python versions they are building for.

But I am also not a major python packager. So I will defer to you two. It just took a while for me to get that written.

Requires: (python-modular-rpm-macros if python3-devel)

One thing to think about is that this means no package can be built for platform-python, which I'm not sure is a good idea.

My recommendation is what Miro said above.
Requires: (python-modular-rpm-macros if python3-devel)
Suggests: python36-rpm-macros

Also, since python3-devel is a Python 3.6 package, and python38-devel does not provide that name, it would be better to do an explicit:

Requires: (python36-rpm-macros if python3-devel)

(Same limitation applies still - it will not be possible to build packages for platform-python.)

EDIT: After a further think, this would break a case where a package wants to be built for python38, but for some reason needs BR on python3-devel. Though, I'm not sure such a case makes sense.

What you suggested in the last comment indeed makes python3-devel not co-installable with python38-rpm-macros in epel.

One thing to think about is that this means no package can be built for platform-python, which I'm not sure is a good idea.

That was the status quo in EPEL before this discussion. I think it's OK, platform-python is for crucial things, crucial things are in RHEL-proper. EPEL packagers expect %__python3 to be /usr/bin/python3.

What you suggested in the last comment indeed makes python3-devel not co-installable with python38-rpm-macros in epel.

Ah that's true. For a second I thought we were making some rule directly for the buildroot. If it's in epel-rpm-macros, users might have that on their own machines and it would complicate things further.

With the release of epel-rpm-macros-8-13 I believe this has been fixed.

Metadata Update from @tdawson:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata