#203 %{_libdir} usage in noarch packages.

Created 4 years ago by kdaniel
Modified 2 months ago

Ticket 161 https://fedorahosted.org/fpc/ticket/161, comment 12 states:

It is fine to use %{_libdir} for its intended purpose at build-time of noarch packages. It would be very wrong to use it for other than its intended purpose.

Correct usage in a noarch package could be something like this:

# We link against libeclipse.so at build time. # At runtime, /usr/bin/eclipse has already loaded the correct version for our arch sed -i 's!/usr/lib/eclipse/libeclipse.so.1!%{_libdir}/eclipse/libeclipse.so.1!g' Makefile

Incorrect usage in a noarch package could be something like this:

# We install into the eclipse directory sed -i 's!INSTALLDIR=/usr/lib/eclipse/plugins/!INSTALLDIR=%{_libdir}/eclipse/plugins/!'

==============

Unfortunately, according to
https://bugzilla.redhat.com/show_bug.cgi?id=847130 comment 3
‚Äčhttp://lists.fedoraproject.org/pipermail/devel/2012-April/165292.html
%{_libdir} does not work in noarch packages.

I'd like to get an "official" statement what is the correct/expected behaviour and use it for opening appropriate bugs.

Proposal:

Noarch packages must not use %{_libdir}, since that macro has no useful value in noarch build environments.

Yes, I was wrong in comment #12. The use case is valid but the use of %{_libdir} is not. Instead, for that use case, you'd want to use a macro that was expanded at the time the eclipse package was built or some value that comes from the eclipse program.

I can get behind that proposal, but would it be useful at all to have some testing of what value it does take under various circumstances? Or would that just be patterning guidelines on behavior the rpm maintainers wouldn't want to promise to maintain?

I believe when I looked recently, %{_libdir} was evaluating inside RPM using the "isa" macros, and since they're all unset on noarch (intentionally), the only part that ever comes out on noarch is /usr/lib.

One then wonders if _libdir could just be unset there as well, rendering it unnecessary to worry about people using it.

Replying to [comment:2 toshio]:

Yes, I was wrong in comment #12. The use case is valid but the use of %{_libdir} is not. Instead, for that use case, you'd want to use a macro that was expanded at the time the eclipse package was built or some value that comes from the eclipse program.
Is our own limited macro included into this category?

At the least, the macro was implemented badly. It could be done with something like this in the eclipse spec file:
{{{
cat << EOF
%%_eclipse_base %{_libdir}/eclipse

EOF > eclipse.macros
}}}
The idea is that %{_libdir} would be expanded when rpm builds the arch-specific eclipse package (I'm assuming eclipse is arch specific... didn't go back and check). That way the package that uses the %_eclipse_base macro on a specific system gets the %{_libdir} that the arch-specific eclipse package built into the macro.

Whether this is okay or not for use with the use case in https://fedorahosted.org/fpc/ticket/161 I'm not sure. We didn't get to the point of definitively knowing what the macro was being used for before the workaround was applied. It's also dangerous to have macros like these if either the people using them don't know what they're doing with them or the build process is not so simple that the people using them aren't tempted to use them incorrectly.

If the macro ends up being used in such a way that the path gets built into the noarch package (for instance, it becomes the search path of an application in the noarch package) then it would be wrong. If the macro ends up being used only at build time (for instance, to configure an environment variable that sets a library search path while the package is running its unittests) then it is okay.

I'm taking a look at swt-charts now to see if I can figure out which case that particular package falls into.

Again, the problem is that if you rely on Toshio's "%{_eclipse_base}" for file pathing in a noarch package, the answer will not be applicable on all possible architectures, which is why that use is a bad idea.

What is wrong with using things like this?

{{{
%check
PYTHONPATH=%{buildroot}%{python_sitearch} python setup.py test
}}}

The definition of %python_sitearch is determined by the buildtime of the python package that is installed on the system (it's evaluated at the buildtime of this dependent package but it's evaluating to a value built into the python package that's installed on this particular system). The same sort of thing could be happening here too. I haven't finished looking at the swt-chart package yet to tell.

First pieces of looking:

x86_64 and x86 builds of swt-chart create packages that differ only in timestamps.

%{_eclipse_base} is being entered into the buildprocess via tycho.targetPlatform. The tycho code says that this is deprecated and shouldn't be used?

I've now built swt-chart without the use of tycho.targetPlatform (ie: changed these two lines:

{{{
- mvn-rpmbuild org.eclipse.tycho:tycho-pomgenerator-plugin:generate-poms -DgroupId=org.swtchart -Dtycho.targetPlatform=%{_eclipse_base}
+ mvn-rpmbuild org.eclipse.tycho:tycho-pomgenerator-plugin:generate-poms -DgroupId=org.swtchart #-Dtycho.targetPlatform=%{_eclipse_base}

  • mvn-rpmbuild -Dtycho.targetPlatform=%{_eclipse_base} install javadoc:aggregate
  • mvn-rpmbuild install javadoc:aggregate
    }}}

This build also seems to differ from the other builds only because of timestamps. So it seems that swt-chart isn't a valid use of %{_eclipse_base}.

At the moment, I don't know of any valid use of %{_eclipse_base} so at the moment I'd have to say that the macro is not included into the valid category case. If you have another example, you can bring it up.

FYI: we worked towards removing that macro and we were successful in many cases. But the basic problem is still here, and the eclipse_base macro seems to be the only item that can complete all other approaches.

Summarizing our scenario:

we need certain Eclipse arch specific jars to be present on a classpath of a noarch build of other packages.

That fits into a proper usage of the arch specific macro in a noarch package. We do understand its limitations and the reason why the original _libdir does not work in noarch build, and we will continue our work towards removing the need of that macro.

Thank's for the clarification!

It sounds like what you need is a helper binary that comes with all builds of Eclipse that you can run from the build env to answer the question of "where do the arch-specific jars live?"

e.g."eclipse-config --archdir" returns "/usr/lib64/eclipse/wherever"

Then, you can just macroize that call:

%global eclipse_archdir %(eclipse-config --archdir)

It will always be right, because it isn't derived from rpm macros. You should never use it for %files, but you can use it to add the arch specific path to the classpath at build time.

rgrunber pointed me to http://pkgs.fedoraproject.org/cgit/jfreechart.git/tree/jfreechart.spec, and its build command:

ant -f ant/build-swt.xml \
-Dswt.jar=%{_libdir}/java/swt.jar \
-Djcommon.jar=$(build-classpath jcommon) \
-Djfreechart.jar=lib/jfreechart-%{version}.jar

This is exactly the case I was talking about.

Thanks for the example!

This seems to work:

{{{
ant -f ant/build-swt.xml \
- -Dswt.jar=%{_libdir}/java/swt.jar \
+ -Dswt.jar=$(build-classpath swt) \
-Djcommon.jar=$(build-classpath jcommon) \
-Djfreechart.jar=lib/jfreechart-%{version}.jar
}}}

Built on x86 and x86_64 mock. No differences aside from timestamps in the generated files.

Additionally, it looks like the build-classpath command from jpackage-utils is exactly what you need for other cases like this?

(And of course, be careful with this -- just like spot's macroized utility and my macro... there's a danger that these paths will get written into a data file, compiled into a .class, or end up in the %files section. Any of these would render the package arch-specific instead of noarch and some are much harder to detect than others.)

Proposal approved: "Noarch packages must not use %{_libdir} (or any macro that contains the literal string "%{_libdir}" in its expansion), since the %{_libdir} macro has no useful value in noarch build environments." (+1:5, 0:0, -1:1)

Amendment approved:

"As an exception to this rule, noarch packages may use %configure (which uses %{_libdir} to set the --libdir value), but packagers should be very careful to ensure that the build/install logic does not actually use this value for anything."

(+1:5, 0:0, -1:1)

What about %{buildtime_libdir}? We discussed it before already [1]. Although for different language, the issue is exactly the same, i.e. populating load path during build.

[1] https://fedorahosted.org/rel-eng/ticket/5257

Spot was against it in that ticket and I was (at best) not sure that it was a good idea. You can propose it in a separate ticket, though, and the full FPC can vote on it.

Proposal: FPC declines to propose a general %buildtime_libdir macro that expands to the libdir of the host being built on.

Proposal to decline was approved (+1: 5, 0: 0, -1: 0)

Login to comment on this ticket.

cancel