#1443 ImageMagick maintainer insists on building with -march=corei7
Closed None Opened 4 years ago by kalev.

ImageMagick in rawhide is built with -march=corei7, which makes it crash on CPUs older than the Core i7. I am failing to find a common language with its maintainer who insists this is not a bug.

Please help mediate the situation. It would be good if this got resolved before the F23 mass rebuild, because many packages use /usr/bin/convert (it's part of ImageMagick) at build time to convert icons from one format to another.

References:


Yeah; https://fedoraproject.org/wiki/Packaging:Guidelines#Compiler_flags applies:

Compilers used to build packages must honor the applicable compiler flags set in the system rpm configuration. Honoring means that the contents of that variable is used as the basis of the flags actually used by the compiler during the package build.

For C, C++, and Fortran code, the %{optflags} macro contains these flags. Overriding these flags for performance optimizations (for instance, -O3 instead of -O2) is generally discouraged.

According to commit log: "GCC updated, problem should be gone." maintainer seems to think it was a gcc problem that is gone now. I think that's a misunderstanding, and incorrect.

Did you tell the maintainer that? If not, I suggest you do.

Hi.
I do not insist so!
Problem with compilation in rawhide indeed solved and it succeeded without that option as you all see.

Real problem should be solved (soon) upstream: http://www.imagemagick.org/discourse-server/viewtopic.php?p=122230#p122230

What I only was insist do not reopen bug "fails to build" with "incorrect build options" and create new one.

Any other considerations and help very appreciated.

And also, %{optflags} contains -mtune=generic:

{{{
optflags : %{__global_cflags} -m64 -mtune=generic
__global_cflags -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches %{_hardened_cflags}
_hardened_cflags %{?_hardened_build:%{_hardening_cflags}}
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
}}}

but not -march, is not?

The latest rawhide build does not in fact seem to have this fixed. From the x86_64 build log:

CFLAGS          = -I/usr/include/freetype2 -I/usr/include/libpng16 -fopenmp -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -march=corei7 -fexceptions -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

note the '-march=corei7' ?

I must confess that the summary and initial comment on bug 1213838 [1] is confusing. Koji doesn't show me any failed rawhide builds in the recent past and the suggestion to use --with-gcc-arch looks=x86_64 wrong to me [3].

Having said that, Kalev's fix looks correct to me. We shouldn't be building with -march.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1213828
[2] http://koji.fedoraproject.org/koji/packageinfo?packageID=425
[3] I can only guess what --with-gcc-arch looks=x86_64 means because I couldn't find the option in any of the gcc man pages that I tried. I guess I just have an old gcc.

The issue is not ImageMagik failing to build, the issue is that we have different types of x86_64 machines in koji. if ImageMagik builds on the newer builders the resulting binaries do not run on the older builders. a lot of packages use /usr/bin/convert in the build processes to do different tasks. if convert is optimised for corei7 it segfaults when run on older hardware. It is a very real issue and the maintainer needs to take a step to ensure that the programs in ImageMagic are able to be run on any supported x86_64 cpu.

Replying to [comment:4 hubbitus]:

And also, %{optflags} contains -mtune=generic:
but not -march, is not?

Fair enough, perhaps we don’t have explicit wording to the effect that distro-chosen -march should not be overridden. In any case, what CPU families are supported by built packages needs to be an ''explicit'', ''distribution-wide'' decision, not something that packages should disagree about, nor something randomly chosen based on what CPU the current build system (and the specific machine within the build system) happens to use today.

We need to tell our users a single set of hardware requirements which hold uniformly across all packages (except for clearly hardware-specific software like drivers).

Replying to [comment:8 mitr]:

Replying to [comment:4 hubbitus]:

And also, %{optflags} contains -mtune=generic:
but not -march, is not?

Fair enough, perhaps we don’t have explicit wording to the effect that distro-chosen -march should not be overridden. In any case, what CPU families are supported by built packages needs to be an ''explicit'', ''distribution-wide'' decision, not something that packages should disagree about, nor something randomly chosen based on what CPU the current build system (and the specific machine within the build system) happens to use today.

Thanks. So it is not guidelines violation. May be they should be updated (or macroses) if we really want force to do not use -march or detection in packages (it looks reasonable for me).

Meantime I would prefer it fixed upstream for ImageMagick in particular. What are you think?

Replying to [comment:9 hubbitus]:

Thanks. So it is not guidelines violation. May be they should be updated (or macroses) if we really want force to do not use -march or detection in packages (it looks reasonable for me).

It's likely not a packaging guideline because it's supposed to be common sense. The CPU family falls out of FESCo decisions about what the minimum supported architecture is. Packages can't go overriding that without breaking the architecture support that is set for fedora as a whole.

You could ask the Packaging Committee to add a guideline along these lines and I'm sure that they'd add one... They have to add other guidelines that it seems everyone should understand implicitly but don't.

Meantime I would prefer it fixed upstream for ImageMagick in particular. What are you think?

This needs to be fixed in the package downstream. Fedora makes a claim that it is available for certain minimum architectures. Having a package that compiles using instructions that won't run on that minimum architecture break that claim. Therefore we have to fix it downstream. Buildtime options really are the domain of the package maintainer even if the upstream is willing to modify the build scripts they provide to make the downstream builders (which includes package maintainers) lives' easier in the future.

Replying to [comment:2 rdieter]:

According to commit log: "GCC updated, problem should be gone." maintainer seems to think it was a gcc problem that is gone now. I think that's a misunderstanding, and incorrect.

Did you tell the maintainer that? If not, I suggest you do.

Replying to [comment:7 ausil]:

The issue is not ImageMagik failing to build, the issue is that we have different types of x86_64 machines in koji. if ImageMagik builds on the newer builders the resulting binaries do not run on the older builders.

It is not really exact that case. When I start investigation (by https://bugzilla.redhat.com/show_bug.cgi?id=1214344) problem was reproduced on rawhide-test machine (https://fedoraproject.org/wiki/Test_Machine_Resources_For_Package_Maintainers). Convert segfaulted. Problem with backtrace and assembly was reported upstream: http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=27449&p=122250. Note GCC version was 5.0.1 20150417. Furthermore, rebuild of IM itself also was failed.
Now it gcc version 5.1.1 20150422 (Red Hat 5.1.1-1) (GCC). As I did not found real bug at that time I do '''not''' insist it was gcc bug or not (I have no snapshots off course and also versions of other components on that point of time), but now on the '''same machine''' with same options, same version built fine.

Meantime, I realize it fixed upstream (scratch build): http://koji.fedoraproject.org/koji/taskinfo?taskID=9822167 . So, if noone argue I'll push it in rawhide.

Meantime, I realize it fixed upstream (scratch build): http://koji.fedoraproject.org/koji/taskinfo?taskID=9822167 . So, if noone argue I'll push it in rawhide.

Thanks, please do. I looked at the log for the scratch build and it seems upstream has changed it so that it now uses -mtune instead of -march by default:

CFLAGS          = -I/usr/include/freetype2 -I/usr/include/libpng16 -fopenmp -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -mtune=nehalem -fexceptions -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

The -march option is gone and it now has -mtune=nehalem instead. It's still not perfect because it overrides -mtune=generic from Fedora's cflags, but nevertheless a big improvement. It should at least no longer generate code that sigill's on older processors.

If possible, I would also strongly suggest doing something to get rid of -mtune=nehalem. I would guess you should be able to pass --without-gcc-arch to configure to that effect.

Thanks!

Is there anything more to do here? Or is it now in hand?

Replying to [comment:12 kalev]:

Meantime, I realize it fixed upstream (scratch build): http://koji.fedoraproject.org/koji/taskinfo?taskID=9822167 . So, if noone argue I'll push it in rawhide.

Thanks, please do.
Done.

I looked at the log for the scratch build and it seems upstream has changed it so that it now uses -mtune instead of -march by default:

CFLAGS          = -I/usr/include/freetype2 -I/usr/include/libpng16 -fopenmp -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -mtune=nehalem -fexceptions -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16

The -march option is gone and it now has -mtune=nehalem instead. It's still not perfect because it overrides -mtune=generic from Fedora's cflags, but nevertheless a big improvement. It should at least no longer generate code that sigill's on older processors.

If possible, I would also strongly suggest doing something to get rid of -mtune=nehalem. I would guess you should be able to pass --without-gcc-arch to configure to that effect.

Thanks!

Ok --without-gcc-arch re-added.
http://koji.fedoraproject.org/koji/taskinfo?taskID=9854579

Kalev thank you very much for effort. Please note, first revert that option in bugzilla ticket has no other intention to demonstrate it is not real reason of fix fail to build as I provide details before. You do not answer several days on question why it needed and I made build without it.

Thanks again.

Thanks! The fixes look good to me.

Login to comment on this ticket.

Metadata