java-1.8.0-openjdk, java-11-openjdk, java-17-openjdk and java-latest-openjdk packages will no longer build i686 subpackages
+1
-1 as-is. The change does not describe the impact on other packages and I feel like the change owner is unaware of the scope. This will potentially lead to an explosive recursive exponential ExcludeArch growth and we don't know if the impact-set won't leak into the multlib-set we actually still want to ship.
I imagine quite a lot of packages (see e.g. protobuf) will need to stop building their java subpackages on i686 which requires more complex changes to the spec file than adding one ExcludeArch line. The change owner does not appear to commit to fixing all this mess themselves. In fact, this kinda looks like a "drop it to rawhide it and go away" kind of change.
I am in favor of the general idea, I just don't feel it is well prepared. Let's:
Note: The other i686 change proposes to start removing the packages with the leaves. While there is a discussion on whether or not this will add unnecessary garbage to our spec files, it is safe -- distribution-wise. This change, however, proposes to rip off a non-leaf package and that can break the distro quite a lot (and what's worse, slowly -- we may be still dealing with the impact after 5 mass rebuilds, as the recursion level grows deeper and deeper).
-1, for now.
As far as I can tell, the change owners are underestimating the impact of this change. The page was updated to list all affected non-noarch Java packages, which would all need to be adapted manually. Additionally, all noarch Java packages will also need to be adapted (so, all Java packages).
Eek, I think I underestimated the impact here. I'll switch my vote to -1, given the above explanations.
Metadata Update from @sgallagh: - Issue tagged with: meeting
AGREED: FESCo awaits more feedback from the Change owners (+7, 0, -0) (sgallagh, 18:21:15)
Metadata Update from @sgallagh: - Issue untagged with: meeting
TY! Will bring it up in JDK team.
@jvanek: my comment from yesterday: Querying x86_64 gives an overly pessimistic view, because there are less packages for i686. And recursive query is not useful, because we don't want to drop all packages that require java. For some packages, we'll just disable the feature that requires java for i686, and the propagation stops there. For other packages, we might want to completely disable the package/subpackage on i686, and in that case the check needs to be recursed. But this needs to be checked for each package individually, to see what makes the most sense.
$ parallel repoquery -q --qf='%{name}' --arch=i686 --whatrequires {} ::: java-headless java java-devel csound-java DecodeIR link-grammar-java octave openmpi-java-devel plplot-java qdbm-java R-java-devel will-crash $ repoquery -q --qf='%{name}' --arch=src --repo=fedora-source --whatrequires java-headless R cocoalib enjarify $ repoquery -q --qf='%{name}' --arch=src --repo=fedora-source --whatrequires java $ repoquery -q --qf='%{name}' --arch=src --repo=fedora-source --whatrequires java-devel BareBonesBrowserLaunch CardManager OpenStego Singular ant antlr apron arduino-listSerialPortsC astyle automake beansbinding bigloo bolzplatz2006 brazil brltty cambozola capstone ceph collectd colossus console-image-viewer cortado cryptlib csound cvc4 diffoscope ditaa dl_poly erlang erlang-corba fernflower filedrop flute freecol freerouting freewrl frysk gdal gdl genders gnulib graphviz hdf hdf5 healpix hyperestraier imagej jFormatString jargs java-gnome java-mersenne-twister java-runtime-decompiler javahelp2 jblas jcip-annotations jcommon-serializer jcuber jdepend jericho-html jgrapht jigawatts jmol jorbis jpanoramamaker jpcap jpype jssc kernel-tools laf-plugin libbase libbluray libfonts libformula libgda libidn liblayout libloader libphidget libreoffice librepository libserializer libsvm libvirt-java libwebp link-grammar mapserver maven-verifier-plugin mecab-java miglayout mp mysql-connector-java naga octave opencv openjdk-asmtools openmpi openni pcl pentaho-libxml pentaho-reporting-flow-engine pl plexus-component-api plplot portmidi postgresql-jdbc ppl procyon proguard python-javabridge python-jep python-jnius qdbm rachota rsyntaxtextarea sblim-cim-client sblim-cim-client2 scalpel shaman sphinx swing-layout systemtap taggle test-interface tomcat tomcat-native tzdata vecmath1.2 vtk will-crash writer2latex xerces-j2 xjparse xmltool yecht z3
That last list is the longest, but it needs to be filtered: we only care about packages which do archful builds and don't already exclude i686.
parallel repoquery -q --qf='%{name}' --arch=i686 --whatrequires {} ::: java-headless java java-devel
unrelated tip:
repoquery -q --qf='%{name}' --arch=i686 --whatrequires java-headless --whatrequires java --whatrequires java-devel
But what repo are you querying? Just the multilib stuff in x86_64? Try the koji i686 repo instead.
Thanx a lot! Jsut note, that from those really only few build archfull (intersect with https://fedoraproject.org/wiki/Changes/Drop_i686_JDKs#list_of_non-noarch_java_packages.2C_transitive_to_chain_of_three )
On contrary, if they do they moreover 100% build also i686.
This is just 100 packages, where tehe safety bell is to drop i686. The transitive chain may be worse, but when I did that transitive check, it grown by 1 in the fourth level.
Thanx a lot. Will continue elaborating
One message from openjdk development: "java-11-openjdk & java-17-openjdk may still experience crashes, even as they stand now, and java-1.8.0-openjdk is still running as an interpreter-only (Zero) build, with performance implications."
parallel repoquery -q --qf='%{name}' --arch=i686 --whatrequires {} ::: java-headless java java-devel unrelated tip: repoquery -q --qf='%{name}' --arch=i686 --whatrequires java-headless --whatrequires java --whatrequires java-devel But what repo are you querying? Just the multilib stuff in x86_64? Try the koji i686 repo instead.
I think to make this work, you have to include --arch=src. So one have to run two queryes and interset... ut you guys know those tricks much better then me.
$ repoquery --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel ... $ repoquery -q --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel | wc -l 849 $ repoquery -q --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel | grep 'src$' | wc -l 140 $ repoquery -q --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel | grep 'noarch$' | wc -l 665 $ repoquery -q --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel | grep '86$' | wc -l 44 $ repoquery -q --repo=koji-{source,i386} --whatrequires java-headless --whatrequires java --whatrequires java-devel --recursive | wc -l 2133
Right, the recursive query is misleading, because it does not account for possibly dropped subpackges.
However, this also does not account for recursive build rerequires (a.src requires java, b.src requires a "binary" package built from a.src and so on).
Do we have sufficient feedback from the Change owners to begin the voting process?
My understanding was that we wait for https://pagure.io/releng/issue/10686#comment-786124
Hello! I'm afraid rcm is a bit reluctant to sign themselves to anything. It seems it is technically possible to stop building noarch pkgs on i686, but even if FESCO support that, they consider the solution as hacky. Also people around OpenJDK got little bit puzzled how first-glance simple change grown. From what I see, except dropping the proposal, there are jsut two courses to do: a) I will write a system wide change to stop building noarch on i686, but it is most likely bigger can of worms then "Drop i686 builds of jdk8,11,17 and latest (18) rpms from f37 onwards ", and I do not feel exactly enlightened enough in that topic keep discussion on reasonable levle b) to adapt java-packages guidelines that i686 must be in ExcludeArches or workarounded to not use java/javac (there are such cases) and to all current java packages inject exclude arch i686 or contact maintainer to do so, otherwise he will face random failures (if theirs builder will be by bad luck i686)
For the (b), I'm not sleeping. I'm investigating what will go wrong if al java packages will suddenly case to exists on i686. It run over weekend, and I made two major mistakes: I forget to record the edges of the char, recorded just nodes, it ended in level4 of dependencies of dependencies of dependencies of dependencies various jdks and mavens... Second I forger to exlude noarchs from intermediate reuslts (as thy will not stop working at all, will jsut not build on i686) so there ended to much pkg (some 30000) from which about 4000 are built on i686. As there ended moreover whole perl stack , I need to rerun my queries, and track that chart better, who is the main culprit.
Any opinion on path a) xor b) is welcomed, any other possible ways to! Note that there is also https://pagure.io/releng/issue/10686#comment-787222 now, but I do not want to bother RCM yet again.
Ok... my algorithm was right.. I can not exclude noarch from the parents set...
Hmm.. the culprint pulling whole system in is:
java-devel~is~req~by~automake
Will exclude this one
hm..another culprit is graphviz.. but will leave this round to finish already.
automake is noarch... that is baaad...
another noarch which is pulling half of the system is autoconf
Hello! I'm afraid that automake and autoconf build-time depndece is a showstopepr to this. Any ideas how to continue? if you wish, I can generate you the chart, but it is currently hige (14k nodes..s.. msotly caused by autoconf.. next generaraion would be without it, but not sre it is worthy.
Something like this is exactly what I was afraid of. I don't think it will be feasible to work around all (direct and indirect) Java dependencies in Fedora (which is also a much bigger scope than the original proposal) within the original timeframe. So what do you think about these steps instead:
Yup. I moreover agree. I already contacted maintainers of autoconf and automake, but I do not belive there will be some immediate action possible, due to unforseen consequences.
I keep thinking. I had included also build-requires, which may be missleading. If I exclude BR on noarch, then the set will be much smaller (will rerun). Because if the BR of java is of is noarch pkg, it do not need to build on i686, and if there is no runtime dep (like autoconf/automake) on java then the whole sub-tree may disappear. wdyt?
Such noarch packages will still need to be modified to add ExclusiveArch: %{java_arches} so that they don't get built on i686 builders.
ExclusiveArch: %{java_arches}
Well, yes... It is not an technical issue. But I must admit that I do not know where and how to let everybody know - some kind of packaging gudeline line I guess?
Actually - ExcludeArch: %{non_java_arches} may be better, as any major arch shoudl be always supported by java.
ExcludeArch: %{non_java_arches}
I'm not sure why we keep updating this ticket with questions about the details of this Change. Shouldn't this happen on the mailing list instead?
Anyhoo, there are already Packaging Guidelines for this kind of scenario ("noarch packages with unported dependencies"), and applying them would look like this:
BuildArch: noarch ExclusiveArch: %{java_arches} noarch
c.f. https://docs.fedoraproject.org/en-US/packaging-guidelines/#_noarch_with_unported_dependencies
If we ever add a new architecture, something like riscv64 maybe, will it support Java from the start? I doubt it. And this double-negative exclusion construct would also break for any third-party package builds for no-longer-supported-by-Fedora architectures.
riscv64
I suggest changing the status of this change to InProgress and restarting the change process if/when a new proposal is ready.
Yes please, am changing to inProgress.
also sorry for misusing this thread, I di not know t is wrong.
Am still in work of best enumeration. Currently I think it is safe to exclude everyhitng what have BuildRequires(just BR!) on some java stack and builds as noarch. As such tool can happily ignore i686 iin built time, and still wil be availabel in runtime if needed. This is exactly the case autoconf and automake.
Not sure if it needs to be announced since the proposal was withdrawn, but tagging for that just in case. I'll open a new issue when/if the proposal is re-submitted.
Metadata Update from @bcotton: - Issue tagged with: pending announcement
Hello! I had reworked the proposal's workflow and moved back to ChangeReadyForFesco
Metadata Update from @zbyszek: - Issue untagged with: pending announcement
The new proposal is reasonable.
I think it'd make sense to start small with a subset of packages which have hundreds of dependencies. I.e. add the %java_arches macro, then handle graphviz, meson, and a few other important packages, and only then start filing bugs for what remains. Initially there'll be a lot a lot of packages in the list, but once those packages low in the stack are fixed, the number should go down significantly.
%java_arches
graphviz
meson
I'm sure that some details will need to be figured out, but I think the whole plan is worth attempting.
--
BTW. I looked at graphviz: it requires java for graphviz-java subpackage. But nothing seems to Require or BuildRequire graphviz-java. So a simple archful exclude in graphviz should be enough in this case.
graphviz-java
PoC: https://src.fedoraproject.org/rpms/graphviz/pull-request/9
Can we/should we get the reworked proposal posted back to the devel list for input from the community?
I mailed devel on the old thread to relook at this revised change.
Hi! It seems there was no reply at all:(
What now? To get the trackig BZ, to commit new macro and to start spamming the main packages I need your consensus.
Tagged with meeting, so it won't get lost.
Metadata Update from @churchyard: - Issue tagged with: meeting
+1 from me
But be prepared that the new plan will take time and might not make it in time for Fedora 37 release.
Yup. Am aware. Thanx a lot!
Looks good to me now, thank you for revising the proposal. +1
The only question I have is what
all noarch direct java, amven ant... dependencies will (todo, generate list as for java-17-openjdk as system jdk) get auto injected ExclusiveArch: %{java_arches}
is supposed to mean. Will there be an automated / scripted approach to file PRs to add ExclusiveArch: %{java_arches} to affected packages?
Yup, it will require some learning and investigations from me, but at the end there should be automation which will inject it, and I will use provenpackager powers to commit, push and build (so no PR planned).
Thanx a lot for review!
Metadata Update from @churchyard: - Issue untagged with: meeting
Thanx! Can I get the tracking BZ please?
Wait a bit, I'm gonna count the votes tomorrow (a week after your comment).
APPROVED (+3, 0, -0)
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/UI4SLUMPSWU2YEMRS4VGF7DMDV4TFJQC/
Closing this ticket.
@jvanek The change s now approved and you are free to proceed, @bcotton will open a tracking bug shortly.
Metadata Update from @churchyard: - Issue close_status updated to: Accepted - Issue status updated to: Closed (was: Open)
tyvm!
@churchyard Why was this change approved even though the corresponding releng ticket was resolved as Can't Fix? Thanks.
What releng ticket?
Sorry, pasted the wrong link. It should have been this:
Well, they asked releng for something that's not possible to do in koji. The final solution was to add ExclusiveArch: %{java_arches} noarch to all noarch java packages.
ExclusiveArch: %{java_arches} noarch
noarch
The Java packaging guidelines will be updated to reflect the fact that this ExclusiveArch will become mandatory for Java packages.
ExclusiveArch
@decathorpe What about other noarch packages that depend on java-devel, for example for building a subpackage or running the test suite? What should those packages do? Could you add guidelines as well?
java-devel
@fweimer what do you mean?
BuildArch: noarch
And the case where you're handling possibly overlapping sets of supported architectures, you'd have to deal with those yourself (for example, determine the intersection between %{node_arches} and %{java_arches} for a fictional package that requires both the NodeJS runtime and a JRE, which might or might not be the same on all Fedora branches). So, same as always when dealing with two (or more) hopefully-overlapping sets of supported architectures.
%{node_arches}
%{java_arches}
Any packages that need to do more complicated things than that (i.e. only build certain sub-packages on certain architectures), you've always been on your own. The Java Packaging Guidelines aren't the place to document advanced RPM packaging topics like that :)
for reference: https://pagure.io/packaging-committee/pull-request/1187
@decathorpe
packages that are all-noarch add ExclusiveArch: %{java_arches} noarch in addition to the already existing BuildArch: noarch
What is the impact on the presence of the package in the repositories with this ? Specifically in this case, the resulting package will be excluded from i686 repositories, right ?
No, "noarch packages with unported dependencies" (that's how they're called in the Packaging Guidelines) are copied into all arch-specific repositories. This is - apparently - a shortcoming of how RPM metadata works, as pungi doesn't have the information that's necessary to exclude those noarch packages from being added to repos for certain architectures.
My last koji bug report about this problem was closed here: https://pagure.io/koji/issue/1843
And according to this closed issue, this is not solvable programmatically, because RPMs don't contain Exclusive/ExcludeArch metadata, only SRPMs do - and pungi operates on RPMs: https://pagure.io/pungi/issue/1518
This leads to packages with broken dependencies (and broken packages peroid) being included in repositories for architectures where they should be excluded. meh
Just to note: we no longer produce i686 repos, only a koji buildroot.
Right, but it also affects the koji buildroot, and other packages on architectures for which we do publish repos.
At least RPM got an explicit fix to make the "ExclusiveArch: noarch" idiom work: https://bugzilla.redhat.com/show_bug.cgi?id=1298668
This sounds like a safety belt to me, as even in case of multilib system, the i686 pkgs are (copied) in the 86_64 repo
Login to comment on this ticket.