#654 glibc file triggers
Closed: accepted 4 years ago Opened 5 years ago by tibbs.

Trying to get file triggers moving again, I bugged a couple of maintainers. spot listened and the texlive spec shrunk by over 110000 lines by switching to triggers, which is great. glibc, however, is proving to be more problematic.

https://bugzilla.redhat.com/show_bug.cgi?id=1380878 is the relevant ticket. Of the two real options, %filetriggerin/postun has the issue that it calls ldconfig rather often and for things which aren't actually libraries, or if you work to exclude those calls then you are calling the shell and grep "often". I wasn't approaching this from a performance standpoint, but instead one of package simplicity, so I haven't put a lot of thought into a question of performance. Perhaps someone else can have a think about it.

If you go for %transfiletriggerin/postun and attempt to only call ldconfig at the end of the transaction, you have to ensure that packages actually include any symlinks which ldconfig would create in order for the libraries to actually be "there" at the end of the transaction. This makes me think that perhaps saving a couple of scriptlets would actually cost more lines and packager effort/overhead/confusion in creating these symlinks and getting them into the %files list.

There's also a claim that packages would have to be forbidden from installing anything in /etc/ld.so.conf.d, which I don't believe to be true. That should be easily handled by a %filetriggerin/poston on that directory calling ldconfig, or simply requiring the scriptlets in that instance.

There's also the question of making sure any transition we might make is actually completed. I think FPC has no issue with trying to make sure that happens; the only issue is manpower to actually do it.

Anyway, something to consider. I know that at least Mageia uses the %transfiletriggerin/postun stuff, but perhaps they have additional guidelines relating to packaging of libraries. I'm not really all that familiar with what they do.

If you go for %transfiletriggerin/postun and attempt to only call ldconfig at the end of the transaction, you have to ensure that packages actually include any symlinks which ldconfig would create in order for the libraries to actually be "there" at the end of the transaction

As far as I know this isn't a problem, all packages come with both the realname and soname so all ldconfig is doing is updating the cache file.

I've also heard that replacing ldconfig scriptlets with /bin/true in the installer doesn't break anything.

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2016-10-06/fpc.2016-10-06-16.00.txt):

Metadata Update from @james:
- Issue assigned to tibbs

5 years ago

Metadata Update from @tibbs:
- Issue close_status updated to: None
- Issue tagged with: committee

5 years ago

There's been a little action in bugzilla. Basically in order for this to progress (i.e. for the glibc folks to even consider switching to scriptlets) we need to amend the packaging guidelines to require that packages include all of the relevant DSO symlinks instead of expecting ldconfig to create them.

Then work with RPM maintainers is required to get something in place for unpackaged DSO symlinks to fail builds. They won't tolerate the idea that an unpackaged symlink and no immediate post install ldconfig call leads to basically random behavior. I think all that's required is that ldconfig is called in %_os_install_post (probably via a script with some provided way to disable it if necessary). This would just result in an error about unpackaged files if the DSO symlinks aren't packaged.

Then I think once queries for problematic packages in symboldb show nothing, they can activate some triggers and we can start removing the scriptlets from packages.

There's the remaining question about packages which add things to /etc/ld.so.conf.d; for those, ldconfig needs to run immediately after the package is installed instead of at the end of the transaction. The obvious solution there is a separate trigger on that one directory, but for some reason I've proposed that three times and have been ignored.

In any case, this has been a bit frustrating, and I'm out of energy to push this forward. Basically there's a strong resistance to change on the glibc side and the expectation that we demonstrate zero possibility of fallout from this.

Metadata Update from @tibbs:
- Issue untagged with: committee
- Issue tagged with: meeting

4 years ago

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2017-08-17/fpc.2017-08-17-16.00.txt):

  • x654 glibc file triggers (geppetto, 16:10:23)
  • ldconfig -Nr %{buildroot} … seems like it should be the command to
    run. (geppetto, 16:26:28)

To make sure it's noticed by FPC as well, I did post a comment in the bug that basically amounted to:

  • This is being made way harder than it actually is
  • It seems like glibc folks won't actually implement it unless forced to

For what it's worth, the only difference between Mageia and Fedora in packaging (w.r.t. libraries) is that Mageia mandates all libraries to be split out into their own packages (like openSUSE does). From the point of view of the file triggers, this doesn't really matter.

For handling virtually all cases, this is all that should be necessary:

%filetriggerin -p /sbin/ldconfig -P 2000000 -- /etc/ld.so.conf.d
%filetriggerun -p /sbin/ldconfig -P 2000000 -- /etc/ld.so.conf.d
%transfiletriggerin -p /sbin/ldconfig -P 2000000 -- /lib /usr/lib /lib64 /usr/lib64
%transfiletriggerpostun -p /sbin/ldconfig -P 2000000 -- /lib /usr/lib /lib64 /usr/lib64

Seems this is progressing. From our end, the glibc folks want us to specify what the file triggers should look for a package which installs something into /etc/ld.so.conf.d that tells ldconfig to look at some location which is not under one of (/lib, /usr/lib, /lib64, /usr/lib64). Currently there are no packages in Fedora which do this so I hope it won't be a blocker, but I will open a separate ticket to cover the draft for that bit.

So it seems that a portion of this is now live in rawhide. The two transfiletriggers are there, but the triggers for ld.so.conf.d are not. So, joy and celebration that there has been progress.

So FPC now has work to do:

I'll add some draft changes here in this ticket in a bit.

Here's a quick go at what needs to change. This is, unfortunately, a lot of stuff. So for the most part we're removing stuff that packagers need to care about, but paradoxically this actually adds stuff to the guidelines. And the last section about ld.so.conf.d feels kind of tortured.

Shared Libraries

Whenever possible (and feasible), Fedora packages containing libraries SHOULD build them as shared libraries. See [[Packaging:Scriptlets#Shared_Libraries]] for information about whether scriptlets are required.

  • Then the relevant scriptlet section needs significant revision:

Shared Libraries

On Fedora 28 and newer, no scriptlets are required. However, the following scriptlets MAY be used, as they will simply evaluate to nothing on newer Fedora releases.

On Fedora 27 and older, ldconfig MUST be called properly in order to regenerate the dynamic linker's cache. If the package or subpackage has no existing %post or %postun scriptlets, simply include the %ldconfig_scriptlets macro on its own line before the %files list. As with other scriptlets, the -n option is used to apply the scriptlet to a subpackage.

# Install the program

%ldconfig_scriptlets -n libs

%files -n libs
license GPL

Using the %ldconfig_scriptlets macro will automatically generate a dependency on ldconfig where necessary. And the macro will do nothing at all on F28 or newer, so it can be used without

If the package or subpackage already has existing %post or %postun scriptlet, then use of %ldconfig_scrfiptlets will cause an error. You can use %ldconfig_post or %ldconfig_postun individually to generate a scriptlet which doesn't already conflict.

Within an existing %post or %postun scriptlet, use %{?ldconfig} on its own line to call ldconfig on those releases which need it. This will do nothing when not needed. When calling ldconfig in this way, you MUST also have the proper dependency on /sbin/ldconfig: Requires(post): /sbin/ldconfig and/or Requires(postun): /sbin/ldconfig as appropriate. (This will generate an unnecessary dependency on releases which do not strictly need it, but this does no harm.

  • An additional section in the scriptlets page is needed for ld.so.conf.d:

Linker Configuration Files

Packages which place linker configuration files in /etc/ld.so.config.d MUST call ldconfig in %post and %postun (on all Fedora releases) even if they install no actual libraries. They MUST NOT use the %ldconfig, %ldconfig_post, %ldconfig_postun or %ldconfig_scriptlets macros to do this, since these macros do not have any effect on Fedora 28 and newer. Instead simply call /sbin/ldconfig directly in both %post and %postun as well as adding the necessary dependencies when necessary:

%post -p /sbin/ldconfig
%postun -p /sbin/ldconfig

or, as part of existing %post or %postun scriptlets:

Requires(post): /sbin/ldconfig
Requires(postun): /sbin/ldconfig

In addition, in Fedora 28 or newer the following applies: If the configuration file added to /etc/ld.so.conf.d specifies a directory into which other other packages may install files, and that directory is not located in the directory hierarchy beneath one of /lib, /usr/lib</code, <code>/lib64 or /usr/lib64, then the package adding the configuration file MUST also include the following file triggers which cause ldconfig to be run automatically when necessary:

%transfiletriggerin -P 2000000 -- DIRECTORIES

%transfiletriggerpostun -P 2000000 -- DIRECTORIES

Replace DIRECTORIES with the space-separated list of directories which the package adds to to the library search path via the configuration files in /etc/ld.so.conf.d.

Metadata Update from @tibbs:
- Issue tagged with: hasdraft

4 years ago

@tibbs, remove -n in -n libs, because it means package named lib, not the subpackage. So it should be %ldconfig_scriptlets libs

Requires(post): /sbin/ldconfig

hmm, shouldn't we propose something like %{?ldconfig:Requires(post): %ldconfig}?


%transfiletriggerin -p /sbin/ldconfig -P 2000000 -- DIRECTORIES

Turned out this didn't work correctly because RPM did pass some extra stuff... So in glibc we switched to

%transfiletriggerin -p <lua> -P 2000000 -- DIRECTORIES

I was avoiding proposing %{?ldconfig:Requires(post): %ldconfig} because that's too much line noise to avoid one dependency which has zero effect. We could easily do %ldconfig_requires, I guess, but even then it's so terribly minor. I did mention that in the proposal but maybe %ldconfig_requires a smart way to avoid the questions (and it's a line shorter than being explicit.

I edited the file triggers bit. However, is it really necessary for packages other than glibc to use lua? There's no point in making it more complicated than necessary, so wouldn't this work?

%transfiletriggerin -P 2000000 -- DIRECTORIES

%transfiletriggerin -P 2000000 -- DIRECTORIES

This should work.

We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2018-02-01/fpc.2018-02-01-17.00.txt):

  • x654 glibc file triggers (geppetto, 17:07:34)
  • ACTION: Policy changes due to glibc file triggers from the ticket
    (+1:5, 0:0, -1:0) (geppetto, 17:15:03)

Well, I wrote this up. And then I realized that all of the macros are only in rawhide. But the updates have three karma apiece, so it shouldn't be too long. Rather than reverting, I'll just hold off on making an announcement for a bit.

Metadata Update from @tibbs:
- Issue untagged with: hasdraft, meeting
- Issue tagged with: announce

4 years ago

Note that there is build override for all of them.

I actually made the announcement a while back; not sure why I didn't close this.

Metadata Update from @tibbs:
- Issue untagged with: announce
- Issue close_status updated to: accepted
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.