#369 Guidance on dealing with the bundled libev in perl-EV
Closed: Fixed None Opened 5 years ago by bochecha.

perl-EV upstream is also the upstream of libev.

When I asked them about building perl-EV for Fedora against the system libev, they explained that they bundled the libev sources in perl-EV because it is ABI incompatible with the system libev, even though the sources are identical.

Upstream suggested getting a libev-source subpackage added to libev, and using that when as a build requirement of perl-EV. At the time, they made the point that this was no different from headers-only libraries which are common in C++ (e.g asio).

Having seen a precedent for other *-source packages in Fedora (e.g tigervnc has a build requirement on xorg-x11-server-source), I assumed it was a correct way of dealing with this.

[https://lists.fedoraproject.org/pipermail/devel/2013-November/192225.html Michael Schwendt expressed his disagreement] with my assumption on devel@ in [https://lists.fedoraproject.org/pipermail/devel/2013-November/192197.html a thread I had opened for unrelated packaging changes in libev].

So, what would be the proper way to resolve this?


I just remembered it, there is also rxvt-unicode in Fedora which uses libev-source for building, as they also bundle libev, just like perl-EV.

Replying to [ticket:369 bochecha]:

When I asked them about building perl-EV for Fedora against the system libev, they explained that they bundled the libev sources in perl-EV because it is ABI incompatible with the system libev, even though the sources are identical.

Could you elaborate this? What exactly is different?

When I asked (note, that was a while ago, back when I was trying to get perl-EV through the review request), upstream had answered this:

The watcher structures differ in layout and the configuration will
certainly differ as well (it will also use 64 bit stat structs, or
whatever perl uses, while the standard libev uses 32 bit stat structs, as
that is what fedora has as default, afaicr

From the last comment it sounds like some of this may be the way we're building libev in Fedora? Can this be resolved by building the system libev differently?

Can this be resolved by building the system libev differently?

I don't know, I guess we could build libev exactly the way perl-EV needs it (and then carry a big downstream patch to make perl-EV use the system libev), maybe...

But then the problem could arise from another package needing libev built in the way we currently build it? Or in yet a different way?

I'm not entirely sure how this works [1], what I know is:

  • upstream libev intends for it to be bundled, and considers it similar to how some C++ libraries are headers-only [2]

  • upstream libev is also the upstream of perl-EV (and it seems rxvt-unicode), and they have no interest in unbundling anything

  • the solution I used seems to be perfectly fine for other packages, for example xorg-x11-server-source / tigervnc, so I'm really wondering why it wouldn't be for libev-source / perl-EV.


[1] I'm not the original packager of libev, I was just a co-maintainer as I needed it for perl-EV, and one day I realized (to my surprise) I had become the package owner, seemingly without any notification.

[2] Such libraries get effectively "bundled" (#include-ed) at build time (just like perl-EV uses the libev-source package) and if the library received an update (say, fixing a security issue), then all using packages would need to be rebuilt.

Which makes me wonder, what is our answer to this kind of libraries? (that might or might not be the same answer as for libev)

Another data point, mesos needs libev to be built in a special way:
https://bugzilla.redhat.com/show_bug.cgi?id=1049554

At a quick glance, this is not the way perl-EV expects libev to be built.

As mentioned by tstclair in the bug report I linked to above, could libev be considered a copylib?

https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Copylibs

Given that the authors actually recommend bundling, and the runtime is not very flexible, I'm inclined to consider this a copylib.

ref:

<tstclair> schmorp, so if they wanted to just set EV_CHILD_ENABLE=0 they could just pass CFLAGS=-DEV_CHILD_ENABLE=0 through.

<schmorp> tstclair: yes, or use a wrapper

<tstclair> schmorp, wrapper?

<schmorp> tstclair: for example, urxvt has a ev_cpp.h file which defines stuff and includes
ev++.h

<schmorp> tstclair: and a ev_cpp.C file, which defines a bunch and includes ev.c

<schmorp> tstclair: urxvt then uses ev_cpp.h as header, and ev_cpp.C as implementation, ignoring the other libev files

<schmorp> tstclair: (while compiling)

So, first of all, as much as I would have loved to participate to the FPC meeting debating this, living in a UTC+8 timezone makes this pretty hard. ;)

It would have been nice to update this ticket with the conclusions, though, but as Remi told me it has been discussed a week ago, I went over the meeting logs to see what had been decided.


{{{
<abadger1999> Proposal: Allow libev build -source package and packages which bundle libev presently need to switch to using the code from the -source package instead. Follow the static library guidelines for making sure that the packages which use -source stay up to date with libev changes.
...
<abadger1999> #info Proposal to allow libev-source (with conditions -- see summary in ticket) approved. (+1:6, 0:0, -1:0)
}}}

So, having a have a look at the [https://fedoraproject.org/wiki/Packaging:Guidelines#Packaging_Static_Libraries static library guidelines], I'm not quite clear what I need to do. It seems that the guidelines are already respected, except with {{{s/-static/-source/g}}}. What am I missing? :)

As for moving all the packages bundling libev to using {{{libev-source}}}, I don't mind trying to identify them and report bugs / send patches (not being a provenpackager, I won't be able to commit/push them and rebuild the packages myself).

However, how to identify such packages? The only thing I can think of is running a {{{tar tf}}} on all the source tarballs on the lookaside cache and {{{grep}}}-ing for "libev". :-/


{{{
<abadger1999> Rathann: k. To accomodate your vote, I'll ask the package maintainer to post why upstream doesn't want to merge changes to make things runtime settable instead of buildtime settable.
<abadger1999> Rathann: (Which could be... "I just don't want to") Is that acceptable to you?
<rathann> yes
}}}

I have no idea whether such changes have ever been proposed.

Should I ask them on their mailing-list to provide an explicit statement about this?

What if they say they would accept such changes, but won't do it themselves? Will libev (and the packages using it) have to be removed from Fedora until such a time where a kind soul submits those changes?


{{{
<racor> abadger1999: I am asking myself, why the Fedora maintainer is so reluctant on change the code himself.
<abadger1999> racor: It would be a fork. I don't think all the places where libev is consumed have been evaluated to see what would need to be made runtime settable are. I think the package maintainer is only interested in the apps using libev; libev itself is code that he isn't familiar with.
}}}

... Exactly.

In fact, I was only ever interested in perl-EV. As I said in [https://fedorahosted.org/fpc/ticket/369#comment:5 comment 5], I was only a co-maintainer of libev (I needed it for perl-EV, so it made sense), and one day I realized I had become the primary owner (I don't remember ever getting a notice about this, it seems to have happened completely automatically).

I really never had any interest in libev, I just somehow inherited from it.

Now, I'm willing to work on this issue to get libev (and packages using it, especially perl-EV) in a better state that conforms to the Fedora Packaging Guidelines, but I have certainly no interest in forking libev and maintaining it myself in the future.

{{{
<racor> abadger1999: That's the normal job as package maintainer :)
}}}

I welcome you (or anyone else really) to take over libev, fork it, and fix it in any way you want. :)


Reading the meeting logs, I have to wonder why libev is causing so much trouble, while the two examples I noted in this thread don't:

  • tigervnc has a build requirement on xorg-x11-server-source, which seems to be exactly the same situation as we are discussing here
  • some C++ libraries have all their code in the headers, and as such cause a similar situation: an update to the library (e.g for a security fix) implies a rebuild of all the packages using them

Okay, sorry I haven't had time to post here what the FPC has decided. To make up for it, it is good news.

The FPC had a variety of things that they wanted to post informationally before I get to the decision:

=== About copylibs ===

There's several ways that libraries can be handled in Fedora. Each of those methods falls into a hierarchy of most acceptable to least acceptable. If there's several options available we want to do the one that's most acceptable. Dynamic linking is the best option. Following this is static linking. Using a source package comes next (it's mostly equivalent to static linking but non-standard). Bundling comes at the bottom of the list. So we would not approve copylibs since the source package approach would work here.

Additionally, copylibs and the other possible rationale listed on the no bundled libraries page are not sufficient to get a bundled library exception. They are merely approaches that can be used to help justify an exception for FPC in the past. Copylibs in particular are problematic because they do not address any of the concerns that the FPC normally has with bundling libraries (in fact, the original copylib exception was made by fesco rather than FPC when fesco was in charge of approving bundling). Many times, copylibs are approved with additional conditions that address some of the shortcomings of bundling (for instance, the copylibs which are only approved when the copylib comes from the same upstream as the application that bundles them).

=== Due diligence ===

FPC would like to find out if (and hopefully why) upstream is unwilling to make the current compile-time options something that can be done at runtime (perhaps via additional API). If you could ask upstream and post the response here that would be great.

=== Final decision ===

However, FPC decided that if upstream remains unwilling to do that, we can accept using the -source package to do this. So please do post the result of talking to upstream, but you can go with the -source route if upstream still refuses.

info Proposal to allow libev-source (with conditions -- see summary in ticket) approved. (+1:6, 0:0, -1:0)

Thanks for your patience!

Thanks Toshio for writing this down.

I just sent an email to upstream, let's wait for their answer.

Finally, I'd like to reiterate my questions from the comment above.

{{{
<abadger1999> Proposal: Allow libev build -source package and packages which bundle libev presently need to switch to using the code from the -source package instead. Follow the static library guidelines for making sure that the packages which use -source stay up to date with libev changes.
...
<abadger1999> #info Proposal to allow libev-source (with conditions -- see summary in ticket) approved. (+1:6, 0:0, -1:0)
}}}

So, having a have a look at the [https://fedoraproject.org/wiki/Packaging:Guidelines#Packaging_Static_Libraries static library guidelines], I'm not quite clear what I need to do. It seems that the guidelines are already respected, except with {{{s/-static/-source/g}}}. What am I missing? :)

As for moving all the packages bundling libev to using {{{libev-source}}}, I don't mind trying to identify them and report bugs / send patches (not being a provenpackager, I won't be able to commit/push them and rebuild the packages myself).

However, how to identify such packages? The only thing I can think of is running a {{{tar tf}}} on all the source tarballs on the lookaside cache and {{{grep}}}-ing for "libev". :-/

So, I asked upstream, here's a copy of the question and answer, for the record.

However, the FPC asked me to ask you whether you would be willing to
make the current compile-time options something that can be done at
runtime (perhaps via additional API), and if not, why?

The simple answer is that most of the configurability can't be done at
runtime. For example, you can't optimise for size or change the ABI at
runtime.

In terms of implementation, it sounds like you're doing it correctly (static library guidelines with s/-static/-source/ ). My interpretation of the FPC vote is that we approved the strategy you've taken for this package. (If there's FPC members that disagree with that, please speak up.)

Identifying such packages: yep, that's the only way I could see doing it too.

Login to comment on this ticket.

Metadata