The abipkgdiff1 program is a new tool that compares the Application Binary Interfaces of two versions of C/C++ ELF binaries carried by two RPM packages. It does so by analysing the content the ELF binaries themselves, including their debug information. It can pinpoint ABI changes, categorize them into harmless and potentially harmful (i.e ABI incompatible) changes, and report about the potentially harmful changes in a way that makes it easy for programmers to understand which part of the source code change induced the ABI change.
The libabigail Fedora package is available in EPEL{6,7}, Fedora{21-23} and Rawhide and is actively maintained.
At the moment, the Fedora Packaging Guidelines propose to use the abi-compliance-checker program to detect ABI changes.
I believe that abipkgdiff is easier to use than abi-compliance-checker because the former only needs the packages to compare, whereas the later needs the user to unpack the packages, and go through other hops to perform the comparison. You can compare what's needed to use the two tools by yourself:
https://fedoraproject.org/wiki/How_to_check_for_ABI_changes_with_abipkgdiff
and
https://fedoraproject.org/wiki/How_to_check_for_ABI_changes_with_abi_compliance_checker
I think that abipkgdiff performs a finer grain type analysis than abi-compliance-checker and so is capable of detecting more subtle incompatible ABI changes.
I also think that abipkgdiff produces less false positives (in general) than abi-compliance-checker. And, in case there are, users can specify "suppression specifications"2 to silence specific change reports in a quite powerful way. I don't think abi-compliance-checker has that capacity.
Last but not least, abipkgdiff is built using the reusable libabigail library3 that provides foundations to extract ABI-relevant information from (or rather, type information really) from C/C++ binaries (and their debug info), manipulate that information to perform interesting task such as comparing ABIs (or types information in general) and reporting about the spotted differences in a controlled manner. So users can write their own tools using this library. Note that the 'abidiff' tool is written using the same library.
So I think we would better serve our maintainers (and indirectly our users) by recommending abipkgdiff (and abidiff to compare binaries), rather than abi-compliance-checker.
I have amended the packaging guidelines to show the modifications I am proposing. The resulting draft is accessible at https://fedoraproject.org/wiki/User:Dodji/Packaging:GuidelinesAmendedForABIChecking.
The exact changes I am proposing can be browsed at https://fedoraproject.org/w/index.php?title=User%3ADodji%2FPackaging%3AGuidelinesAmendedForABIChecking&diff=428250&oldid=428239.
+1 (since I won't be at the meeting)
I think, it's too early to recommend libabigail.
In particular, I can not spot any man-pages, examples/use-cases.
Technically, I am having doubts libabigail is superior to abi-compliance-checker, because abi-compliance-checker also APIs. Something at least I can not find a counter part for in libabigail.
Replying to [comment:3 corsepiu]:
I can not spot any man-pages, examples/use-cases.
The manuals for all the commands are available online at https://sourceware.org/libabigail/manual/libabigail-tools.html. In particular the manuals for abidiff is at https://sourceware.org/libabigail/manual/abidiff.html and the one of abipkgdiff is at https://sourceware.org/libabigail/manual/abipkgdiff.html.
The manual for abidiff comes with extensive (IMHO) examples and use cases at https://sourceware.org/libabigail/manual/abidiff.html#usage-examples.
Note that the Fedora packages come with local version of these manual, in the html and info formats.
As for the library itself, it also has extensive API documentation in which almost all the public entry points of the library are documented at https://sourceware.org/libabigail/apidoc.
There is even a PDF documentation which talks more about the different classes of use cases (from upstream projects to downstream project) at http://dodji.seketeli.net/talks/gnu-cauldron-2015/libabigail-state-of-the-onion.pdf.
There can always be more documentation, but in all fairness I don't think Libabigail's weak spot is documentation at this point. Or if it is, I'd welcome specific points to support your statement.
Libabigail based tools don't look at header files, if that is what you are saying. That is true. But there is a significant part of the API that is reflected in the debug information of the binary. And that part is taken care of by libabigail just fine, by looking deeply into what the debug information gives us. And if there are stuff missing from debug information, well, we can add it (if it's really necessary) because we are compiler people. Just to give you an idea of what I am saying, think about the fact that debuggers don't need to see header files either, and yet they can understand the whole program.
The reason why Libabigail doesn't look at header files is that we think it's a dead end, in practice, for languages environments that use the preprocessor like C and C++. It's not enough to be able to parse the header file, you need to be able to parse and understand all the build configuration too, to know about the values of the macros that were used at build time to compile the binary. So that means you need to understand all the possible build systems used in the distro (automake, cmake, the custom ones, and whatnot).
A giant task that even abi-compliance-checker doesn't get right, because, well, it's extremely hard to do well. In particular, abi-compliance-checker invokes GCC to compile the header files you give it, so that it can then look at the result of the compilation to see if anything changed. But then, that is extremely fragile because, how does abi-compliance-checker know what preprocessor macro to set before compiling those header files? At the moment, the answer is, it doesn't. And given its current setup, at best it would be up to the user to tell the tool which preprocessor macro value to use. Which would be terrible in terms of user friendliness IMHO.
In other words, I think that the current design of abi-compliance-checker (wrt API checking at very least) is inherently flawed and hard to change without re-writing most of the tool.
Besides, what may seem like an additional feature of abi-compliance-checker is actually a rigidity that hinders its usefulness as a distro-wide ABI checker. Think about it. To use abi-compliance-checker to compare the ABI of libfoo1.so and libfoo2.so, you have to write two XML descriptors in which you have to say where the header files of libfoo1.so and libfoo2.so are. Otherwise, the tool cannot operate. Do you Imagine if you have to do that for thousands of packages in a distro? Not practical. And you know what, it's not even practical for one package. And I say that it's not needed to check for ABI compatibility. And you know what? it's not even useful to check for API compatibility. I can give you a simple example, if you really insist, where abidiff detects API changes (and their ABI impact) on by just looking at two *.so file where abi-compatibility-checker can't even operate. I am not sure this bug report is the place to do that, but I can do it if you want.
In light of all this ...
I'd say that yes, libabigail is young, but I think it's already better than abi-compliance-checker. By better I mean less fragile, more user friendly (less rigid), performs more in-depth ABI change impact analysis, and, generally speaking, has more features. The only thing where abi-compliance-checker would be ahead, IMHO, is the html reports. Libabigail only emits textual reports for the moment.
But I think that is more easily fixable than fixing abi-complicance-checker on the fronts where it lags behind libabigail tools today.
Replying to [comment:4 dodji]:
Replying to [comment:3 corsepiu]: I can not spot any man-pages, examples/use-cases. The manuals for all the commands are available online Irrelevant. If you want have your package to be promoted prominently you can not avoid shipping them locally. Note that the Fedora packages come with local version of these manual, in the html and info formats. OK, I just noticed, I missed libabigail is one of these packages, whose maintainer outsmarted himself by splitting out user-man-pages into a docs packages (I hate doc packages - They are harmful to usability= As for the library itself, it also has extensive API documentation in which almost all the public entry points of the library are documented at https://sourceware.org/libabigail/apidoc. API-docs are not relevant to end users.
The manuals for all the commands are available online Irrelevant. If you want have your package to be promoted prominently you can not avoid shipping them locally.
Note that the Fedora packages come with local version of these manual, in the html and info formats. OK, I just noticed, I missed libabigail is one of these packages, whose maintainer outsmarted himself by splitting out user-man-pages into a docs packages (I hate doc packages - They are harmful to usability=
As for the library itself, it also has extensive API documentation in which almost all the public entry points of the library are documented at https://sourceware.org/libabigail/apidoc. API-docs are not relevant to end users.
To cut a long story short: I just had a case, I which I had to check for ABI compatiblity. Due to your request, I tried to apply libabigail and failed miserably. I therefore call the claim of "libabigail is easier to use" to untrue and the fact libabigail does not check APIs to be very restricting to this tool.
API-docs are not relevant to end users.
You're the one to mention APIs in the first place and you're complaining that the fact that they are documented is irrelevant?? While it proves that non-only the API exist but that it is documented enough for people to use, yes maybe not end-user, but do you expect end-users to use abi-compliance-checker's API ?
Due to your request, I tried to apply libabigail and failed miserably.
There might be bugs :-) Oh, and abi-compliance-checker does have bugs too. Can you point me to the packages on which the tool failed? You can just send me links to those via emails if you like. Or just give the information here.
I therefore call the claim of "libabigail is easier to use" to untrue and the fact libabigail does not check APIs to be very restricting to this tool.
Again, libabigail does check for API. It does so by looking at the debug info. Do you have a specific case where it missed an API change?
Replying to [comment:6 pingou]:
API-docs are not relevant to end users. You're the one to mention APIs in the first place and you're complaining that the fact that they are documented is irrelevant??
You're the one to mention APIs in the first place and you're complaining that the fact that they are documented is irrelevant??
You misunderstood: libabigail's API and docs thereof are irrelevant to users, who want to check a package. The API of a package to check is not irrelevant. abi-compliance-checker checks it, libabigail doesn't seem to provide this feature.
Replying to [comment:8 corsepiu]:
You misunderstood: libabigail's API and docs thereof are irrelevant to users, who want to check a package. The API of a package to check is not irrelevant. abi-compliance-checker checks it
Indeed, I thought in your sentence:
because abi-compliance-checker also APIs the verb missing was has, while based on your comment I guess you meant checks
has
checks
The API of a package to check is not irrelevant. abi-compliance-checker checks it, libabigail doesn't seem to provide this feature.
Ralph, you keep saying that Libabigail tools don't detect API change. That is simply not true. I gave you a long and detailed answer earlier and you don't seem to get it. I take it as I was not clear. So let me try again differently.
Here is what abidiff says about a certain change in a binary:
http://dodji.seketeli.net/report.txt
And here is what abi-compliance-checker says about the same binary:
http://dodji.seketeli.net/compat_report.html
Not only does libabigail say exactly where in the type the change occurs (the change occurred in a base class of class S, making eventually making S to grow) but it does also tell you what public entry point is impacted by the type change. And guess what, that public entry point is an API entry point!
And as you can see the abi-compliance-checker report, although in html, is much less detailed (and useful) to spot where in the source code the change occurred. So the API change that abi-compliance-checker is supposed to check better than libabigail is actually better described by Libabigail!!
So, it would be more useful if you could be more specific from now on.
Thanks.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-11-19/fpc.2015-11-19-16.00.txt):
Replying to [comment:11 james]:
ACTION: Dodji Will tweak the draft for the week after next. (geppetto, 17:36:24)
Right.
I have modified the packaging guideline change I initially proposed. The amendment I am now proposing can be reviewed at https://fedoraproject.org/w/index.php?title=User%3ADodji%2FPackaging%3AGuidelinesAmendedForABIChecking&diff=428398&oldid=428239.
The draft resulting from that amendment can still be read ''in extenso'' at https://fedoraproject.org/wiki/User:Dodji/Packaging:GuidelinesAmendedForABIChecking.
Please note that to make that amendment "work", I have updated the wiki page that it references; that page now '''also''' talks about how to compare the ABIs of two ELF binaries, along with how to compare the ABIs of binaries carried by two packages. The referenced wiki page is https://fedoraproject.org/wiki/How_to_check_for_ABI_changes_in_a_package. The exact changes I authored can be reviewed at https://fedoraproject.org/w/index.php?title=How_to_check_for_ABI_changes_in_a_package&diff=428397&oldid=428244.
I have also updated the referenced wiki page that describes how to compare ABIs using abi-compliance-checker to make it talk about how to compare the ABIs of two versions of ELF binaries too. The changes I authored can be reviewed at https://fedoraproject.org/w/index.php?title=How_to_check_for_ABI_changes_with_abi_compliance_checker&diff=428396&oldid=428256.
Cheers.
We discussed this at this weeks meeting (http://meetbot.fedoraproject.org/fedora-meeting-1/2015-12-03/fpc.2015-12-03-17.00.txt):
Announcement text:
The shared library section of the guidelines has been updated to refer to a separate page on ABI comparison tools. https://fedoraproject.org/wiki/Packaging:Guidelines#Shared_Libraries https://fedoraproject.org/wiki/How_to_check_for_ABI_changes_in_a_package * https://fedorahosted.org/fpc/ticket/579
Metadata Update from @dodji: - Issue assigned to tibbs
Login to comment on this ticket.