#995 BLAS and LAPACK packaging
Closed: accepted 4 months ago by iucar. Opened 2 years ago by iucar.

With a growing number of implementations of BLAS/LAPACK (including 64-bit versions, threaded, with OpenMP support...), there have been several discussions and yet we still lack a proper mechanism to allow users to switch between them. This is another try on that front.

Recently I found FlexiBLAS, a wrapper library with runtime exchangeable backends, and I believe this is the proper mechanism we were looking for. TL;DR, it wraps the complete BLAS/LAPACK API (so BLAS/LAPACK backends don't require any changes, and BLAS/LAPACK consumers must compile against FlexiBLAS, that's it), and redirects any call to the selected backend (with a nice fallback mechanism if the backend doesn't support a particular method).

Prior discussion in ticket #588.

As a packager, when do I use BuildRequires: pkgconfig(flexiblas) and when BuildRequires: pkgconfig(flexiblas64)? Do I only use the second if I exclude 32bit arches, or...?

If your package uses the 32-bit interface (this is the most common case), the former; if your package requires the 64-bit interface (8-byte integers), the latter. Of course, then you need to exclude 32-bit arches (I should indicate so in the draft for completeness). AFAIK, currently we only have two examples of this: packages petsc64 and suitesparse64, which are in turn subpackages (so they don't exclude any arch: they simply use the 64-bit interface where available). This 64 suffix usage is aligned with current packaging practices for reference BLAS/LAPACK, as well as blis and openblas.

I edited the draft to reflect this properly, suggesting

%if 0%{?__isa_bits} == 64
BuildRequires: pkgconfig(flexiblas64)
BuildRequires: pkgconfig(flexiblas)

for packages with 64-bit support, with a note that one must exclude 32-bit arches if the package only supports the 64-bit interface.

The package has been accepted and it's available in rawhide. If this proposal sounds reasonable to the FPC and there are no major concerns, I would like to submit it as a self-contained change for F33. The only issue is that I would need the help of a provenpackager to rebuild all BLAS/LAPACK dependent packages.

To me, it makes sense in general. However I am not that familiar with FlexiBLAS -- what are the long term commitments if we decide to make it the default? One specific concern I have is that it is GPL licensed -- what are the implications of that for the Fedora packages that use it? Other one is that the development seem to be quite "slow" and "one man show" -- https://gitlab.mpi-magdeburg.mpg.de/software/flexiblas-release/-/commits/master (I lack better terms to describe this)

About the licensing, I'm no expert. Maybe @spot could provide some input here (also he maintains both the reference implementation and consumers).

About the development, I had the same concerns. I contacted Martin, the main developer, 3 weeks ago, after reading his paper about it. He told me that there were no new releases since 2017 because he had been very busy writing his PhD thesis (so completely understandable; been there, done that), but the project is very much alive. They develop and use this tool for research in high-performance computing.

Also, you don't see many commits because, as he told me, the development repo is not public. They use that one only to push releases. That's just a way to ensure they have an advantage to use the new features (mainly in the profiling API) in publications, and then they release the code. Apart from that API, the package just need development to add new functions that appear in Netlib once in a while.

Moreover, I don't know many upstreams as responsive as Martin, and this gives me much confidence. Version 2.0.0 was definitely not ready for this proposal, but Martin listened to my suggestions, implemented a bunch of changes and fixes and here we go: version 3.0.0 in no time. Also, even if the day-to-day development remains not public, I suggested to mirror the repo on GitHub to attract more attention and hopefully collect issues, and here it is too: https://github.com/mpimd-csc/flexiblas.

So, as I said, I had the same concern and approached the project cautiously, but after a few days working with Martin to bring this proposal, I can say I have every confidence in the future of this project. Besides, implementing this change, as well as reverting it in case anything happens, is as easy as rebuilding a limited number of packages by just changing a BuildRequires line.

I think this should indeed go trough a change proposal. Hopefully FPC members can give you enough input, but considering the upcoming deadlines, I wouldn't wait for an approved statement.

Metadata Update from @iucar:
- Issue close_status updated to: accepted
- Issue status updated to: Closed (was: Open)

4 months ago

Login to comment on this ticket.

Related Pull Requests
  • #1152 Merged 4 months ago