#9119 LTO by default for package builds
Closed: Fixed 3 months ago by mohanboddu. Opened 11 months ago by law.

  • Describe the issue
    Please check the impact of the proposed change here:

https://fedoraproject.org/wiki/LTOByDefault

In particular note that we must not ship .o/.a files with the LTO byte code. The plan is to strip this out at the end of the rpmbuild process. It's conceptually similar to stripping debugging information, but we have to do it on any .o/.a files.

  • When do you need this? 2019/12/30

  • When is this no longer needed or useful? N/A. The LTO bytecode stripping will likely always be desired because the LTO bytecodes from one GCC release can not necessarily be handled by a different GCC release.

  • If we cannot complete your request, what is the impact? We likely would not be able to turn on LTO for Fedora builds and we would miss out on the performance and code size improvements that result from link time optimization.


Metadata Update from @mohanboddu:
- Issue tagged with: changes, f32

10 months ago

Metadata Update from @mohanboddu:
- Issue tagged with: change-ack, mass rebuild

10 months ago

@law Correct me if I am wrong but it seems like the work is not done yet, so, are you planning on deferring to F33?

Or your LTO work starts after mass rebuild which includes gcc10?

I'd planned to reach out today on this issue, but you beat me...

I think the code generation and related aspects are in good shape. What is not is the interactions with GDB -- we're really just now digging into those and there's more problems than anticipated. With that in mind I think deferring to F33 is the wisest choice.

I think the only PR that needs to go forward right now is the one for stripping LTO bytecodes out of any installed .o/.a files.

(removed, sorry, misread the discussion)

Metadata Update from @mohanboddu:
- Issue untagged with: f32
- Issue tagged with: f33

9 months ago

@law Whats the status on this change?

Coming along quite well. The opt-outs are mostly in place and I'm chasing down more more issue over the weekend with the expectation it'll be flipped on by default Monday/Tuesday.

@law can you please address my comment on redhat-rpm-config PR so that I can merge it and build during Monday (I'm out on Tuesday)?

Will do. Meeting load this morning was awful, so probably Tue/Wed makes more sense anyway.

Igor, are you referring to the request to use a conditional for the lto_cflags reference, ie
%{?_lto_cflags}. I just want to make sure I'm addressing the right thing.

I'll issue a fresh PR tomorrow which makes a couple changes. Dropping the -flto-partition=one, it's no longer needed, changing the lto_cflags reference to use a conditional and rebasing against the current redhat-rpm-config.

After much thought I think we want to keep -ffat-lto-objects for now as a fail-safe. It sucks up more time on the builders, but I think we need a post-build step which verifies that any installed .o/.a has real symbols or sections (after stripping the LTO bytecodes). Otherwise it's too easy for someone not familiar with how LTO works to install a .o/.a file which has no real contents. I'll volunteer to write that test and also verify all the packages in the distro get updated appropriately. This can happen after the mass rebuild that apparently starts Wednesday as it just makes things build faster.

I'm also going to be watching the mass rebuild just in case there's LTO related issues. -- particularly on s390x which has been problematical in past mass rebuilds due to its different tuning in the inliner.

@law mass rebuild will start on 22nd (in 17 hours), should we delay it? If so, for how long?

Igor, are you referring to the request to use a conditional for the lto_cflags reference, ie
%{?_lto_cflags}. I just want to make sure I'm addressing the right thing.

No, not really..

  • "%global _lto_cflags %{?_lto_cflags} -ffat-lto-objects -flto-partition=one"

  • %_lto_cflags -flto -ffat-lto-objects -flto-partition=one

There is simply no reason to have comment that will change -flto -ffat-lto-objects -flto-partition=one into the -flto -ffat-lto-objects -flto-partition=one -ffat-lto-objects -flto-partition=one because those options are duplicated. I thought you meant to put some different example.

Otherwise it's too easy for someone not familiar with how LTO works to install a .o/.a file which has no real contents.

I think we should just do it and let people fix problematic packages if there are any.

I'll issue a fresh PR tomorrow which makes a couple changes.

If you just force-push into your branch - it will update an existing PR on src.fp.o.

@law so what's the state here? should we consider delaying mass rebuild, defer LTO for F34 or?

Oh, that's long since gone in my copy here :-) No worries about that.

WRT -ffat-lto-objects. The problem is you don't know something went wrong until someone tries to use the installed package with an effectively empty .o/.a to build something else. That's why I think we need something that checks the installed .o/.a's to ensure they're non-empty after stripping the LTO bytecodes and symbols. I'd rather fail safe now, write that checker, then build the world to identify the problem packages.

WRT state. I don't think this should hold up the mass rebuild. Hell, until yesterday I wasn't aware we were even doing a mass rebuild for F33 -- packages that build after merging the PR will get the benefit of LTO. Those that aren't won't, but everything still works just fine.

FWIW, I'm trying to chase down Jakub to find out what his plan WRT gcc-10.2 which folks want installed prior to mass rebuild. The upstream release hasn't happened yet, but I know he was shooting for this week.

Merged and built as redhat-rpm-config-163-1.fc33. @mohanboddu FYI.

Closing the ticket as the work here is done.

Metadata Update from @mohanboddu:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 months ago

Login to comment on this ticket.

Metadata