#2670 Consider triggering contingency plan for F35 WirePlumber change
Closed: Rejected 7 months ago by zbyszek. Opened 7 months ago by adamwill.

Tom Seewald made a good case on devel@ for why the wireplumber change may not be really ready for F35:

"Now that Fedora 35 is in beta and more people are testing it out, it appears that wireplumber does have some regressions compared to pipewire-media-session. Namely that audio output switching for flatpak applications does not work [1], and that desktop system sound volume (e.g. notification sounds) can not be controlled independently from the main system volume [2][3]. There is also an issue with changing bluetooth speaker volume [5]. From developer comments it may not be likely that all of these problems will be fixed in time for the F35 release, and the current workaround is to switch back to pipewire-media-session [4].

Clearly this is late in the release cycle, but I am concerned that wireplumber will provide an overall worse audio experience for F35 users. So ultimately I am asking if we should re-evaluate whether or not wireplumber should be the default pipewire session manager for F35. Are there other features landing in F35 that specifically depend on wireplumber, and if so do they outweigh the currently known regressions?

[1] https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/59
[2] https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/51
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2003403
[4] https://bugzilla.redhat.com/show_bug.cgi?id=2003403#c4
[5] https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/58
"

So I'm filing this to ask FESCo to consider whether we should invoke the contingency plan (which is to switch to the old session manager). If we want to do that it'd be good to get it done in plenty of time for Final so we can re-test that things work on upgrade and so on.

@wtaymans @bcotton


FWIW, I like to have shiny new things just as much as others, but I think the request to consider backing out of wireplumber for F35 has merit. Since upgrading to F35 I have touched way more rough edges than I did with the default switch to PipeWire, to a degree that I'm not sure I'm comfortable with. No show-stoppers, but many annoying things that weren't annoying enough on their own to spend time on reporting them (sorry! I'm busy 🙈). Given that the switch to PipeWire itself was largely fine and very well-received, it would be a shame to squander that goodwill by breaking audio again just one release later. (Sorry if this is not idiomatic English, it's still early for me.)

So, if it's still possible to not do the switch on upgrade to F35 (and to revert it for those users who have already upgraded), I think it would be good to have wireplumber still as opt-in for Fedora 35 and reconsider it for F36, just to make sure we can smooth out the rough edges before users start bleeding on edges again.

+1 to trigger the contingency plan

-1
If I have a vote?
F35 wireplumber has been more reliable than F34 pipewire-media-session on all my hardware.

Let's put this on for Monday's meeting so that @wtaymans can talk to us about this.

Metadata Update from @ngompa:
- Issue set to the milestone: Fedora 35
- Issue tagged with: meeting

7 months ago

Metadata Update from @ngompa:
- Issue set to the milestone: None (was: Fedora 35)

7 months ago

Metadata Update from @ngompa:
- Issue set to the milestone: Fedora 35
- Issue tagged with: self contained change

7 months ago

FWIW, I have seen regression in audio handling on my laptop too recently. But I haven't had the time to figure this out enough to file a bug report.

I want to gather more data before trying to make a decision. It indeed seems like there's a bunch of issues. But in particular: are those issues really in wireplumber, and not e.g. in pipewire itself or other libs, and is it enough to flip back to the old session manager?

F35 wireplumber has been more reliable than F34 pipewire-media-session on all my hardware.

That's a good datapoint. But in general, regressions count for more than new features. Other things being equal, we'd generally pick avoiding a regression over two features of the same weight, even though the net gain might be the same in both cases. This is because in the first case we have more stability, and this is has value too. (Users for whom the features are important enough, can opt-in into the new thing earlier.)

FWIW, I have seen regression in audio handling on my laptop too recently. But I haven't had the time to figure this out enough to file a bug report.

I want to gather more data before trying to make a decision. It indeed seems like there's a bunch of issues. But in particular: are those issues really in wireplumber, and not e.g. in pipewire itself or other libs, and is it enough to flip back to the old session manager?

From the bug report comments that I have read, these issues are with wireplumber and not with pipewire. Switching back to pipewire-media-session has been the recommended fix.

Here are some example developer/maintainer quotes:

"As an update: this is a missing/incompatible feature in wireplumber but requires some upstream work so it's unlikely it'll be fixed until 0.4.3 (or whatever the next version number will be). Meanwhile, it's best to use pipewire-media-session to work around this bug: "dnf swap wireplumber pipewire-media-session" should do the trick." [1]

"I just double-checked and WirePLumber does not handle PW_KEY_MEDIA_CATEGORY property, so I suppose this is why it does not work.
I checked the code in pipewire-media-session and, if the media category is set to "Manager", the client is granted all permissions: https://gitlab.freedesktop.org/pipewire/pipewire/-/blob/master/src/media-session/access-flatpak.c#L104
We need to do the same in WirePlumber." [2]

There have also been two more comments posted [3][4] since I brought this up, and they are both reporting problems with wireplumber that are resolved by switching back to pipewire-media-session.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2003403#c4
[2] https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/59 (comment 2)
[3] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/C7USY2CRKDNKQH6A34KXWAQUWPQKNLS3/
[4] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/SLBP6ZVRFFMA2WK4MD7VK7265Y2LFVSI/

After upgrading to F35, I had no sound at all. In the sound panel of system settings, I noticed that my sound level was displayed as muted even though the volume slider was not at the minimum level. Eventually (after 10-20 minutes of frustration, including a reboot that didn't help) I discovered that I had no output device selected, and manually selecting my headphones was enough to correct the problem. Headphone detection used to work automatically.

I discovered that I had no output device selected, and manually selecting my headphones was enough to correct the problem. Headphone detection used to work automatically.

My issue is a variant of this: when I connect a new sink (e.g. an amplifier over bluetooth or headphones over bluetooth), puvucontrol shows the new sink as the default (green checkmark is depressed), but sound actually goes to other sinks. When I click on the checkmark, suddenly the sink is switched. This used to work relatively recently, and I suspect it's broke with the switch to wireplumber.

Metadata Update from @ngompa:
- Issue assigned to ngompa

7 months ago

The group came to the consensus in today's meeting:

AGREED: QA will meet on Friday to decide if WirePlumber is in good enough shape. If not, @ngompa will revert after the end of that meeting (+9, 0, -0) (@ngompa, 19:38:30)

Metadata Update from @ngompa:
- Issue untagged with: meeting

7 months ago

I've scheduled a special QA meeting to make this decision at 16:00 UTC on Friday (2021-10-08) in #fedora-meeting-1.

Hi everyone, I am the WirePlumber maintainer.

I am sorry to hear about all these issues. I was aware of a few of them, but not all.

My (obviously biased) opinion is that fedora should stick to WirePlumber and try to fix these issues. This is simply because using pipewire-media-session will, in my opinion, prolong the problem and postpone facing similar issues to the next release. The community's decision has been for a long time that WirePlumber should be the default session manager. pipewire-media-session was a proof of concept that has lived far longer than it should have. The problem is that as long as distributions use p-m-s by default, new pipewire features have to be supported in p-m-s first and then be ported to WirePlumber. This is the very reason that some of these issues exist right now, because new features were added recently in pipewire and nobody has taken the initiative to add support for them in WirePlumber. Unless WirePlumber gets used by a wider audience, I don't see that changing, so we will likely face similar issues again in F36.

At the same time, I am happy to hear that you have given this time until Friday, because I believe that these issues can be analysed and fixed by then. In more detail:

-> https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/59

I fixed that this morning. It may have sounded like a complex problem, but it was really a few lines in a config file.

-> https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/64

Wim fixed that one earlier. Also a simple change.

-> If Wireplumber is used, some video files open with ~ 20 s delay in vlc

I understand how to fix that, but will take me 1-2 days to shape up the logic for handling iec958 audio. This is an example of a new feature that made it into pipewire recently but wasn't added to WirePlumber.

-> Changing Gnome system sounds volume doesn't work with WirePlumber

Wim volunteered to work on this one. The idea is to introduce a hack, using a C module that translates the interface that pipewire-pulse uses to the interface that our lua script is using.

-> https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/58

This is not a wireplumber issue as far as we can tell. It's something lower level (bluez/kernel/firmware). I will keep an eye on it, though, in case I manage to reproduce it again.

-> "when I connect a new sink (e.g. an amplifier over bluetooth or headphones over bluetooth), puvucontrol shows the new sink as the default (green checkmark is depressed), but sound actually goes to other sinks. When I click on the checkmark, suddenly the sink is switched"

Works for me, but I will have a look again at the logic. Perhaps there's something that doesn't align 100% with the logic that p-m-s uses. This is all in a lua script and it should be trivial to fix as soon as we find the root cause.

-> "After upgrading to F35, I had no sound at all. In the sound panel of system settings, I noticed that my sound level was displayed as muted even though the volume slider was not at the minimum level. "

Again, there might be some logic diverting from what p-m-s does in the relevant lua script. I will try to confirm it tomorrow. Maybe something related to the device priorities.

These are all the issues that I could spot in this thread. Other issues that have been reported:

-> https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/62

Also happens with p-m-s...

-> https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/60

Looks like some bug in the policy lua script. I can reproduce it, so it shouldn't take me long to fix.

If there are more issues that I am not aware of, please let me know asap.

@gkiagia there's also the following bug - https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1683#note_1089668 - there's probably some race condition and the wireplumber doesn't start correctly (and as a result only a2dp profiles are available).

@gkiagia there's also the following bug - https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1683#note_1089668 - there's probably some race condition and the wireplumber doesn't start correctly (and as a result only a2dp profiles are available).

Thanks! This issue was fixed a week ago in master.

-> "when I connect a new sink (e.g. an amplifier over bluetooth or headphones over bluetooth), puvucontrol shows the new sink as the default (green checkmark is depressed), but sound actually goes to other sinks. When I click on the checkmark, suddenly the sink is switched"

Works for me, but I will have a look again at the logic. Perhaps there's something that doesn't align 100% with the logic that p-m-s uses. This is all in a lua script and it should be trivial to fix as soon as we find the root cause.

What kind of debugging information should I capture to help with this?

Today I have no sound, the sound icon does not appear in the gnome-shell menu indicator, and it's not possible to select any output device in gnome-control-center because everything is insensitive:

Screenshot_from_2021-10-05_07-40-10.png

Where should I report this? Are there any logs that would be useful?

-> "when I connect a new sink (e.g. an amplifier over bluetooth or headphones over bluetooth), puvucontrol shows the new sink as the default (green checkmark is depressed), but sound actually goes to other sinks. When I click on the checkmark, suddenly the sink is switched"

OK, I upgraded to latest versions of everything and restarted, and the issue does not reproduce.

@gkiagia there does not appear to have been an update of wireplumber this week, and the special meeting goes in 12 hours or so. Is an update planned before then? If not, it seems likely we'll have to pull the feature. Thanks!

Even if you release an update in the next 12 hours, that's hardly enough time for us to test it. I plan to spend most of the next 12 hours sleeping, not testing Fedora updates. ;)

To be fair, I think the agreement was to have the update available on Friday, not to have it ready by Friday 00:01 UTC.

Hi everyone,

My apologies for being silent throughout the week. I have been very busy fixing these issues...

I just released WirePlumber 0.4.3 which includes a bunch of fixes from all these past weeks:
https://gitlab.freedesktop.org/pipewire/wireplumber/-/releases/0.4.3

The most important issues have been fixed in this release; notably:
* Issues with no sound after re-login or with bluetooth misbehaving after login (all related to missing logind integration and GVFS/DBus interference) have been fixed
* The "System Sounds" volume control issue has been fixed (credits to Wim)
* Issues related to selecting and using sink monitors as sources have been fixed
* Access issues with flatpak apps have been fixed (tested with EasyEffects, which works fine)
* Policy issues related to virtual nodes have also been fixed

The only remaining issue that I can reproduce is the delay in VLC when starting playback of a file that has an encoded audio stream suitable for SPDIF passthrough (VLC tries SPDIF by default and fails). Unfortunately I ran out of time. I will keep working on this today, of course, and I hope it will be possible to do another package update very soon.

Helped @gkiagia with some testing on the latest package sets for F35 with Version 0.4.3, Release 1.fc35 (tested on main branch before @wtaymans made a package available)

Tested the following applications:

Firefox:
Youtube
BigBlueButton
Meet.google.com

Chrome:
UberConference
Meet.google.com
Vimeo
Microsoft Teams

Other apps:
Hydrogen (flatpak)
Ardour
pavucontrol
Sound panel in control panel
Spotify (flathub flatpak)
easyeffects (flathub flatpak)

wireplumber_testing_2021-10-08_09-55-58.png

All test results seem good with up to 5 audio devices connected (Bluetooth, USB-audio, HDMI, ...). Tested audio switching for sinks and sources.

Screenshot_from_2021-10-08_10-04-27.png

Hope this helps.

Tests were repeated with the packages from @wtaymans with the same positive results.

As requested by FESCo, QA held a special meeting on this topic today. Minutes, log. Our decision was:

AGREED: After reviewing the current status of the bugs that have been referred to, other bugs, and our own testing results, QA agrees to recommend that we stick with wireplumber for the Fedora 35 release (adamw, 16:57:25)

https://bodhi.fedoraproject.org/updates/FEDORA-2021-b70755fdc3 is stable now.
I don't think we need to do anything in FESCo. I'll close this as rejected, since we are not invoking the emergency action.

Metadata Update from @zbyszek:
- Issue close_status updated to: Rejected
- Issue status updated to: Closed (was: Open)

7 months ago

Login to comment on this ticket.

Metadata