PipeWire and fixing the Linux Video Capture stack • Hacker News Board

Re: PipeWire and fixing the Linux Video Capture stack

#31

Post by HMH » Sun, Oct 03, 2021, 5:03 PM UTC

jmclnx wrote: post #30

I had to read the blog on lynx due to light grey text on white. I wish people will realize many people will have a hard time with that coloured text. Well a bit concerning in a way. Does this change require GNOME and/or Wayland ? The way I read it it seems so, but little detail there. I wonder because I use fvwm2 as opposed to Desktops, which I really do not like at all.

> had to read the blog on lynx due to light grey text on white.

No need to go that far, this site supports reader mode.

> Does this change require GNOME and/or Wayland?

PipeWire is not bound to Wayland or a specific desktop environment, it runs (mostly) fine under GNOME, KDE Plasma as well as sway. The desktop interfacing is done via flatpaks xdg-desktop-portal, as long as your desktop implements that you should be fine.

Re: PipeWire and fixing the Linux Video Capture stack

#32

Post by beermonster » Sun, Oct 03, 2021, 5:57 PM UTC

zibzab wrote: post #2

I've heard so much nice things about this, but just can't figure out how to get it to work. AUR has packages for replacing PA and jack with pipewire, but when I tired this I ended up with a dead ardour and not even timeshift could get me back to a working system.

No need for the AUR. It’s in extra in the official repos.

See https://wiki.archlinux.org/title/PipeWire#Installation

I’ve had zero issues.

Re: PipeWire and fixing the Linux Video Capture stack

#33

Post by BlackLotus89 » Sun, Oct 03, 2021, 6:33 PM UTC

alias_neo wrote: post #20

I don't suppose anyone has hints for switching to wired headset on Ubuntu "on-plug". I switched to pipewire recently and the experience for Bluetooth is amazing. On an average day I have audio output set to HDMI while I listen to Spotify and work; when I plug in a wired headset for a meeting the gnome switcher dialog pops up but when I select headset, it doesn't work since moving to pipewire, output remains with HDMI…

There are only scripting solutions right now based on pacmd and acpi calls. My pinebook pro needs a hack like this as well for when I want to use my headset.

So putting a script /etc/acpi/ should do the trick, but is a bit hacky.

Re: PipeWire and fixing the Linux Video Capture stack

#34

Post by speeddemon » Sun, Oct 03, 2021, 8:22 PM UTC

boudin wrote: post #18

Iolaum wrote: post #14

Earlier quoted context omitted.

Have you tried installing zoom as a flatpak from flathub? Using it at Fedora Silverblue 34 (Gnome 40 on Wayland) and screen sharing works (using it often at work).

Zoom screen sharing on wayland has been hacked around Gnome's private d-bus screenshot api. It doesn't work in any other environment.

That's not really a big deal, you could just develop a small shim for that d-bus API if you really needed to. But I agree, it would be better for them to switch to Pipewire.

Re: PipeWire and fixing the Linux Video Capture stack

#35

Post by speeddemon » Sun, Oct 03, 2021, 8:36 PM UTC

HMH wrote: post #27

> In fact in the first few releases of Fedora Workstation where we shipped PipeWire we solely enabled it as a tool to handle screen sharing for Wayland and Flatpaks. PipeWire is truly a godsend for screen recording on Wayland, not very long ago, if at all, every desktop environment / compositor had their own home brewed protocol and developing anything related to screen recording on Wayland was a massive pain [1]. I…

>I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this!

It likely won't ever get an official Wayland extension, for a number of reasons. Screen capturing is inherently insecure and so it shouldn't really be happening over the Wayland socket. It also doesn't need to happen over the Wayland socket or be related to the windowing protocol at all, attempting to do it that way is just re-accumulating technical debt left over from X11.

>I fear that this means if flatpak does not need a feature, even if it makes sense in another context, it probably won't ever be implemented.

I wouldn't think of it so much as just flatpak, but the API needs to be usable from within a sandbox, and it needs to be done in such a way that it can be reasonably presented to the user.

>For example something I require for my project are proper window names and their position

This is a really bad idea, don't do this. This is guaranteed to mess with the window manager and run into any number of syncing issues with window sizes being reported incorrectly. It's also a misuse of the screencasting API. I checked the issues you posted and I don't think you're going to get much farther trying to hack around the API. If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way.

Re: PipeWire and fixing the Linux Video Capture stack

#36

Post by HMH » Sun, Oct 03, 2021, 10:15 PM UTC

speeddemon wrote: post #35

HMH wrote: post #27

> In fact in the first few releases of Fedora Workstation where we shipped PipeWire we solely enabled it as a tool to handle screen sharing for Wayland and Flatpaks. PipeWire is truly a godsend for screen recording on Wayland, not very long ago, if at all, every desktop environment / compositor had their own home brewed protocol and developing anything related to screen recording on Wayland was a massive pain [1]. I…

>I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this! It likely won't ever get an official Wayland extension, for a number of reasons. Screen capturing is inherently insecure and so it shouldn't really be happening over the Wayland socket. It also doesn't need to happen over t…

> attempting to do it that way is just re-accumulating technical debt left over from X11

I see, I do not know much about the internal workings of Wayland. The problem I do see though is that screen recording is an essential feature that has been neglected for years due to the lack of a standard.

> This is a really bad idea, don't do this.

While I agree that getting window positions may be questionable, providing a human readable title for the streams that have been selected seems obvious, yet is missing.

> syncing issues with window sizes being reported incorrectly

This seems rather theoretical to me, while of course true, I doubt it is going to matter in practice.

> It's also a misuse of the screencasting API.

Hmm, do you think there is a better solution, that isn't giving up and having each desktop environment do its own thing?

> If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way.

This is also my conclusion, alas it's not happening and one of the reasons why I think Wayland is not quite there yet.

Re: PipeWire and fixing the Linux Video Capture stack

#37

Post by alias_neo » Sun, Oct 03, 2021, 11:27 PM UTC

BlackLotus89 wrote: post #33

alias_neo wrote: post #20

I don't suppose anyone has hints for switching to wired headset on Ubuntu "on-plug". I switched to pipewire recently and the experience for Bluetooth is amazing. On an average day I have audio output set to HDMI while I listen to Spotify and work; when I plug in a wired headset for a meeting the gnome switcher dialog pops up but when I select headset, it doesn't work since moving to pipewire, output remains with HDMI…

There are only scripting solutions right now based on pacmd and acpi calls. My pinebook pro needs a hack like this as well for when I want to use my headset. So putting a script /etc/acpi/ should do the trick, but is a bit hacky.

Interesting, good to know it's not just me.

I'll switch my work laptop back to pulseaudio as being able to switch on plug/unplug is critical for my average day.

My personal laptop can stay on pipewire since I exclusively use Bluetooth there.

Hopefully it's a feature that'll be implemented soon.

Re: PipeWire and fixing the Linux Video Capture stack

#38

Post by speeddemon » Mon, Oct 04, 2021, 4:18 AM UTC

HMH wrote: post #36

speeddemon wrote: post #35

Earlier quoted context omitted.

>I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this! It likely won't ever get an official Wayland extension, for a number of reasons. Screen capturing is inherently insecure and so it shouldn't really be happening over the Wayland socket. It also doesn't need to happen over t…

> attempting to do it that way is just re-accumulating technical debt left over from X11 I see, I do not know much about the internal workings of Wayland. The problem I do see though is that screen recording is an essential feature that has been neglected for years due to the lack of a standard. > This is a really bad idea, don't do this. While I agree that getting window positions may be questionable, providing a hu…

There is a standard API for screencasting and it works fine if you're only doing screencasting. Since the API is in the xdg portal, it sits at a layer above Wayland and X11 and in theory it should make it easier for you because it works the same on both of them and you could avoid writing window system specific code and just have one Linux backend using the xdg portal. But in practice it seems like your use case needed to access window system specific functionality anyway.

In my opinion, what you're asking for is an entirely new thing. You want an API for window management, but there has never really been a window management API on Linux, and every desktop environment was already doing its own thing. It has nothing to do with Wayland specifically. Under Xorg, the X11 APIs can sort of be hacked to do what you're asking under certain circumstances, and it might work ok on some window managers, but it's not going to work in every case. (And it doesn't play nicely with sandboxes either)

To get the human readable title seems like it would be an issue with how Mutter publishes the stream, i.e. not an issue with the portal itself. But I haven't looked into this enough in detail. I also don't know enough about your application to know why you would need the window sizes and positions. My only point is that if you intend to use that to draw an overlay over a screencast, or otherwise try to reconstruct what the compositing process is doing, then that's not going to be completely accurate. And it can't really be accurate without exposing various internals of how the compositor works, which puts any proposed API design back to square one...