- Device emulation: uinput covers this; requiring root is reasonable for what it does.
- Input injection. Like XTEST, but ideally with permissions and more event types (i.e. tablet and touch events.) libei is close but I think it should be a Wayland protocol.
- UI automation: Right now I think the closest you can get is with AT-SPI2, for apps that support it. This should also be a Wayland protocol.
None of these are actually easy if you want to make a good API. (XTEST is a convenient API, but not a particularly good one. Win32 has better input emulation and UI automation features IMO.)
Also the tangent about how crazy the compatibility layers are is weird. Yes, funny things are being done for the sake of compatibility. XWaylandVideoBridge is another example, but screen sharing is an area where Wayland is arguably better (despite what NVIDIA has to say) because you can get zero copy window and screen contents through PipeWire thanks to dmabufs.
Some of the lack of progress comes down to disagreements. libei mainly exists, by my best estimate, because the GNOME folks don't like putting things in Mutter, and don't want to figure out how to deal with moving things out of process while keeping them in protocol. (Nevermind the fact that this still has to go through Mutter eventually, since it is the guy sending the events anyways...) However, as far as I know, lack of progress on UI automation and accessibility entirely comes down to funding. It's easy to say "why not just add SetCursorPos(x, y)" and laugh it off, but attacking these problems is really quite complex. There was Newton for the UI automation part, but unfortunately we haven't heard anything since 2024 AFAIK, and nobody else has stepped up.
If Wayland lasts as long as X11 did, it's preposterous to not spend the time to try to get the "new" version of these things right even if it is painful in the meantime.
After all, it isn't like UI automation on Linux was ever particularly good. Anyone who has ever used AutoHotkey could've told you that.
- Device emulation: uinput covers this; requiring root is reasonable for what it does.
- Input injection. Like XTEST, but ideally with permissions and more event types (i.e. tablet and touch events.) libei is close but I think it should be a Wayland protocol.
- UI automation: Right now I think the closest you can get is with AT-SPI2, for apps that support it. This should also be a Wayland protocol.
None of these are actually easy if you want to make a good API. (XTEST is a convenient API, but not a particularly good one. Win32 has better input emulation and UI automation features IMO.)
Also the tangent about how crazy the compatibility layers are is weird. Yes, funny things are being done for the sake of compatibility. XWaylandVideoBridge is another example, but screen sharing is an area where Wayland is arguably better (despite what NVIDIA has to say) because you can get zero copy window and screen contents through PipeWire thanks to dmabufs.
Some of the lack of progress comes down to disagreements. libei mainly exists, by my best estimate, because the GNOME folks don't like putting things in Mutter, and don't want to figure out how to deal with moving things out of process while keeping them in protocol. (Nevermind the fact that this still has to go through Mutter eventually, since it is the guy sending the events anyways...) However, as far as I know, lack of progress on UI automation and accessibility entirely comes down to funding. It's easy to say "why not just add SetCursorPos(x, y)" and laugh it off, but attacking these problems is really quite complex. There was Newton for the UI automation part, but unfortunately we haven't heard anything since 2024 AFAIK, and nobody else has stepped up.
https://blogs.gnome.org/a11y/2023/10/27/a-new-accessibility-...
Color management is the perfect example of how a simple ask can be complicated. How hard could it really be? Well, see for yourself.
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m...
If Wayland lasts as long as X11 did, it's preposterous to not spend the time to try to get the "new" version of these things right even if it is painful in the meantime.
After all, it isn't like UI automation on Linux was ever particularly good. Anyone who has ever used AutoHotkey could've told you that.