>If the Linux kernel had worked the same way, we'd never have gotten Linux.
Yes we would. In fact it's exactly how Linux works and has worked for decades. New features get added in drivers and then get rolled up into driver subsystems when enough hardware supports them.
There is a way to protect against LD_PRELOAD: use sandboxing. X11 is bad for security because it's an intentional escape around the sandbox.
The "tear it down and build something else argument" is also going nowhere, anything else you build is going to run into the same questions of how to go about implementing these features.
>The Wayland architecture does not give me a choice in my screenshot, video recording, and remote desktop tool
Yes it does. Use the screenshot, screencast and remote desktop XDG portals for that. It's understandable you're confused because the X11 way was to try to jam everything into X extensions whether it made sense or not. The overall trend lately is to move APIs into other components (XDG portals, pipewire, DBus, systemd, etc.)
Portals are a really weird thing. They're mostly pushed via the Flatpak/GNOME crowd, and are intended to be cross platform by using an external messaging bus: d-bus (e.g.: they don't use the X protocol and Wayland). The same interface can be used in both Xorg and Wayland, and that's the only upside of portals.
However, their implementation is not meant for native applications and are pretty inefficient. For example, the screenshot portal will save the image into disk and return its path -- so if you wanted to render the image on screen, you now need to read and decode a file. The screenshot is encoded, serialised to disk, serialised FROM disk and then decoded again just to get pixels from the compositor back into the compositor.
Portal really don't make sense if you're writing native wayland clients, since portals just really end up using the wayland protocol under the hood, with two or three services in the middle handing messages over just to expose a "standard" API.
The portal only allows an external application to request the service from the compositor. He portal does not allow an external application to provide additional functionality beyond what is already provided by the compositor.
In my use case, I want to screen share a specific window, not the whole screen; this is functionality previously provided by a third-party X11-based tool, which cannot be provided from an XDG portal (even on those compositors that support portal at all).
Yes we would. In fact it's exactly how Linux works and has worked for decades. New features get added in drivers and then get rolled up into driver subsystems when enough hardware supports them.