I think your design taste is sound. But there are some constraints.
The VR/AR vendor market has largely collapsed into a few big players with vertical-integration ecosystem-capture dreams, so high-investment opaque monoliths is what's available. Valve's OpenVR apis being the biggest multi-vendor exception. And they're immature, in a complex and still-evolving design space, making it harder to abstract commonality. And gaming VR imposes strict performance constraints, so some paths may have little slack with which to pay some abstraction costs. So having vendor-specific monoliths, joined only at the top, could be the right choice. Any failure to have their javascript-level joining be flexibly mixable... seems regrettably traditional for browser apis, and I don't know the cultural background for that.
I've mixed Vive lighthouse and custom optical tracking, Vive and WMR and now drone HMDs, on linux. I took a custom browser-as-compositor stack. (But there's also the Vrui VR Toolkit.)
The VR/AR vendor market has largely collapsed into a few big players with vertical-integration ecosystem-capture dreams, so high-investment opaque monoliths is what's available. Valve's OpenVR apis being the biggest multi-vendor exception. And they're immature, in a complex and still-evolving design space, making it harder to abstract commonality. And gaming VR imposes strict performance constraints, so some paths may have little slack with which to pay some abstraction costs. So having vendor-specific monoliths, joined only at the top, could be the right choice. Any failure to have their javascript-level joining be flexibly mixable... seems regrettably traditional for browser apis, and I don't know the cultural background for that.
I've mixed Vive lighthouse and custom optical tracking, Vive and WMR and now drone HMDs, on linux. I took a custom browser-as-compositor stack. (But there's also the Vrui VR Toolkit.)