And while we're at it, if you need remote X, consider Xpra instead of ssh -X - it's way faster, can survive connection drops, and is far more flexible.
The best metaphor is "tmux for X programs": the X server runs on the same host as the X programs, xpra then "forwards" the windows over a configurable transport (e.g. via SSH) to the client. X was IIRC intended for local networks, so forwarding it over the Internet is sloooow (both by verbose protocol and naive latency handling) - this takes care of both issues.
IIRC it's written in Python, so it runs in most places: https://xpra.org/
There's no supposedly for me using compression. I just checked - it consistently takes 3x longer to launch an xterm without compression than with compression (15 vs 45s!).
Not a bad tip, but using gzip compression over the wire seems pretty stone-age. The proper solution is surely to use a modern lossy video-compression algorithm. Is that possible with X?
It's not something I know a lot about. Is this where VNC steps in?
I don't know much about it, but I'm fairly sure that compressing the Xorg data stream with lossy compression is going to mess it up completely and would require a complete overhaul of the protocol to make that work. VNC is indeed the more standard unix thing (insofar as remote GUIs can be considered standard on unix-likes) that applies lossy compression to the pictures being sent over.
