Hm, I can't think of a good overview off the top of my head (maybe I should sit down and write a good one sometime), but the way it works is:
An application will typically use a GUI toolkit like GTK or QT to do it's UI. This library will use Xlib[1] (or XCB[2] in some cases) to do it's event handling. Event handling in X consists of handling button clicks, sending messages to and from the window manager, cut and paste, etc. (The WM to app protocols are detailed in the ICCCM and the EWMH documents)
The painting will be done through a drawing library. for GTK apps (and Firefox, which isn't a GTK app, but does a good job of pretending it is through nasty hacks[3]) will use Cairo[4] to paint, and QT apps will use QT's paint engine (Arthur, I believe) to draw their UI.
Both of these engines will use XRender (which has largely replaced core X drawing) to actually put the pixels onto the screen, but works at a very low level (antialiased trapezoids and porter-duff compositing of images, more or less). They also fall back in software if the X server doesn't have XRender, or they've detected a buggy version of it.
That's the application side of it.
The fallback library that's used in Cairo is shared with the X server's software XRender implementation, and is called pixman[5]. Making this faster will likely make both the X server and the Cairo image surface faster.
Finally, the X server accelerates drawing to the screen through either the Exa accelleration architecture, or through XAA.
Exa is the more modern of the two, and it accelerates commonly used stuff that XRender wants for things to be fast. This mostly means alpha-transparent blends and such.
XAA is rather old, and is somewhat complicated to implement. Furthermore, it doesn't accelerate very much interesting stuff for modern apps -- when was the last time you saw an unantialiased wide line?
[1] Xlib: the X protocol library and more. http://tronche.com/gui/x/xlib/
[2] XCB: a new X protocol binding. Designed for transparent thread safety and asynchronousness: http://xcb.freedesktop.org/
[3] Firefox renders GTK widgets offscreen, mashes them up, and paints them on screen in the XUL layer to pretend to be GTK
[4] Cairo is a vector graphics library based of the postscript model: http://cairographics.org
[5] Pixman is essentially a software implementation of XRender's drawing operations: http://cgit.freedesktop.org/pixman/
Would it be possible or useful to create an X extension for higher lever (ie, toolkit) primitives for remoting? Ie, an app can send 'list box widget in GTK' to the display server?
Possible, although it would be rather difficult to do well in a flexible manner, and would be unusable by every app that currently exists. More or less, it would be a replacement of the entire X event handling model, entire X drawing model, etc.
And it would make writing apps more difficult -- very few of them use entirely stock widgets, and it would be nearly impossible to securely and performantly implement new widgets on this system.
Plus, it's diametrically opposed to the X "Mechanism, not policy" mantra.
I wouldn't expect anything like this to take off, but it's been tried in the Y window system, and more interestingly -- and more dead-in-the-water -- in the NEWS window system.
Why would it have to replace the existing drawing model? I'm thinking of a way for an app to ask the server if it can render a particular set of high level primitives, if the server responds that it can, the apps sends those instead.
That would allow existing apps and existing display servers, new apps and older display server, and old apps and newer display servers to work fine.
If both the app (at the toolkit level) and the display server are recent, however, remoting is massively sped up.
An application will typically use a GUI toolkit like GTK or QT to do it's UI. This library will use Xlib[1] (or XCB[2] in some cases) to do it's event handling. Event handling in X consists of handling button clicks, sending messages to and from the window manager, cut and paste, etc. (The WM to app protocols are detailed in the ICCCM and the EWMH documents)
The painting will be done through a drawing library. for GTK apps (and Firefox, which isn't a GTK app, but does a good job of pretending it is through nasty hacks[3]) will use Cairo[4] to paint, and QT apps will use QT's paint engine (Arthur, I believe) to draw their UI.
Both of these engines will use XRender (which has largely replaced core X drawing) to actually put the pixels onto the screen, but works at a very low level (antialiased trapezoids and porter-duff compositing of images, more or less). They also fall back in software if the X server doesn't have XRender, or they've detected a buggy version of it.
That's the application side of it.
The fallback library that's used in Cairo is shared with the X server's software XRender implementation, and is called pixman[5]. Making this faster will likely make both the X server and the Cairo image surface faster.
Finally, the X server accelerates drawing to the screen through either the Exa accelleration architecture, or through XAA.
Exa is the more modern of the two, and it accelerates commonly used stuff that XRender wants for things to be fast. This mostly means alpha-transparent blends and such.
XAA is rather old, and is somewhat complicated to implement. Furthermore, it doesn't accelerate very much interesting stuff for modern apps -- when was the last time you saw an unantialiased wide line?