I've been wanting to hear this for a long time. Not that I like one more than the other, I just think that Canonicals effort should be towards developing, fixing and improving one UI, not four.
I'm not sure Canonical supports XFCE in any way, and now they also no longer support KDE (financially, that is). There were hiring one guy to work on Kubuntu, that's not much if you ask me. So we're down to two (Gnome-shell and Unity), and the support of gnome-shell is almost mandatory. Unity itself is based on gnome3, so they need to help maintain and improve it.
There is no harm in letting community driven efforts take place. Having a variety of desktop environments available to pick from has its advantages. For example, Xubuntu is really useful for people who need to cut down memory/CPU requirements.
Xubuntu is also really useful if you want a superior desktop UI. I'm not sure if it's gotten harder or easier to make a default Ubuntu install use Xfce4 on the very newest version, it'd be nice if that was an install-time option instead of the separate distro mess. (It used to be just sudo apt-get install xubuntu-desktop, but I've been out of the Ubuntu world for long enough I have no idea if they changed that and it wouldn't surprise me.)
I'll be on Gnome2 until it dies, after that I'll have a full switch to Xfce4 since it's the only one that can compete with Gnome2 at the moment. I'll probably seriously experiment with various tiling managers around that time too. I'm more excited about the future developments of Wayland which I'm glad Canonical is supporting but not so much gnome3 and unity.
So far, I'm not a big fan of gnome3 (and Unity), but I'm going to give the gnome developers some credit and trust the direction they took. They spent countless hours thinking about what to do - I didn't.
They refactored most of gtk, simplifying many things, and in the long run, when people are done migrating everything, I believe it will turn out for the best.