Exactly. I'm not a fan of any of the "popular" desktop environments. I liked Gnome 2 and didn't mind using it (RHEL desktops) but for several years XFCE has been my favorite. It has all the features I need but isn't too intrusive and stays out of my way. I'm not really a big Ubuntu fan either but I can't complain too much about Xubuntu.
It's also the only one I've tried that allows me a taskbar per screen with only window entries from that screen, something I've done since Gnome 2, You can sorta do it with plugins with Gnome but they break all the time, KDE can do it in theory but KDE is awful (I'd like to like but never have).