Agree wholeheartedly - I went jogging up in the santa monica mountains in brentwood (near la) yesterday morning with my fivefingers (sprint), on a trail with lots of loose stones and gravel. The feeling of walking on gravel and stones was incredible, as long as I didn't step on anything sharp directly (but that'd be enough for it to hurt in any shoe). I liked it even more because it's something I enjoy doing barefoot but usually don't because it's harsh on my feet :)
What's up fellow Angelino! I've been thinking about these things for surfing since there are so many damn rocks everywhere. I don't recall a time whenever I've gone into the ankle snapping, toe jamming continental shelf and felt anything incredible about it (other than on the pain scale) :D, so the more I read, the more interesting these look. How do they hold up against sweat / moisture?
Anyway...I honestly couldn't say. I don't surf, I only swim, and I've never gone swimming with mine. Other than that, I don't think I've noticed anything weird about how they handle sweat or moisture compared to other shoes...they get wet when you're wearing them somewhere wet, although I find myself sweating less on my feet with these because I'm not wearing socks with them (although you can wear toe socks) and they don't smell yet (they're only about a month old). The upside is tossing them into the washer to clean them.
I have the sprint version though, vibram makes a "flow" version made of neoprene that might be more appealing to you if you surf.