I think that's the point: You can't make an objective decision about it, so it turns into a bunch of feely stuff, and that's really bad to base hiring decisions on.
Because it causes you to skip over qualified candidates simply because they don't want to play ping pong all afternoon or partake in the office kegger.
