You've already described the solution to the problem: we upvote things that we think are unfairly downvoted. If you say mildly controversial things, you can often watch this happen on your comments. Not that I wouldn't like reasons attached to downvotes and flags...