It would certainly take more to get to the front page with downvoting. I think the trouble with downvoting is if you have a polarizing story that is interesting the vocal minority could in practice downvote stories that don't align with their worldview. Thus censoring an otherwise merit based story economy.
I was toying with different downvoting schemes but all of them seemed flawed in a fundamental way:
You could have downvotes after a certain threshold.
You could have downvoting carry a different weight based on how much karma you have.
You could have downvotes only after a certain time has passed thus making stories with polarizing opinions not get squashed immediately but still allow for the opposition to have their voice heard.
You could have downvotes only enabled on submissions from members under a certain karma threshold.
I am sure there are many more ways to do the downvoting but I think there is a more fundamental problem with downvotes in that it creates a kind of hostile community. It might be worth having the occasional dumb story come to the front page to keep this place a civil place. Downvotes in the comment section should be reserved for a person that does not further the discussion or is overly rude to another member here.
What about N downvotes per day, possibly based on the log of your karma? This reserves them for something that really bothers me, but limits the power of a like-minded community. It might also help to display the downvotes in red next to the total, since a downvote would be more scarce than an upvote, and may be more informative.
Not a bad idea... I like the attenuating nature of the log of the karma. It would probably need to be coupled with some of the other ideas in the parent post to be fair to new stories with only one karma unfairly down voted by one or two dissenting voices.