Yeah but how long should I spend on software that I know nothing about the quality of if it's not easy to get up and running? Do you spend a day? two days?

While it's mostly just from experience I find that virtually no one makes a decision based on how easy it is to get up and running with a piece of software but it is an impression that stick's in people's minds and gets repeated when they talk about it later.

I completely agree, and I'm not saying that MongoDB is better because it has an easier install. The question was just "Why did you go to MongoDB over Riak?", and I did because I was able to start evaluating it faster, and discovered that it met my needs handily, deprecating the need to wrestle with my Erlang install any further.