In my experience proprietary software is often far more complex, in fact in another league than most FOSS SW for now (source: have installed and reinstalled so many Oracle databases, Enterprise Manager and APEX installations I have lost count.)
Depends on the software and author. Case in point: sendmail; virtually everything ever made by RedHat (the software is free, but there's no useful documentation if you're not a paying customer… cf. NetworkManager, PulseAudio, GlusterFS … only systemd seems an exception, for now).
Good point! However, I would agree that server/system software that is "best-in-breed" tend to be FOSS. But I'm afraid I have no studies or other hard facts to support my case.
Configuration can be a pain. Sane defaults help and smart UIs help not to mention good documentation.
I've been playing with some software over the last week that has wiki for documentation and an active user forum with people willing to help. But it's completely disordered and incredibly hard to discover. Half the battle with usability is making it easy to use and understand.