If malware is already on your system then your system is already compromised. It could also patch firefox or download a firefox with signature verification disabled.
Or it could just send your password store file to some server in russia, encrypt your harddrive and extort money from you.
Really, if malware is on your system then some extension sideloading is not really a big concern in the grand scheme of things.
I totally cannot follow that argument. To me it's like being relieved that your wallet hasn't been taken after someone knifed you and you're rapidly losing blood.
This isn't my argument - it's the argument used by Chrome, Firefox and other browsers. It's why browser plugins like NPAPI are being disabled (Chrome did it earlier this year).
Yes, local attacks are not impossible without this, but the point is to make them harder. A simple switch that opens up a lot of entry points is an easy target for malware.
Some malware might not need an easy target, but you at least prevent some malware by removing it. The harder it is, the fewer attacks will succeed.
But you can't add such a switch - if it's there, malware can access it. A switch might prevent other problems, but not that main one.