Approval voting with a binding "None of the above are acceptable" would be a great improvement over the status quo. I suspect that NOTA would have beaten both Clinton and Trump.
I think you are moving this up with ranked choice. Approval voting allows you to vote for every option either ones or not at all. So you either approve of the option or not. If you want you can vote for everyone. However, that's the same as voting for nobody. You could however also vote for everybody except for one option. That would be the equivalent of voting against somebody. Example ballots could be:
Voter one:
Green - yes; dem - yes; rep - no.
Voter two:
Green - yes; dem - no; rep - yes.
This would result in the green party winning in the above example.
The vote would be very close if not identical to approval ratings we frequently see for various candidates. This would have probably gotten us Bernie, since he was the only candidate running at all with favorable approval ratings. Both candidates who made it beyond the primaries were disliked by the majority, which is unfortunate to say the least.
As an added benefit we could completely leave off primaries if we want. There is very little spoiler effect in this system.
Because there is much less reason to strategically vote for your second choice in order to prevent a candidate you definitely don't want. A friend of mine who is very active in this area also claims there are studies about this. But I've honestly never read them. It seems intuitively true to me. Which I acknowledge is not ideal.