I assumed the same, but I've seen in CNN's live coverage on election day (unfortunately I can't find it now) that in an exit poll (I don't remember where) only 2 of 5 Trump voters said they voted against Biden (and not for Trump), but 3 of 5 Biden voters said they voted against Trump (and not for Biden).
That's how it's been for at least decades. Democrats voted for Gore from hate of Bush. So this is a structural problem you're stuck with. Unless you come up with a stronger force than a pandemic with 100,000 fatalities.
Ranked choice elections, or something that gives seemingly fringe candidates a chance. Skip primaries for party candidate and use the extra time for more debates.