It's entirely your choice who you empathize with. If you don't act over your indifference, it doesn't harm anybody. It's impossible to empathize with everybody.
The criticism people are raising is about a kind of hypocrisy, even when lying is entirely to oneself.
I know a certain Jesus Christ who would disagree with you. In seriousness though, visiting poor rural American during Trump is a really numbing experience. You see the people there flagging Trump signs everywhere while living in dirt poor conditions. It is hard not to emphasize with them even though they are calling for everything you are fighting against.
After all, the establishment has failed those people. The democratic party has severely failed them. Democracy it self has failed them. I know that really capitalism has failed them and they would find better answers under socialism then under Trump (after all Trump has failed them as well). But you must still understand where this is coming from. That is you must empathize with them, even though their views are wrong.
If you are looking for a clearler symptom that you don't empathize with them, there isn't any.
What is not a moral failing, by the way. But check your "Left" label, because it's probably wrong.