I don't remember where I saw it, and I'm struggling to find it now, but there was some interview where he explicitly claims to not be racist and claims to not hate black people, and even claims to not be a white supremacist, but that he merely believes that the races shouldn't mix and that America should be a white ethno-state.
He knows that racism is not socially acceptable, so he makes bad faith claims of not being racist while expressing obviously racist views. And the problem is, it works. It doesn't take much searching to find racists that don't think they're racist. The fact that people don't think Donald Trump is racist, despite a long list of racist remarks and actions [0], is proof of that.
And of course, later, Milo Yiannopoulos later released an audio recording of an explicitly racist rant. [1]
