I mean what you say sounds pretty reasonable. I wonder it also sound reasonable if I say "Well black men have it really easy, they have great skin tone and better physique than average. All they have to do is to not dress like punk or wear hoodies and have hair cut short so they don't have run in with police often"
There is an element of truth in this- just remove the word 'black' from your comment,as it's not necessary: I see a lot of young adults(I was one of them some years ago), who dress and behave in public in a way that it almost guarantees unnecessary attention(not in a good way), including from police. Dress like a normal person,not like some low life drug dealer,who spent 10 years on heroin, drop the idiotic language -'oi bruv' and so on and suddenly life becomes better. Even the hardcore criminals were fast to realise that it's much better to wear a suit and a tie than a tracksuit and and a hoody.
Anyway, colored men (mainly indian guys in my wife's practice) significantly increase their attractiveness by dressing nice and well groomed, so same rules apply.
But what I get from your explanation is that it's all the womens fault of not looking past that. Well maybe it's time for you to face reality.