The alt-right is not "middle-of-the-road conservative". The alt-right is literal-not-figurative fascism with memes and, occasionally, good dress sense. If you consider white supremacists (oh, sorry, white nationalists (oh, sorry, ethno-nationalists)) to be "middle of the road conservatives," there is a profoundly deeper problem in your calculus.
I didn't say that they are middle-of-the-road conservative. The alt-right is a meaningless catch all that currently means "conservatives or nationalists with strong opinions who I disagree with". For reference, I don't belong to the alt right, and I am economically conservative, socially moderately liberal.
What makes you think that Lauren Southern is a white supremacist?
Do you think that someone should be allowed to try to impede human trafficking operations?
Do you think that in the absence of these NGO ships going out to sea that so many people would be drowning on rafts in the middle of the ocean?
Because if Lauren is right, then you are suggesting that incentivizing a life-endangering risk for the purposes of being taken in to human trafficking is an ethical action.
Most likely the association goes alt-right->white nationalism->fascism.
The most common refutation is that the alt-right is not a white nationalist movement, but that is entirely beside the point. The alt-right is absolutely a nationalist and isolationist movement. It was also certainly started by white nationalists and white supremacists, and those two groups are still fully present and influential in the greater movement.
Nationalism, race-cleansing and isolationism are not necessarily fascism although they are intimately related. There are a great number of fascists in the movement, including many who call themselves libertarians, but there are also a lot of people who are anti-authoritarian. Actual, literal fascists believe the government may take ultimate control of the economy and people to accomplish the goals of the nation, particularly in times of "emergency", which most of them believe this is. They do not believe in democracy under any circumstances, only in an umbrella of direct dictatorial control.
So, taking that all together, does the alt-right overall meet the definition of fascism? Yeah, I think so. The alt-right's mandate is to prevent jobs from leaving the country, end international trade (mercantilism, in fact), and to bar immigrants. They're a little wishy-washy on foreign intervention and have strong opinions both ways. They want these goals to be accomplished through direct action by the president- convincing or ordering corporations to hire inside the country, and ordering an end to imports. They want to ban immigration for the safety of the country and many want to additionally protect cultural homogeneity. Trumps cabinet very much recalls Mousollini's corporatism, with energy company reps in charge of the EPA and secretary of state.
Trumps whole thing is a lone wolf against the rest of the government, trying to take control and convert it to what it should be. That applies to the government, economy, and society. His alt-right supporters support that in one or more categories. To the alt-right, Trump is the only one who can restore jobs where they should be, restore the people to who they should be, and fix how people act and the expectations they have of each other. That's pretty textbook fascism.
There's a lot of confusion out there due to a relatively brief appropriation of the term "alt-right" by a political tendency that is now more commonly termed "alt-lite".
The dividing line that's emerged is "civic nationalism" (alt-lite) vs "ethnic nationalism" (alt-right)
ie: anybody can be an American provided they adopt the right values and behave as an American, vs only white people can be American and everybody who is not white (+ jews) needs to be made to leave.
Needless to say, the latter tendency is especially packed with literally textbook fascism.
Probably true. So we fight them. It stinks and it sucks, but it's on all of us to do so to the best of our ability and those of us who were dealt a more fortunate set of life's lottery balls (and I say this as an affluent, straight, white, cis American) have a duty to use our good fortune to help others.