The hypocrisy cry is only an ad hominen fallacy if we are saying that he is wrong because he is a hypocrite. If the accusation is not part of a presumably logical argument, then the accusation does not constitute a logical fallacy.

If I am sitting down for a nice steak dinner with someone and they start telling me that I shouldn't eat meat because of cruelty in modern cattle farming, I'm not going to accuse them of being wrong... but I am going to tell them to shut the hell up. Nobody likes hypocritical whining.

This is an important point that people often misunderstand in internet arguments. Attacking or insulting someone is probably not a good idea in general, but it doesn't constitute an "ad hominem" and doesn't affect the rest of the argument presented unless it's being argued that the other person is wrong because e.g. they smell bad. If it's just mentioned that they smell bad and they're wrong for other reasons, it's not that fallacy.

I do understand that in order for an argument to be fallacious it has to actually be part of an argument.

There's a bit of subjectivity to this. Generally when someone makes an argument and the response is merely "you're a hypocrite", the inference is that person #2 believes that person #1 is wrong and that's the best counter they've got. I understand there's some inference on my part, but that's how I usually see it.

Someone does not have to literally say "You are wrong because you're a hypocrite" in order for the fallacy to apply.

See, looking over this thread I see lots of "You are wrong, AND you are a hypocrite" but relatively little "You are wrong BECAUSE you are a hypocrite".

Maybe I'm giving people here too much credit.

"Ad Hominem Tu Quoque." But when someone criticizes me, I try to remember that the Ad Hominem Fallacy is any suggestion that someone's words are wrong because they are objectionable. The good side is that I can say my words may still be right. But by the same token, the personal criticism may still be valid even if my words are right.

So maybe I should think about what people are saying even if it doesn't mean that what I am saying is wrong.

Oh of course, the sword cuts both ways. Their criticism of you may be perfectly valid and worth noting. But it's still no refutation of your original claim, which I think is how it's generally presented.