They're not suggesting that they don't take issue, and so they don't need to take offense seriously.
They're suggesting that the people who conceivably might take issue generally don't and are instead being patronized by and condescended to by privileged, unaffiliated outsiders who assume -- without consent -- to speak on their behalf. And they don't take those people seriously.
It's totally reasonable to disagree with that view, but it's the not the same view your reply tries to engage with.
The thing is, you wouldn't use the slur except to invoke the mean-spiritedness that the people who find the slur offensive associate with the word. If you're using it because you think like-minded people will find it funny that you're using a term other people find offensive, that's still precisely the same mean-spiritedness.
No, you’re expressing a different, more lucid point of view (“the people who conceivably might take issue generally don’t”), which can be engaged with. For example, I would argue that it’s reasonable to take offense on behalf of people who can’t be part of the conversation at hand. (Otherwise it would be fine for whites to spew racist slurs in a group of only white people. If we disagree on that, we’re having the wrong conversation.) I would also point out that taking offense on behalf of others is a time-honored practice (“nobody says that about my little brother and gets away with it!”) But the GP (GGP?) did not say “the people who conceivably might take issue generally don’t.” They didn’t say “no one has standing to be offended by this term.” They just said “it’s not offensive” about a term that is offensive enough that we’re having an entire argument about it. That’s schoolyard-level discourse.
An alternative is to use “on the spectrum”. For example, your s.o. or someone else you’re arguing with is getting on your nerves so you say: “Hey! are you on the spectrum today or what?”
