> Maybe the other person is being vastly overpaid.
Maybe; but if you think like that, then no sample can give you that information. Because you won't get any absolute reference point either way. In any case, such situation is a good reason to talk about your salary.
So your argument is wrong; the sample size is not important (that doesn't mean it's useless, though!). Either you have some reference point, and then you can judge the fairness of the salary (even from one data point), or you don't have a reference point, and then no sample size will help you get it.
Let me rephrase that: If you get told that one person is earning far more than you, then you don't know if you're the anomaly or if they're the anomaly.
If you get told that a large representative sample of people all get paid far more than you, it suggests that you're the anomaly.
There is something wrong with your thinking, but I can't quite put my finger on it. Let me try:
Either you assume a prior distribution of salaries or not. If you don't, then you don't know mean salary, and so group of any size will not tell you if you're anomaly. Because they can all be anomalous as well.
On the other hand, if you assume a prior distribution (which is pretty much what Bayesian statistics does), then even one sample will modify the prior, and you gain information (i.e. "suggestion that you're anomaly or not").
Of course more samples is always better, but if you can make conclusion from multiple samples, then you can make conclusion from one sample.
It seems to me that in the first case, you're saying we cannot assume any prior, but in the second case, you're doing exactly that - assuming that there is a mean - 1st moment of the prior distribution and maybe even other moments - which indicate whether an observation is anomaly or not.
Maybe; but if you think like that, then no sample can give you that information. Because you won't get any absolute reference point either way. In any case, such situation is a good reason to talk about your salary.
So your argument is wrong; the sample size is not important (that doesn't mean it's useless, though!). Either you have some reference point, and then you can judge the fairness of the salary (even from one data point), or you don't have a reference point, and then no sample size will help you get it.