> If it wouldn't be the most used, would it be used as point of reference and called the tonic?
Yes. The tonic is determined by what the key is, not what chord is most used. (It's hard to establish the key without slipping the tonic in somewhere, but that doesn't mean it needs to appear frequently.)
> I-V and IV-V seems to be the same with the point of reference moved to IV. And IV-bVII-V is I-IV-V.
I don't think this is right. V-I has the root moving by a 5th, while IV-V moves by a second. And the second example can't be right because the relative distances of the roots of the first and last chords are different (again, a second versus a fifth).
> Is a fixed point of reference really needed? Doesn't it just add unnecessary complexity?
Yes. The tonic is determined by what the key is, not what chord is most used. (It's hard to establish the key without slipping the tonic in somewhere, but that doesn't mean it needs to appear frequently.)
> I-V and IV-V seems to be the same with the point of reference moved to IV. And IV-bVII-V is I-IV-V.
I don't think this is right. V-I has the root moving by a 5th, while IV-V moves by a second. And the second example can't be right because the relative distances of the roots of the first and last chords are different (again, a second versus a fifth).
> Is a fixed point of reference really needed? Doesn't it just add unnecessary complexity?
Yes, it's a useful analytical tool.