I've heard this before, of course [1]. And it may be. But it doesn't really matter because far-flung parts of the universe cannot interact with each other. Which assumes universality of some physical laws, like relativity. An assumption which cannot really be tested. Even if aliens have FTL, it will, by hypothesis, only function in their part of the universe, and so cease working in ours.
In truth, I'm quite happy with the physical laws we have. It means that we are protected by a strong natural barrier. It gives us time to figure out how to become a great civilization without interference of a conqueror. Odds are against us[2], of course, but that need not stop us from giving it the old college try!
