True in general, but when he bought them, they were priced very low. Even after he bought millions (presumably) of the swaps, other investors thought he was an idiot and didn't follow suit, so the price of the swaps probably didn't go up much. It was only after the sub-prime fiasco got into third gear that other investors began clamouring for the swaps.

By all rights, his initial foray into buying the swaps most likely had minimal effect on the price. Ultimately he felt they were incorrectly priced by the market and made a bet on them.