Not if you can deduct the mortgage interest. At a 32% marginal rate your 3.5% mortgage only costs you 2.4%. At that point you'd be better off putting the money in a high yield CD or savings than in mortgage principal.