While in the plane, the rules change (and CHANGE A LOT). Refusing to fly is actually possible... On condition of removal of the passenger involved (if you claim false reason, that's perjuring yourself for no good reason). And the captain has full right to deny flight to specific passenger.