Sure but, as redblacktree points out, you usually don't leave it behind in conjunction with your passport data.

Also, with RFID you give an attacker more options for gathering more data without anyone noticing. For example at the entrance to an airport you might gather hundreds of identities per hour.