Get the 2nd factor... that's NOT sms.

SMS as a second factor is fine. As bad as it is it can't reduce your security compared to just a password.

SMS as an account recovery mechanism is the problem.

SMS is the 2nd factor. Actually it's worse than just a 2nd factor because a compromised phone number can usually be used for password recovery

You are confusing two different things.

The problem is one factor account recovery, because it means you have one factor auth.