If you are a good employee and you handle your departure professionally, you can instead tell your old employer that your new job fell through and go back to your old job.

If you burn your bridges on your way out, then you can never go back. What happens if your new job lets you go after a week? If you are worried about a job pulling an offer a week before you start, you should be worried about it being pulled a week after you start.

You are protecting yourself from an offer being pulled before you start by exposing yourself to extra risk if they pull the offer after you start. It seems like a poor trade.