No, it's the opposite. The advice we were given was that it was very important that their first encounter with the allergens was to eat them, and not to allow skin contact or anything before that. This meant that with my youngest we had no peanuts in the house until she was old enough to eat solid food, and peanut butter was the first food she ate. It's still her favourite food!
If you're avoiding skin contact, then you might not have time to wipe the peanut butter mess off your hands as you rush to rescue your kid who's got into trouble.
It wasn't about them not being in the house - it was about not eating them, and particularly not feeding them to her 2 year old sister. We didn't throw them away - we just didn't buy any as we weren't going to be eating it for a while.
Babies change fast. I can certainly imagine it. You won't know when exactly the flip happens, so you err on the safe side. Also, spread of stuff though the air and by hands: I thought we just spend lockdowns realising that crap spreads far and wide.