I'm surprised the candid picture case is so compelling. I can pull my phone from my pocket, double click the power button to enter camera mode, then tap volume button to shoot, very quickly. Under 2 seconds easily.
Guessing based on what I see: The act of simply pulling out your phone can break the moment. Kids tend to zero in on it immediately, making it the new center of their attention.
I dunno, I have lots of photos of kids on my phone just like I described. It's true, sometimes they do do that, but also, because I take so many, they got over it.
Don't forget having to frame the shot, making sure it's in focus, ensuring your subject doesn't get distracted by your motion, and not fumbling your phone in the hurry to capture the moment... as opposed to pressing a button on the glasses you're already wearing.