Educate. Make up some kind of information that is easy for legit persons to exchange but difficult to look up online. In case of parents/grandparents, it can be information in their life which isn't available online (good luck finding the town my mother was born in online!).

We use this in our family for generations - anyone ringing our doorbell and asking to be let in needed to know the full name of my siblings. We have a similar mechanism in place for our kids.