I believe the main reason people are scared about self-driving cars is that they all run the same software, so any unresolved bugs might cause multiple cars to be involved in accidents, which is something we don't expect from human drivers.