This is not necessarily true on an individual level though. Driving skills, judgment, risk-taking, alcoholism, etc. are nowhere close to evenly distributed.

It's likely we'll go through a period where autonomous vehicles can reduce the overall number of accidents, injuries, and fatalities if widely adopted, but will still increase someone's personal risk vs. driving manually if they're a better than average driver.

But we don't live by a purely utilitarian principle of ethics. "I'm sorry Mrs Jones, I know your son had an expectation of crossing that pedestrian crossing in full daylight in a residential area without being mown down by a machine learning algorithm gone awry, but please rest assured that overall fewer people are dying as a result of humans not making the common set of different mistakes they used to make".

All sorts of other factors are relevant to the ethics: who took the decision to drive; who's benefiting from the drive happening; is there a reasonable expectation of safety.

Yeah yeah, I get it. Moral philosophy is full of gray areas.

I don't see how that remotely supports "AI cannot be allowed to make mistakes some humans would not", which was your decidedly absolutist position above.

How about: "We should allow autonomy in most cases, though perhaps regulate it carefully to better measure its performance against manually-directed systems"