Regarding drivers that are drunk or otherwise driving illegally-- why would you want to exclude them from such a study, if you can not exclude them from real traffic?
If our current system of checks and incentives is not able to keep them off the road, then you need to take them and the accidents they cause into account when comparing driving safety with an autonomous system.
Also note that the margins are too big, since only about a third of accidents involve intoxicated drivers, and the accident reduction rates from the paper are much higher than that.
I don't want to come across as harsh or accusing, but the "Methodology" section contains a lot of detail (and a more nuanced discussion could be had without disregarding it entirely :P).
Because they don't disappear even with half the non-illegal vehicles being autonomous — some crashes also happen due to avoidance manoeuvres.
So the fairer comparison would be to say that if existing SDVs were instead being driven by human drivers, we would have seen ~60 property claims more and ~40 bodily injury claims more (forgot the exact numbers from the paper), which is what, like 0.1% improvement in claims overall (certainly still a great result for the small comparative number of miles travelled, but doesn't jump off the page really).
But because the miles traveled are so far apart, nothing is really a "fair" comparison.
But.. they would disappear? Not all of course. But some drive under the influence because there is no other convenient way to get home after a night out. And Waymo solves that!
And they are highly aware of their "small number statistics", they literally provide uncertainty ranges. For both SDVs and HDVs. And their values are consistent with increasing data. I.e. they ~only get smaller uncertainties but no significant drift. But they are statistically significantly better (which wasn't the case two years ago)
Are you suggesting Waymo would become available where cabs aren't due to economics, or that Waymo-like technology in privately owned vehicles would solve that? Because I wouldn't buy the former, but I can see how the latter would, yet we are far away from that.
I am not referring only to "small number statistics", but to obvious miscomparisons. We don't want SDVs to beat "average human driver", because average is worsened by impaired humans — we need them to "beat" at least an average, non-impaired human. Going by the numbers alone, they would achieve that too, but not by the same amount.
But more importantly, we need them to compare both in identical driving conditions, which is where most of my complaint really is. Unless Waymo has driven in the same amount (proportionally) of rush-hour traffic, road-works, risky-drivers-around conditions, parking into tight spots and garages... of miles out of their 25M miles as their human counterparts, we could very well be looking into Waymo-in-simpler-situations vs humans-in-all-situations. And the numbers would be meaningless, which to me they seem they are.
And it is important that they have achieved similar success with 6x the miles, but they need to account for more challenging conditions. And while some tricks can work while they are in small numbers (eg. avoiding road works by re-routing), it will lead to congestion elsewhere when such technology dominates the traffic.
I would also note that it is impressive, regardless of the conditions they operated in, that they were involved in exactly 2 "bodily injury" claims over those 25M miles, and both of those appear not to have been the fault of Waymo.
If our current system of checks and incentives is not able to keep them off the road, then you need to take them and the accidents they cause into account when comparing driving safety with an autonomous system.
Also note that the margins are too big, since only about a third of accidents involve intoxicated drivers, and the accident reduction rates from the paper are much higher than that.
I don't want to come across as harsh or accusing, but the "Methodology" section contains a lot of detail (and a more nuanced discussion could be had without disregarding it entirely :P).