> In the case of Waymo, we have some tentative supporting evidence from this and other studies Waymo has run. However, that is still insufficient, even ignoring the lack of audits by non-conflicted parties, to strongly conclude Waymo is safer than a human. The evidence is promising, but it is only prudent to wait for further confirmation.
You are not making a distinction between concluding unsafe and not being able to conclude safe. It is standard practice to not presume safety and that positive evidence of safety must be presented. Failure to demonstrate statistically sound evidence of danger is not proof of safety. Failure to disprove X is not proof of X. This is a very important point to avoid fallacious conclusions on these matters.
To discuss your specific points. Yes, the data is promising, but it is insufficient.
Traffic fatalities occur on the order of 1 per 60-80 million miles. Waymo has yet to reach even one expected traffic fatality yet. They appear to be on track to doing better, but there is not enough data yet.
The reports Waymo present are authored by Waymo. Even the Swiss Re study is in cooperation with Swiss Re, not a independent study by Swiss Re. The studies are fairly transparent, they point to various public datasets, there are fairly extensive public reporting requirements, and Waymo has not demonstrated clear malfeasance, so we can tentatively assume they are “honest”. But we have plenty of examples of bad actors such as Cruise, cigarette companies, VW , etc. who have done end-runs around these types of basic safeguards.
Waymo operational domain is not equivalent to standard human operational domain. They attempt to account for this in their studies, but it is a fairly complex topic with poor public datasets (which is why they cooperated with Swiss Re) so the correctness of their analysis has not been borne out yet. When Waymo incorporates freeways into their public offerings this will enable a less complicated analysis which would lend greater confidence to their conclusions.
Waymo is still in “testing”. As their processes appear to be good, we should assume that their testing procedures are safer than should be expected out of actual deployment or verification procedures. That is not a negative statement. In fact, it would be problematic if their “testing” procedures were less or even equal in safety to their deployment procedures. That is just how testing is. You can and must apply more scrutiny to incomplete systems in use and prevent increased risks especially while under scrutiny otherwise you are almost certainly going to be worse off in deployment where there is less scrutiny. We have yet to see how this will translate out to deployment, so we will need to wait and see if safety while under test will appropriately apply to safety while in release. This is analogous to improved outcomes for patients in medical studies even if they are given the placebo because they just get more care in general while in the study.
Anyways, Waymo appears to be doing as well and honestly as can be determined by a third party observer. I am optimistic about their data and outcomes, but it is only prudent to avoid over-optimism in safety-critical systems and not accept lazy evidence or arguments. High standards are the only way to safety.
Assertion: "50M miles shows that Waymo is safer than humans".
Counter-point: "That's false because Cruise had an accident for which they were at fault".
OP: "The existence of a case or some cases where a self-driving car caused injury has zero value. What matters is the rate of cases per mile driven."
You: "You do not get to counter-argue."
Yes, they do. OP's point is valid. One can't refute the original assertion by citing one accident by another company. It's a logical fallacy (statistically speaking), and a straw-man (Waymo can't be safe, because other self-driving cars have been found at fault). The validity of the original claim has nothing to do with an invalid counter-claim.
> However, that is still insufficient, even ignoring the lack of audits by non-conflicted parties, to strongly conclude Waymo is safer than a human.
When you have a large, open, peer-reviewed body of evidence, then yes, that's exactly what you get to claim. To reject those claims because Waymo was involved is ad-hominem. It's not how science works. It's not how safety regulations or government oversight works. If you think it's insufficient, you can attack their body of work, but you don't get to reject the claim because they haven't met some unspecific and imaginary burden of proof.
> In the case of Waymo, we have some tentative supporting evidence from this and other studies Waymo has run. However, that is still insufficient, even ignoring the lack of audits by non-conflicted parties, to strongly conclude Waymo is safer than a human. The evidence is promising, but it is only prudent to wait for further confirmation.
You are not making a distinction between concluding unsafe and not being able to conclude safe. It is standard practice to not presume safety and that positive evidence of safety must be presented. Failure to demonstrate statistically sound evidence of danger is not proof of safety. Failure to disprove X is not proof of X. This is a very important point to avoid fallacious conclusions on these matters.
To discuss your specific points. Yes, the data is promising, but it is insufficient.
Traffic fatalities occur on the order of 1 per 60-80 million miles. Waymo has yet to reach even one expected traffic fatality yet. They appear to be on track to doing better, but there is not enough data yet.
The reports Waymo present are authored by Waymo. Even the Swiss Re study is in cooperation with Swiss Re, not a independent study by Swiss Re. The studies are fairly transparent, they point to various public datasets, there are fairly extensive public reporting requirements, and Waymo has not demonstrated clear malfeasance, so we can tentatively assume they are “honest”. But we have plenty of examples of bad actors such as Cruise, cigarette companies, VW , etc. who have done end-runs around these types of basic safeguards.
Waymo operational domain is not equivalent to standard human operational domain. They attempt to account for this in their studies, but it is a fairly complex topic with poor public datasets (which is why they cooperated with Swiss Re) so the correctness of their analysis has not been borne out yet. When Waymo incorporates freeways into their public offerings this will enable a less complicated analysis which would lend greater confidence to their conclusions.
Waymo is still in “testing”. As their processes appear to be good, we should assume that their testing procedures are safer than should be expected out of actual deployment or verification procedures. That is not a negative statement. In fact, it would be problematic if their “testing” procedures were less or even equal in safety to their deployment procedures. That is just how testing is. You can and must apply more scrutiny to incomplete systems in use and prevent increased risks especially while under scrutiny otherwise you are almost certainly going to be worse off in deployment where there is less scrutiny. We have yet to see how this will translate out to deployment, so we will need to wait and see if safety while under test will appropriately apply to safety while in release. This is analogous to improved outcomes for patients in medical studies even if they are given the placebo because they just get more care in general while in the study.
Anyways, Waymo appears to be doing as well and honestly as can be determined by a third party observer. I am optimistic about their data and outcomes, but it is only prudent to avoid over-optimism in safety-critical systems and not accept lazy evidence or arguments. High standards are the only way to safety.