> while cars are ofc different i think the same learnings apply
They are very different. Unit tests and stress tests can be conducted on pure software. You can monitor them and if there are failures you can retry.
Robot software needs all the stuff you use for regular software, but then you have this huge space of counterfactuals that can only be verified using large scale simulation. How do you know the simulator is realistic? That it has a sufficient level of scenario coverage? That the simulation was even run? That the car didn't catastrophically regress in a way that will lead to a crash after 1M miles of driving (which in a scaled service will happen quite frequently)?
Oh and a fault can result in a death for Waymo, but not for Google Search. So that's kind of a major difference.
They are very different. Unit tests and stress tests can be conducted on pure software. You can monitor them and if there are failures you can retry.
Robot software needs all the stuff you use for regular software, but then you have this huge space of counterfactuals that can only be verified using large scale simulation. How do you know the simulator is realistic? That it has a sufficient level of scenario coverage? That the simulation was even run? That the car didn't catastrophically regress in a way that will lead to a crash after 1M miles of driving (which in a scaled service will happen quite frequently)?
Oh and a fault can result in a death for Waymo, but not for Google Search. So that's kind of a major difference.