Taking his point to the extreme why use 8 cameras? just use 4? 1? One photo-diode?
Cameras can also fail at run-time there can (and is) be variability in how they're mounted, in the lenses, in the sensors. They can get blinded or not get enough light. Their cabling can fail random components can fail.
Tesla has claimed that vision outperforms vision+radar but anecdotal reports don't seem to support that conclusion. IMHO these technologies are not directly replaceable, but are complementary. It's like you can't replace your ears with your eyes (yeah, you can read lips, if they're visible).
But sure, there is a sweet spot. Is Tesla really optimizing for best performance at any cost or are they optimizing making more money and selling that to us as an improvement? That's really the question and I don't think we got a frank answer there.
I would also add that Tesla's sensor systems, while perhaps higher quality, are not exactly new ideas. In one form or other laser/radar-based systems have been in cars going back to the 90s for early collision avoidance, automatic cruise control, etc.[1] Longer in other applications.
At least one study seems to suggest those sensors when deployed in automatic emergency braking systems do have a measurable impact on collisions.[2]
Let's say the failure rate on the sensors was 1 in 100 (I'd be shocked if that many were defective). That means 99 other Teslas are using mutli-sensor systems and not driving with degraded capabilities. It's an asinine claim that doesn't pass basic logic tests. The only way they weren't a substantial improvement is if Tesla's measurements were conducted in only the absolute most ideal conditions for cameras and no other scenarios.
> Is Tesla really optimizing for best performance at any cost or are they optimizing making more money and selling that to us as an improvement?
More likely they had a fixed budget and optimized with that constraint, if they made a rigorous decision at all. But this is guesswork.
I'm not speculating about how Tesla made the decision, just commenting on Karpathy's answer. His answer is correct even if it isn't true, i.e. even if it isn't what Tesla actually did.
There are plenty of well-known analogs, like the mythical man-month. We all know that throwing more x at a problem is routinely counterproductive, even without cost as a constraint.
It's like the joke about the mathematician in the hot air balloon ... His answer is correct but it's not useful. It is correct there is some optimal solution short of an infinite number of sensors/technologies and larger than no sensors. The argument that Tesla is converging on the optimal solution vs. the more or less known reality that they couldn't get the components they needed to build enough cars is weaselly. But hey, necessity is the mother of invention. Also he can't actually share anything from his work in Tesla because presumably he's under NDA but he's gotta say something.
> It seemed like all the "full cost" negatives Andrej mentioned were related to Tesla's ability to execute, and not what would actually produce better results.
This is objectively wrong, and it's the only substantive part of the discussion. The rest is fantasizing about things nobody actually knows ("It's media training!") and imputing questionable motives to someone who hasn't done anything to deserve that ("He only cares about Tesla's bottom line!").
You could take any one single point in a complex multifaceted argument to the extreme and basically strawman it to death. But that’s not helpful.
I believe his point was to provide a new perspective on the problem, not to reduce the problem to a single reason. I highly doubt the only reason Tesla chose to use vision only in the short term was motivated by a single datapoint.
Even if it was the most important point... in this one person (on a large team’s) mind... it doesn’t necessarily mean it was the most important in the sum of the complex process it took to get to the decision.
So I don’t really see the value in taking it to the logical maximum because it’s not only illogical that they would be evaluating this one idea in isolation but even on its own they would still be balancing the optimal performance they got from x vs the optimized value they got from y, then compare it to the teams ability to work with both x+y(+z) at the same time.
For ex: You’d probably need 8 cameras pointing different directions vs one highly capable rapidly spinning LiDAR to even compete with it, so why even ask? These problems a) always have context and b) can't be so easily simplified and broken down.
Although you might make a good point that Tesla used this same poor logical-maximum reasoning to determine why not get rid of ALL sensors besides vision.
At least 6 are needed to get a 360 degree view around the car, which obviously is necessary. Think of the 8 cameras as a single better sensor. It's a question of having one very good sensor then or many to fuse.
Cameras can also fail at run-time there can (and is) be variability in how they're mounted, in the lenses, in the sensors. They can get blinded or not get enough light. Their cabling can fail random components can fail.
Tesla has claimed that vision outperforms vision+radar but anecdotal reports don't seem to support that conclusion. IMHO these technologies are not directly replaceable, but are complementary. It's like you can't replace your ears with your eyes (yeah, you can read lips, if they're visible).
But sure, there is a sweet spot. Is Tesla really optimizing for best performance at any cost or are they optimizing making more money and selling that to us as an improvement? That's really the question and I don't think we got a frank answer there.