I don’t understand why self-driving systems don’t have any kind of fallback protections. Like however hard full self driving is to get right, surely a hard rule for “don’t drive at full speed into solid objects, no matter how good an idea it seems at the time to the AI” is not that hard to implement?
Tesla uses only cameras for FSD, not lidar, so it can't tell what's a solid object and what's not. Driving in foggy conditions where everything is fuzzy to the camera isn't helping.
I'm surprised FSD stayed engaged, and I'm also surprised the driver thought it was a good idea to use FSD in this weather.
These types of comments are unhelpful. The expectations on the driver are getting vaguer and vaguer. Can't drive in damp weather? Yet it's Full Self Driving? Tesla wants to market it as powerful AI but leave all responsibility with the driver, don't encourage that narrative.
I'm not a fan of Elon's branding either, but if you've ever actually driven a Tesla, it's very obvious very quickly that "FSD" is not fully autonomous driving, and Tesla itself festoons "Beta" all over the place. So, yes, the driver deserves some responsibility for relying on it in manifestly unsuitable conditions.
100%. Whatever the actual limits, flaws, and failings of FSD, the outsize criticism of the branding is crazymaking. (as I've commented in many of these threads) no one who actually uses the product for even a short period of time is deluded into thinking it is fully autonomous by the labeling.
My pet theory for some of these dramatic fails: the driver is not risk-averse, and knowingly uses FSD at some limit "to see what happens" (I know I do it from time to time, though not in anything truly risky).
It turns out that if you loudly proclaim something is "Fully Self Driving", with a tiny asterisk, people assume it's fully self driving and engage it in all sorts of places where they shouldn't.
And that fault lies squarely with marketing, and much less with individual drivers.
Note that "not disengaging" is one of the critical differences between L2 and L3/L4 systems. The latter are required to disengage when they're outside their safe operating domain like this.
When lidar/radar gets to commodity pricing, I'm certain Tesla will reverse course on "vision only". As it is today, FSD is very impressive, though not autonomous in all conditions.
Camera only perception systems don't have any indication of what is solid vs not. This is why Cruise and Waymo use LiDAR. They have exactly what you're talking about. Even then it's hard -- consider soft vegetation or trash.
Furthermore: tailgater collisions are a thing. You can't just stop on a dime, especially on fast roads (even just 45mph roads).
I’m always amazed when people talk about using Tesla self driving features. Every time I get in a Tesla the driver has the screen showing this little block version of how it perceives what’s around it. It always misses stuff, cars jump in and out, things randomly vanish. Maybe that UI is buggy or doesn’t really show what the self driving would use, but I can’t imagine seeing that and then wanting to let the car drive itself.
There's a probabilistic nature to how the image sensor data is interpreted.
The neural network processes data to classify objects and predict what each object is. It doesn't always have a perfect static representation of each object. It's constantly updating its understanding based on the image sensor data. That jiggly jitter you see is basically the system refining its prediction in real-time.
Other companies have put quite a lot of effort into perception stability because it has a large effect on the downstream quality. It's hard to estimate higher order derivatives like velocity and acceleration well if your position estimates are unstable.
> That jiggly jitter you see is basically the system refining its prediction in real-time.
GP is saying that a "car that jumps in and out" and "things that randomly vanish" do not look very refined. Just like missing a freaking moving train doesn't look very refined.
Actually, no. Look up videos of "range-gated imagers". With multi-hit data for each scan point and appropriate processing, you can see through fog.
This technology has been around for years but not talked about much outside military applications.
Thanks for that, it makes sense. Same techniques are available in radar. This will give you a higher noise floor and also lower signal if fog sits between you and the subject. It probably does improve the situation over visible light, but only up to a point.
The biggest challenge is identifying said solid object. It is only within the last five years that cars with radar have been able to detect stationary objects with some level of reliability, prior to that, if it wasn't moving, there is a good chance that your Adaptive Cruise Control wouldn't detect it, as otherwise they had a tendency to stop for the bin that is on the side of the road, or for parked cars.
A train is a particularly challenging problem, since it is moving at high speed, even with two cameras it is card to really figure out how far away it is. Just because it is easy for humans, doesn't mean it is easy for computers, and vice-versa.
Here’s the problem: Tesla only uses camera input - no lidar or other sensor tech. So it’s a visual model ensemble that needs to decide what a solid object is.
Models don’t reason. They classify and predict. If the prediction is wrong, explaining why is pretty hard, or maybe impossible. So you just need to iterate over and over on the data to try and fix it. It might, and it might not.
My understanding is that it's actually quite difficult to implement a rule for "don't drive at full speed into solid objects" because solid objects are somewhat difficult to reliably identify.
That video also seems like they were driving insanely fast on a foggy day. Quite possibly setting the speed offset way higher than the speed limit. A human could have easily not seen the train as well at that speed.
During fog your speed limit should be determined by your braking distance and visibility. Period.
I wonder whether the autopilot would have performed better if travelling at a more reasonable speed.
He doesn't seem to drive that fast, he drove at a constant speed only to stop at the very last second. Had he been driving too fast, he would have been right into the train and probably not here to post on Twitter.
The car didn't slow down even as the train was clearly visible. An attentive human would have not only seen the train but also the very obvious flashing lights and have absolutely not problem stopping in time.
EDIT: From the link, it seems FSD was set by the car to about 61~63 MPH on a 55 MPH road, which is the speed most people drive on ( https://teslamotorsclub.com/tmc/threads/ap-fsd-related-crash... ). Considering the conditions, it is a little fast, but not unreasonably so. It took about 5 seconds from seeing the train to the crash,
most of it at constant speed, so, a distance of about 140m. At that speed, stopping distance is about 100m, enough to stop in time. And that's for the unlit train, lights can be seen from much further as they are all visible at the start of the video.
It's possible/likely that if such a safeguard is implemented, the self-driving systems would be unusable as it'd trip the safeguard either way too frequently or at unacceptable times.
Obviously they have that. The issue is that they don't recognize solid, stationary objects all the time. Which you'd think would be easy, but turns out it isn't.
