Thanks for the recommendation, ordered. Hartley is very valuable to know how we got to where we are, from hand tuned features and RANSAC to more modern full photometric optimization and ML. It answers a lot of "why don't we just... " type questions while still providing the foundations.
Wrobel and Förstner is definitely a different style, to some degrees the two books complement each other. But it focuses much more on the stats side and gaining an intuitive understanding, which to me is really important, and if I wanted RANSAC the algorithms in Hartley are out of date anyways.
This is a complex question. As you pointed out, having a high fidelity map helps to identify the part of the environment that change, creating 'deltas'.
Another reason is that autonomous vehicles take data from a wide variety of sensors, including cameras, RADAR, LIDAR and others.
None of these modalities is perfectly, and there is always ambiguity and drift. High fidelity, high resolution maps provide a strong prior that helps to resolve these issues.
Besides the sensing issues, high fidelity maps also provide priors with respect to planning and prediction of the behaviors of other road users. A busy intersection should be approaches differently to a small backroad.
In the end, autonomy needs both real-time processing and high fidelity maps to perform effectively.
Somewhat different to the suggestions so far, but Thrun et. al's 'Probabilistic Robotics' is a very good applied probability text, with a focus on physical systems.
Plenty of worked examples and problem sets as well.
Not sure why you're getting downvoted, this is a legitimate take.
My thinking is this is two-pronged. It won't make sense to make the infrastructure investment until there are sufficient proof points that this is 1.) something people want and 2.) something that's economically viable.
It's likely that the early geofenced version of passenger autonomy will demonstrate what further infrastructure is needed and how much it would cost.
One interesting side point; dedicated lanes for autonomous traffic are already being proposed on some roadways, particularly interstates in the U.S and highways in Europe. The economic benefits from autonomous logistics (e.g. trucking) are more readily capturable, so the infrastructure investment might make more sense there up front.
He’s getting downvoted because not many people talk about “smart roadways” since it became clear that sensor-based machine learning autonomous vehicles were commercially feasible, and dare I say inevitable.
Good autonomous vehicles don’t work primarily by sensing and detecting the roadway, they work by sensing the environment and having a complete, detailed 3D map of drivable areas, matching their environment to the map as they go along. This only requires annotation of the terrain, not clear road markings or anything like that.
The 3D detail of the environment is the smart roadway. With always-on GPS and an incredibly detailed map of the environment stored on a honking hard drive in the trunk, there’s never a question where you are in the world.
In fact, now that I think about it, this makes reduced infrastructure possible, since you don’t even need things like signs or streetlights in an all-autonomous world, and can even get by with narrower lanes.
I work in autonomous vehicle R&D, and if we had an environment filled with reliable sensors it would considerably simplify our problem.
There is no such thing as a _complete, detailed 3D map of driveable areas_. Most current efforts rely heavily on high resolution, large scale, semantically labeled maps, but on any given stretch of road this is only a first approximation of the environment. Live sensors embedded in infrastructure that could pass along real-time information and updates, particularly from directions our on-board sensors can't capture, would be very useful indeed.
And yes, road markings are part of the semantics we use.
So how much will it cost to upgrade 4.7 million miles of roadway in the US? How long will it take? What does the technology look like? What impact will it have on the economy to close huge swaths of road?
These are all questions to be addressed, along with 'how many of those miles should we upgrade', 'what will we get out of it', and 'how do we make a self-driving car'.
Similar background and profession - putting aside whether an individual would ever want to own one, general purpose passenger autonomy won't happen for probably another decade. Limited domain autonomy will be here in before 5 years, though. It'll just suck.
But the first step to getting good at something is sucking at it.
We are at the peak of the autonomous driving hype cycle, and the trough of disillusionment is going to wipe out a few players, and see the massive consolidation of others. These start-ups are very capital intensive and once the funding environment shifts, it's going to be a big problem for a lot of them.
The initial launches of autonomous taxi services will be underwhelming - relative slow, sometimes frightening (there will be accidents for sure), and with limited deployment areas in limited weather. The novelty will wear off quickly, and then the real work is going to start.
There are basically going to be two ways to survive it - be part of a larger organization that can shoulder a long-term R&D burden to go from 'toy' to 'real infrastructure'. I can see Cruise and Waymo making it that way, unless GM gets cold feet.
The other survivors will be either super lean like Voyage, who went straight to revenue generating niches like retirement communities and rely on being downstream of the technical innovation being done elsewhere (no in-house vehicle or sensor development to cut down R&D costs and move quickly).
Or else they'll be in niches like logistics (e.g. Peloton, Nuro) where the parameters of the game are different, and the structure is more B2B than B2C.
One other play is the autonomy-tech licensing structure like Aurora is trying, but that's a hard sell, especially since they're dealing with German automakers who (from first hand experience in this domain in particular) are clueless about autonomy.
This is an incredibly exciting, risky time to be a part of this new, emerging industry. It feels in many ways like the very very early P.C era, where everything is very much still in play. I'm glad I made the career shift to get there.
Without revealing too much, German automakers have expertise primarily in logistics, with some expertise in automotive design.
There is a vast pyramid of Tier-1 and Tier-2 suppliers that feed into that logistics chain. A lot of those suppliers are promising automakers that they can deliver either components that feed into autonomy, or else autonomy in full. This fragments the effort across literally hundreds of small teams, many of whom have little to no expertise at all in the necessary disciplines.
Additionally, every German automaker also has its own in-house autonomy team, and the relationships between in-house, contracting, and supplier teams is chaotic. Many managers see autonomy as another way to build a small empire and make a name for themselves, and it just results in a huge organizational snarl.
The fact that this is an area no one knows how to execute on yet since it's brand new, coupled with these gross inefficiencies, will mean autonomy efforts from the automakers will be stillborn. They will eventually learn from it though, acquire the right teams, and get on with putting it into their cars just as they're doing with EV.
You're mentioning startups that will be wiped out. The big players in this scene are companies like Google, GM, and Uber. Not startups by any stretch of the imagination.
There are many many startups working on some fraction of the autonomy stack (e.g. Aurora) or the entire vertical (e.g. Zoox). I call out several specific ones in my comment if you care to read it through.
One interesting thing about the 'zero' approach is that it learns through self-play. So you can use a database to bootstrap the learning, but in principle you'd only need the game rules and enough training time.
- Thrun et. al, Probabilistic Robotics
- Multiple View Geometry, Hartley / Zisserman
- An Invitation to 3D Vision, Ma
- Pattern Recognition and Machine Learning, Bishop
-Convex Optimization, Boyd