What I would love to see is an end to end neural network soln. On one end camera input comes through, on the other outputs for speed and steering angle.
But rather than a black box, it's explainable what the different layers are doing. If neural nets are turing machines then we should be able to compile some parts of the net from code.
Then the net is a library of layers. Some Layers trained with back prop, some compiled from code.
I'm working on adding the throttle. This is difficult because you need to drive the correct speed and stopping or running off course can mess up the training data. This project was inspired by Otavio's carputer which does predict throttle, steering angle, and odometer.
But rather than a black box, it's explainable what the different layers are doing. If neural nets are turing machines then we should be able to compile some parts of the net from code.
Then the net is a library of layers. Some Layers trained with back prop, some compiled from code.