LIDAR data might be more sparse but it is also more reliable, being an active sensor so it isn't affected by night.

The real way is to use the LIDAR to add to the depth probability distribution in the stereo depth estimation. This way you aren't throwing any data away. LIDAR often gives probabilities as well, for example, and this can be used to eliminate reflections.