Physics is also very linear. You cannot know what will happen at time t+1 until you know exactly what happened in time t.

Not exactly true, there are subcases where you can isolate islands of objects and be sure they wouldn't interact.

The computation is, but the scene description isn't. In the general case, any ray can hit any object from any intersection, which means effective use of cache lines is the hard bit.