I think there is more to it than just seeding the RNG consistently.

For example: what if you have a minor change in the area of the screen that got the first primitive (and hence used the first few RNG numbers)? Then all the remaining primitives will use different RNG numbers.

I think the problem becomes more difficult: you want to find the primitives that result in the minimal change wrt the previous frame.