If you assume FTL travel will never be developed then distance and time are simple limiting factors. How do you keep a cohesive civilization going when communication takes 200 years? Or even just 20? Here on Earth entirely new languages and cultures arose across distances that wouldn’t even cross a state line when communication was limited to a small handful of travellers and merchants. Any colony further away than 5ly would quickly diverge. I’m pulling that number out of my hat but I’m sure you could figure out the effect of time spent in journey on willingness to travel. Not many people would commit significant chunks of their lives to interstellar business trips. Radio communications won’t solve it either since they’d be out of date and essentially one way if it took decades to get a response. No I think any interstellar colonization effort would immediately create competing civilizations distinct from their homeworld.
Consider a colony of bacteria multiplying by splitting. Each new pair of cells is independent and do not cooperate. Some die, some stay put. Nonetheless, the “colony” spreads and explores new territory with zero coordination of these activities. Certainly not an intelligent centralised leadership!
Even if our first interstellar colonies diverge immediately and some even turn into reclusive hermits, some may expand, repeating the cycle.
Hard to say what would happen but I think we still need to avoid the assumption that each star system remains relatively static especially over very long periods of time. You also need to consider the purpose behind colonization, if it is to spread the existence of your civilization to new worlds then no one says those worlds must be uninhabited.