The problem is that these people are going to be living out their entire lives on the ship. While you can maintain genetic viability by supplementing from frozen stock. Maintaining a viable culture and a society that the caretakers will actually want to live in is another thing. Forcing anyone to live a culturally and socially impoverished life is cruel.

I would suspect you would want to have at least 10-20 people in each 5 year age bracket. Then people will have a fighting chance at developing their own social lives and maybe even their own culture.