I think you can work this out backwards: if it was genetic, to a significant degree, then we wouldn't see it around. I believe that the largest predictor for the number of kids is socioeconomic circumstance, and culture.