Is the desire to have kids (as opposed to just sex) genetic? Given that pretty much everything about our personalities seems to be genetic to some extent, it seems pretty likely.
I think you can work this out backwards: if it was genetic, to a significant degree, then we wouldn't see it around. I believe that the largest predictor for the number of kids is socioeconomic circumstance, and culture.