This remark always comes from people who do not have children. I have never seen a parent express regret over building a family.

I firmly believe this is because it'd be incredibly socially unacceptable to do so.

Most of the parents I'm close to are in the "wow, this actually kind of sucks" category.

Against what I hear about parenting, I'm mostly left feeling unsatisfied and unfulfilled, but what am I going to do? Maybe it'll come later.

I have. Very much I have but I'm also a safe person to share with generally so I find myself the holder of many other peoples secrets.

Presumably, outside of accidental pregnancy, people who would be unhappy with children don't have them and people who would be happy with children do have them. It's not really that surprising if people who choose a particular life path tend to be happy with it.

There was a great NY Times opinion piece on exactly this. You should try and look it up.

I've never heard of a millionaire express regret over making millions of dollars.

That doesn't mean it's impossible to be happy without a million dollars.