Why wouldn't there be one within walking distance, or even driving distance (assuming most people would want to drive their kids to nourishing activities)?

You must live in a wealthy place.

In my experience in most places libraries are not common. They use lots of space and money, that usually is needed for more important things like building sewers in neighborhoods that have none.

I live in the US, in the Midwest. However it seems that the average distance to a library across the country is around 2-5 miles.

While probably too far to walk for the average American, it's certainly not too far as to be unreachable.