Definitely not just a rich kid thing. The camps are usually run by small-ish organizations like the local church. And considering the point is to bring in children, there are often options for those who need financial assistance.
Besides, rich kids go on overseas vacations during the summer.
Well in most cases losing one of two incomes in the household costs a lot more. In the US, anyway, it is a lot more common to have a stay-at-home mom in the upper incomes; the working class is mostly every-adult-must-get-a-job (or two or three). Occasionally there are grandparents available, but in some cases they're in another town, and in some cases they are deceased. Summer "camps" are definitely not a rich thing (in the US, anyway).