How would you work out the cost to society of elite universities? I'm happy to consider other things.
I should disclose that I think the US has very little society, the US is very much about individuals alone or in small groups, not a society as a whole. That's another side of the point I make above. In Europe these institutions are overwhelmingly publically funded and part of a wider social fabric. In the US the opposite is true...
If we consider the cost of something to be "that which has to be given up to have it", then part of the cost of smart people going to elite universities and being funneled into finance is that they don't get funneled into doing something useful [0]. Much as a number of very smart people get funneled into coming up with ways to sell advertising space on the Facebook. These smart people could have gone into any number of industries that are a lot more useful to society (which basically means "other people") than finance and selling adverts.
[0] Harsh and hyperbolic for effect, but it makes a point.
That's a fine measure of cost. But then we have to ask the same question: what is the alternative?
* Those smart kids could go to state institutions. But then we would need to fund those institutions so they could get a comparable education to the one they would get going to an elite university. And we have been cutting those very funds for decades (and increasing the amounts spent on sport stadiums instead of educational facilities). So that won't work.
* we could move them once they graduate from finance to socially useful professions like education. But then we'd have to pay teacher like we pay bankers and no one wants that. Everyone agrees education is important, but not important enough to fund teachers. Same for researchers or medics or social workers or defence attorneys.
People going into finance are making a rational informed choice that we encourage. If we really want less bankers and more teachers, we should pay for it. There are plenty of disillusioned finance workers who would come over (I am one of them), they just don't want to be charged for making the world a better place...
I suspect many people have a broader definition of "cost to society" than just "how much money the public purse has to give to someone".