> if you judge everyone by (objective) standard tests, Goodhart's Law takes effect, and we breed a generation of highly optimized test takers.

As opposed to a generation of people highly optimized for a fuzzier metric that nevertheless exists. I know a bunch of graduates from elite universities who all just happened to be really into debate/mock trials/mock UN, competed at the national/international level at obscure sports, founded a school newspaper (we had five new school newspapers every year because everyone wanted to be a founder, few of them last once the founder graduates), "founded" a "business", etc in high school[1]. They're all in consulting/banking/law/ML now. Mildly obfuscating the metric doesn't actually make Goodhart's law go away, it just means you have to be in the right crowd to game the metric.

[1] I, on the other hand, actually liked maths and became one of your "highly optimized test takers". Funnily enough, all the elite American schools said no, while Oxford said yes. I am now a PhD student (in maths) at one of the elite American institutions that rejected me as a high school student.[2] If you have a scarce resource, and a system that distributes it, people are going to game the system. The trick is to have better tests so you can't study to the test without actually, you know, learning.

Take a look at the SAT (subject) math section. That's the test that should be distinguishing between the student that goes to Harvard and the student that "merely" goes to Georgia Tech. Obviously it can do no such thing, it's way too easy. This is by design. The American system does not want to distinguish between the top 3% and the 0.5%, because then Harvard would have to take the poor genius over the solidly talented but unexceptional rich legacy kid whose dad is worth billions and will donate tens of millions.

Now take a look at the MAT, the maths test that tells you if you'll be going to Oxford or Imperial. Or the STEP, the analogous test for Cambridge.

[2] Disclaimer: I may be a little bitter to this day. In my defence, I teach the wannabe consultants now, and while some of them are legitimately very competent, boy does the system have false positives.

Sounds like a better system is to have minimum requirements, whereby those that meet them are likely to graduate. And then a lottery.

Everything else is just a new set of rules to game.

Rice University had about 600 freshmen when I matriculated over 2 decades ago and they rejected 16 perfect SATs that year. I feel for you, though I know that if you really are so into math then maybe this was a much better path for you than doing undergrad at an American institution where the goals in undergrad are usually broader and fuzzier. Contrast that with grad school where I’d be more inclined to be biased for the top student in the specific subject. My alma mater is a top-20 school and as unfair as it might seem, I appreciated the fruit of their quest for diversity among matriculants. Not only ethnic diversity, but people of a slightly wider set of interests too. I felt interacting with the other students was the best part of my college experience. It’s true I don’t know all the stories of the people who weren’t admitted, but when dealing with such tiny acceptance percentages it really feels more like a lottery for a very scarce resource. So it’s definitely not fair, and they’re deliberately not always choosing the highest scores. My mother always said life isn’t fair and I’m not sure if the outcome would be what you or I would hope for if the process was made more “fair” for undergrad selection.

* edit: typo on mobile :)

I remember looking into the STEP after graduating for "fun".

They were such classic Oxbridge style questions, sure you can probably game them but as you say, once you've done the graft to game those types of questions, you're pretty much qualified to be there because the rest of your degree will just be more of the same.

Playing devil's advocate a little... it seemed like it worked out ok for you? There are different universities with different selection criteria, and any particular university might have multiple selection pools. There's room for rich scions and math geniuses at Harvard. And in your case, Harvard's loss was Oxford's gain.

I feel like you're expecting some sort of clear stack ranking of students. Even if you take tests at face value, do you accept the student who aces math but bombs literature, or the student who is merely above average in both? See also: computer science, chemistry, biology, physics, history, physical fitness, social sciences, languages, music, art...

We have examples of test-regime-driven educational advancement in Asia. From what I've read, it doesn't seem very appealing to me, either as a student or as a parent.

> Even if you take tests at face value, do you accept the student who aces math but bombs literature, or the student who is merely above average in both?

This is the kind of tradeoff I'd be happy to see colleges worried about. Complicating the fact is that US universities generally don't expect students to be set on a specific major while European ones do, making the US problem a bit harder. But I don't think this is what US unis are trading off on. I think they're trading off merit for parental wealth + future wealth (which is greatly effected by parental wealth).

Re: your comment about different unis with different criteria, I'd be a lot less worried if the diversity was evenly distributed geographically. I had the privilege of being able to choose from multiple countries. For the vast majority of high school students, studying abroad is not an option.

I'm venturing off into speculation here, but I don't think there's anything so organized or homogeneous about college admissions. Probably there are thousands of judges across various universities, each with their own biases, each trying to pick a hundreds of candidates among tens of thousands of applicants.

"US unis" don't seem like a homogenous block to me. But then, I went to a state school and received a fantastic CSc education that I would not trade for ivy league at any price. I've also interviewed/hired countless people and never even noticed what school they came from. I guess the policies of Yale don't bother me that much either way.

Those exams are essentially tedious calculus problems. It’s the high school math equivalent of the National spelling bee.

Added wrinkle is that for Oxbridge you'll get interviewed as well. I recall being asked interesting things that they didn't cover in class, things that are more deep understanding than parroting syllabus materials.

False dichotomy; there's another option, which is to use a lottery: http://revisionisthistory.com/episodes/44-the-powerball-revo...

Have we checked for extraterrestrial influnce? Seriously it is Larry Niven's straight out of the Known Space's series Puppeteer plot.

In all seriousness it works well if you have a pool sufficiently quality homogenous but would tend towards the mean of the pool.

They're obviously not interested in what's better or they would expand to accept them all! But if they grow, they'll dilute their elite status.

Certainly the test is not the be all end all, other factors are important. Grades, recommendation letters, activities, and an understanding of the students background.

However it can help us evaluate who is able to perform in a higher learning environment and that they have the knowledge to study advanced degrees. I had 4 standardized tests in my educational career, 4th grade state test, 10th grade state test, ACT and GRE. Only 2 of those were non-repeatable. How many do students take these days?

> if you judge everyone by (objective) standard tests

That's why I don't like standard tests. When I was studying, entrance exams were oral. You were given three problems (chosen at random), had some time to solve them, then presented your solutions to the examiner. Examiner would then ask follow-up questions.

There is no way to prepare for such test other than to learn the subject. No multiple-choice questions you can Kaplan.

What do you want your child selected on for an elite school then? The bedrock reality is that most children, by definition, won't make it into an elite school. How do we pick?

This question is phrased oddly. I want my child to enter an educational environment where he will flourish. It's not obvious to me that having the brand of an 'elite' school is all that significant.