> And what an IQ test, tests, is your ability to recognize and exploit novel patterns, generalized across different domains (word problems, pictures, music), under time pressure
It's worth pointing out that most criticisms of IQ tests are that the amount of novelty in the problems is pretty heavily dependent on your cultural background.
Very true! I'm surprised there isn't some well-known trick of scientific process to screen this out, actually.
One that comes to my mind pretty quickly: after the first test to collect people's "novel" results (the one that'll be used as input for the final score), have them take 2-5 more tests with the same tasks (enough so that the questions are utterly non-novel by the end.) Show them the correct answers for the problems on the test they just did between each session.
Now take the scores from all the tests (including the first one), and fit a curve to them. The second derivative at the point of the first test then (might!) represent the novelty they were experiencing at that point. Now figure out how much extra novelty they experienced compared to an average member of the population, and use that as a coefficient to the final IQ score from the first test.
It's worth pointing out that most criticisms of IQ tests are that the amount of novelty in the problems is pretty heavily dependent on your cultural background.