Indeed. People do try to overcome this, for example see the difference in results between "Show Offline Test" and "Show Mensa Norway" on https://trackingai.org/IQ
Even the lower value is still only an upper-bound on human-equivalent IQ, as we can't really be sure the extent to which the training data is (despite efforts) enough to "train for the test", nor can we really be sure that these tests are merely a proxy for what we think we mean by intelligence rather than what we actually mean by intelligence (a problem which is why IQ tests have been changed over the years).
My intention in this sub-thread is more of the economic issues rather than the technical, which complicates things further, because if you have an AI architecture where spending a few tens of millions on compute — either from scratch or as fine-tuning — gets you superhuman performance (performance specifically, regardless of if you count it as "intelligent" or not), then any sector which employs merely a couple of hundred workers (in rich economies) will still have an economic incentive to train an AI on those workers to replace them within a year.
This is still humans having jobs, and still being economically relevant, but makes basically everyone into a contractor that has to be ready and able to change jobs suddenly, which is also economically disruptive because we're not set up for that.
Even the lower value is still only an upper-bound on human-equivalent IQ, as we can't really be sure the extent to which the training data is (despite efforts) enough to "train for the test", nor can we really be sure that these tests are merely a proxy for what we think we mean by intelligence rather than what we actually mean by intelligence (a problem which is why IQ tests have been changed over the years).
My intention in this sub-thread is more of the economic issues rather than the technical, which complicates things further, because if you have an AI architecture where spending a few tens of millions on compute — either from scratch or as fine-tuning — gets you superhuman performance (performance specifically, regardless of if you count it as "intelligent" or not), then any sector which employs merely a couple of hundred workers (in rich economies) will still have an economic incentive to train an AI on those workers to replace them within a year.
This is still humans having jobs, and still being economically relevant, but makes basically everyone into a contractor that has to be ready and able to change jobs suddenly, which is also economically disruptive because we're not set up for that.