> how do you create good tests for a more advanced AI than you yourself are (or have control of)?
I think I answered that. You use more resources to create it than answer it. When creating it, you can have a team take lots of time, use all sorts of tools, etc. When taking it, you constrain time and tools. That’s the general pattern of how to make a question harder than you can answer.
There are also lots of trapdoor questions, especially in math. An overly simple example is factoring numbers.
I think that can only work with very small, incremental steps in intelligence, though. Will a ten year old human be able to create a meaningful test for an adult with high intelligence? No matter the resources you give the young one, they usually will not be able to.
There also might be thresholds, barriers, were small increments in intelligence are not possible. But that is speculative, I will admit.
I think I answered that. You use more resources to create it than answer it. When creating it, you can have a team take lots of time, use all sorts of tools, etc. When taking it, you constrain time and tools. That’s the general pattern of how to make a question harder than you can answer.
There are also lots of trapdoor questions, especially in math. An overly simple example is factoring numbers.