You still will need clear benchmarks as the reward for RL. With Chess, the rules are simple but you may not have a clear loss function for a complicated architectural challenge.