What is the SOTA for benchmarking all of the models you can run on your local machine vs a test suite?

Surely this must exist, no? I want to generate a local leaderboard and perhaps write new test cases.