I thought that it might be a rare chance to invoke the NFL theorem appropriately, but I guess I was wrong. The NFL talks about a uniform distribution of problems. A case that is probably never the case. At least for habitable universes.
Nevertheless, the theorem basically states that there are games where AlphaZero will be beaten by another algorithm. Even if those games are nonsensical from our point of view.
Games drawn from this uniform distribution can't even be implemented in our physical universe (you would need exponentially large lookup tables to store the rules). There is no chance of ever encountering any of them.
Of course, there are "games" like "invert sha-512" that can be implemented in our world but are probably impractical to learn. But NFL has nothing to say about them; a game that simple has zero measure in a uniform distribution over problems.
I forget, was it Alpha or one of the others (Leela, Kata, FineArt,...) which had a weakness against... I wanna say the Micro Chinese (?), where it would consistently play the same suboptimal sequence that let players beat it easily if they took that path.
Nevertheless, the theorem basically states that there are games where AlphaZero will be beaten by another algorithm. Even if those games are nonsensical from our point of view.