I know a surprising amount about of Google and TPU's :)

This is not accurate - they are neither cheap nor really built for inference.

I'd have to go look hard at what is public vs not if you want me to go into more.