When would you use which?

Whichever model works better for your use. It's hard to know without testing it at the moment.

I've found Gemini to be better at some use-cases, and GPT-4 better at others for my specific taste and use-case. You can kind of go by the benchmark scores to have an idea if it's good at logic, creativity, etc.

Obviously another small model would be specialized in determining that.

Is it models all the way down?