But if you do want to break up your keys more, make sure you specify IdentityFile and Identities Only in the per host definitions in your ssh config.
By default assuming you use an ssh agent (no forwarding) with multiple keys and a default ssh config, the behavior is to just try to auth with every key in order.
So if you're worried about the ssh server identifying you, you're still exposing yourself. I don't think this is much of a concern but worth noting.
Slightly more important: you're wasting time during the initial connection to fail authentication a few times. This can matter more with higher latency
Even more important: sshd has a configurable number of times a client is allowed to fail authentication in a session attempt. If you have too many other keys in your agent you will just fail to auth before it tries the key that is actually valid for that host.