Because it's your cert that you just installed on your server, so you know its thumbprint.

This is effectively what you are doing every time you connect to a server over SSH and say 'yes' to that message with the funny string asking, "Are you sure you want to connect?" It's analogous to pinning a self-signed certificate.