That's not normal. It doesn't happen on any of my accounts or clients. Verification takes a moment if you're in a lot of rooms, but it exchanges all keys.
been a pretty reliable issue when I've set up a new device. Whatever keys the client is getting, they're apparently not useful.
(This general flakiness of features just sometimes not working as they should is probably the main reason I haven't tried to recommend friends to switch to element)