More of a tangent as I agree with your point, but...
I think this diversity, while large, is still less than in many other languages. As one example of many, just within Europe there are accents/dialects of German in Switzerland that are largely mutually unintelligible even to some other Swiss German speakers, and nearly entirely unintelligible to most Germans. This is also true for dialects of Italian, or even Dutch, according to friends from those countries. It's generally even more fragmented outside of European languages.
Unlike English, some of these cases don't even have easily standardized registers. In my experience, people in Birmingham or Glasgow, for instance, can fluidly switch to a more standardized accent and dialect that's natively or "intuitively" understandable to a speaker from North America or South Africa.
I think the primary factor that causes this perception is the lack of any proper dialect continuum between English and another language. There are of course a few minor ones like English<->Scots and English<->Jamaican Patois, but nothing significant that any sizeable number of speakers would encounter.
Dialect continuums between languages are basically the norm around the world. You can even connect Sicilian to Portuguese for example.
I would assume the primary reason English isn't part of any major one is because Britain is an island, otherwise English would probably connect to Dutch and join the West Germanic continuum. It would then appear just as fractured as any other language.
This is a good point. When I was in Switzerland they had a guy from Muothathal on TV. TV decided to subtitle him. I thought Swiss German was weird, but this fellow seemed to be speaking a completely different language, to my beginner ear.
The thing is though, you can ask people to speak High German instead of mundart, eg at meetings. They admit it's different enough to warrant explicit statement.
You can't be telling your Liverpudlian or Brummie to speak the Queen's English, that would be rude.
I think this diversity, while large, is still less than in many other languages. As one example of many, just within Europe there are accents/dialects of German in Switzerland that are largely mutually unintelligible even to some other Swiss German speakers, and nearly entirely unintelligible to most Germans. This is also true for dialects of Italian, or even Dutch, according to friends from those countries. It's generally even more fragmented outside of European languages.
Unlike English, some of these cases don't even have easily standardized registers. In my experience, people in Birmingham or Glasgow, for instance, can fluidly switch to a more standardized accent and dialect that's natively or "intuitively" understandable to a speaker from North America or South Africa.