I wonder if it in part has to do with English's overall status as a "second class" or even "lower class" language even in its own native land during the years after the Norman invasion. Up until a certain point the aristocracy spoke mostly Norman French, and even after they switched to English they did so with a form heavily filled with French loans and written with French-derived orthography.

What I mean to say is, if the language is on the whole not the language of the nobility, even the formal pronouns would not be perceived as elevated, really. If you really wanted to impress, you'd switch to French (or even Latin, etc.)