Because I could then build the entire interface client-side in a browser?

Don't get me wrong, I'm not a webdev guy. I have no front end/fullstack experience. I'm a longterm sysadmin/DevOps person. But if you're a power user, you want control over your mail interface/workflow. In that case, all your mailserver should be doing is accepting email for you, storing and indexing it, and serving it via API to clients you're using. I like IMAP, but it doesn't easily support some Gmail conventions (multiple labels per message).

IMAP and SMTP could easily be condensed into an XML/JSON API that could be done over HTTPS; I'm not familiar enough with CalDAV to say that though.