Nerds like you and me know about reader mode, but I would bet at least 90% of web users don't and even if they do it's another step in the way of accessing content.
Any impediment you put in the way of readers is going to lose a certain percentage of them. Even nerds who know about reader mode are sometimes going to see a badly formatted blog post and just close the tab.
