I agree that if you had a large (in the dozens, if not hundreds) number of dependent components, it could be an issue. However, I've noticed no performance degradation with my usage -- I use a top-level AuthContext component to handle user data/login state, and another for access/caching of the app's main data.
In a single view (for lack of a better term), I'd guess the largest number of components that access either context is six.
In a single view (for lack of a better term), I'd guess the largest number of components that access either context is six.