The brain will keep around information that is used automatically. In my experience its a very bad idea to forgo learning something out of the expectation that it won't be used. I did this often, and it was catastrophic for me -- it turns out we're not very good assessors of what we will need.
There's a related corollary to this question though, which is whether systems that require less human memory enable you to remember other things better.
For example, your phone can dial dozens or hundreds of your friend's phones without you remembering their numbers. This means you never need to use their phone numbers and hence don't remember them. A reasonable question you might ask is whether you are now free to spend that brainpower memorizing other, more useful things? Or are you actually less capable of memorizing other things because you are exercising your memory less?
There are other similar examples, for example password managers and even just general Google-able trivia. Is it a good thing or a bad thing for general intelligence that humans need spend less time memorizing these things?
I felt this way for a long time about geospatial directions, but when your phone / gps dies and you have no map, it's kind of a terrifying feeling of helplessness
I skirt around this by instead of depending on turn by turn I study the main route and then only go to turn by turn at the endpoint. That way I gain the knowledge of the macro-infrastructure and main roads - I tend to give directions to out of towners even in cities I’ve been in for a week.
So from what I understand, there is a tradeoff to whether or not you memorize something.
When you have a piece of knowledge memorized, it has the capability to be activated in parallel with other known knowledge. These parallel activations result in a new / novel idea to your neural network. If these activations haven't been had before, they may be a breakthrough in science, technology, or otherwise.
So in one aspect, if you do not memorize, you can not associate and create new things.
Luckily, with phone numbers and locations it might not be important or valuable to have them memorized. But that's hard to say. Many scientific breakthroughs and discoveries were made by people who had cross-discipline knowledge and associated two things that no one thought were valuable together. Potentially, even googleable factoids could potentially contribute to a scientific breakthrough.
Doyle was a crank.