Memory seems rather like a multi-tier cache hierarchy to me. You can only really keep something like seven objects in your mind at once, and everything else seems to be pushed to slower sections of your memory.
Within this analogy cache eviction strategies start to have massive effects on our ability to process information.
There's also the fundamental theory of this which ultimately rests on thermodynamics, e.g. [1] [2].
Basically, any finite organism only has a finite number of configurations available to it; but to 'remember' something is to a set up a long-term correlation between the past and the present, which uses up some of the 'available degrees of freedom' to e.g. respond to shorter-term changes in the environment.
If you never forget anything, then eventually you 'run out of bits', whereas a system that allows some forgetting has greater flexibility in e.g. forgetting the distant past in order to remember to tie your shoes right now.
There are some neuroscientists that are starting to bring thermodynamic views into looking at actual biological systems as well [3]
Within this analogy cache eviction strategies start to have massive effects on our ability to process information.