We have lossless memory for models today. That's the training data. You could consider this the offline version of a replay buffer which is also typically lossless.

The online, continuous and lossy version of this problem is more like how our memory works and still largely unsolved.