Yes. My naive intuition about this is you need the extra parameters precisely to do the learning because learning a thing is more complicated than doing the thing once you have learned how. There are lots of natural examples that fit this intuition eg in my mind "junk" DNA is needed because the evolutionary mechanism is learning the sequences which work in a similar way. You don't need all that extra DNA once you have it working but once you have it working there's little selection pressure to clean up/optimise the DNA sequence so the junk stays.