I'm afraid this analogy doesn't hold much water: facial features inhabit a continuum with very nice smoothness properties, whereas program behaviour changes dramatically when you perturb its source code, even minimally (this is why mutation testing is effective, for instance).

Also, when you average you don't really kill "defects", but rather outliers. An "outlier" statement within a program is very likely to do something important, e.g. taking care of a corner case, otherwise it wouldn't be there.