Only in pure lambda calculus. But that's not because of closures. That's for all variables everywhere; captured, free, whatever. So your original statement — "usually the captured values in a closure can't be mutated" — still has no basis even if you insist "But, pure λ-calculus has no mutations". It's not because of closures that the captured variables became immutable.

When used in any of many lambda calculi that do have mutations, closures support mutation just the same as any variable. Whether a variable is free or captured, that doesn't change if or how it can be mutated.