In a 2005 autobiography, he described some of his work as having "provided strong support for the “Big Bang” interpretation of our earlier discovery". - https://www.nobelprize.org/prizes/physics/1978/penzias/biogr...

Strong support is obviously not the same as confirmation, so I think you're right that he might have disagreed with this word choice.