HN is protected a bit better with its karma requirements. But I feel like recently I've been getting way more upvotes than I feel I "deserve" for some comments, which scares me a bit.

It's easy to get massive upvotes on anything that jibes with the HN groupthink. It can also be instructive to "troll" with anything that the groupthink will be appalled by no matter how true or rational it may be. That's the surest way to compensate for undeserved karma.