I think that the recent mathematical+philosophical(?) work by “johnswentworth” on AlignmentForum/LessWrong on Abstraction as “Information at a Distance” is an intriguing approach towards how it could be that abstractions which are “natural” could arise. I think this seems like it might fit fairly well with the concept of natural kinds (though probably not a 100% match? ) .

Separately, I think the idea of degrees of naturalness of a concept seems to make sense?