Selfishness will cause you anxiety and guilt. It will cause pain to others. It will make you a jerk.

A society of jerks inflicting pain thrives less than one where we are appropriately empathetic and charitable to others.

If you're going to argue against selfless empathy, etc. I suggest we have different axioms and this is not a discussion that will go well.