I see your point that there's no way that weight levels are entirely predetermined by genetics. To your point there are various cultures around the world who suffer more or less from obesity.

I don't understand this genetic determinism of weight to be the claim of Set Point Theory, which to my understanding is involved in energy homeostasis, which you acknowledged. Your body sets your metabolism, and it can tune it up or down. There are homeostatic processes in place such that your body "knows" whether it is underweight or overweight.

I'm not sure if there is a more specific objection you have to set point theory. It certainly has its limitations as to what it can explain, but it seems to be a useful concept or model.