I would guess they do it because they want to minimize the chance that someone will install an unapproved app to someone’s phone and cause harm. I know it’s already pretty hard but Apple seems to be very particular when it comes to this.
That’s an opinion. Apple’s take is that they sell ”everything that runs on your phone has gone through our reviews, so you can trust it isn’t malware”
That, in their opinion, makes it their job to prevent people from permanently installing software on other people’s phones. I’m sure they would remove the “permanently” if they could, but developers have to test builds so frequently that they can’t review them all.
You can only sideload for free if you are willing to reinstall every X days.
They don't need to test an app if you're not asking them to distribute it through their store.