You joke but wireguard and, very easily, tailscale, solves most crap you normally need to fix. Close everything with ufw and put tailscale, and then you trivially have access from any device / desktop.
I just rely on UniFi and HomeKit for out of home access! But you're so right I could also access remotely via my Wireguard server or Tailscale running on my ubiquiti console... Wish I could hook up baby events like "poop diaper" to Grafana.