Except it requires VPN when not on the same LAN for some reason...

How should it work? Send everything through a central server? Try to hole-punch through NAT? Ubiquitous IPv6 might help, but we're not there yet.

A self-hostend STUN/TURN server maybe? I don't know but I'm def not installing a VPN for this. Considering I might have another one already running. The issues that would arise from this are numerous.