That other programm is knockd, which needs to listen to all traffic and look for the specified packets.
Granted, that program is really small and could be easily audited, but that same time could have been spent on trying apparmor/seclinux + a good vpn and 2fa
I much prefer the approach I read about in https://github.com/moxie0/knockknock (use a safe language, trust basically only the program you write and the language), to a random port daemon written in C which pulls libpcap to sniff everything.
To some extent knockknock also trusts the Python interpreter which is not ideal (but maybe OK)
In Linux, simple knocking (fixed sequences of ports) can be done entirely in the kernel with nftables rules. Probably could even have different knock ports based on day of the week or hour or source IP.
Granted, that program is really small and could be easily audited, but that same time could have been spent on trying apparmor/seclinux + a good vpn and 2fa