Back in ancient times - the 1980s - one of the DAK-style electronic toys mail-order places sold an interesting box. It worked like this:

1) pick up line

2) play a pre-recorded message (default: "Please enter your extension.")

3) wait for the touch-tone sounds (or rotary clicks, those still being a thing)

4) when the correct three-digit code was entered, your phone would ring.

No code, no ring. Simple. (and no voicemail, either!)

And if "extension 123" got compromised, you could just change it to "456" and let people know.