Either call it out, or stop talking and forget where you stopped. Having to awkwardly restart an interrupted conversation sometimes gets the message across.

When I was younger, a mix of both fixed that nasty habit of mine.