>My first thought would be to show it when the user taps and holds. But that also triggers the browsers default context menu for links

The first doesn't exclude the second. That's the way how https://xkcd.com/ is doing it. It just shows both.