As much as I would also like to drop double clicks and long presses and only use fast and discoverable interactions I think that ship has sailed...

A less militant solution might be highlight-on-press. We can at least ensure the UI is as visibly responsive as possible even if actions need to wait for gestures to complete.