Great point, and I suspected the problem might not be as easy as it appears at first glance. (Because of course it isn't...)

I also considered the case when you're rapidly scrolling through a page- if a naive approach simply made things non-interactable if they've recently moved, that would neuter re-scrolling until the scrolling halted, which is NOT what people want