I tend to dislike pure infinite scrolling and instead prefer on-demand loading.
With on-demand loading you would set the virtual height of the scrollable area to the height necessary to show all (or a huge chunk of) your data, and then page in chunks of the data as it approaches the viewport.
Then dragging the scrollbar works, and touch scrolling shows a reasonable representation of how much data is left.
The problem is that many of the places they use infinite scrolling have such large datasets that doing this would make the scrollbar borderline useless for small movements (move back half a page or 1-2 pages) -- which probably account for 99% of scrollbar uses in this context. Google search for "cat"...
Of course if you actually advanced extremely far in a typical "infinite" scrolling view, the scrollbar would similarly become pretty useless, but in practice very few people actually scroll all that far, so it seems better to optimize for that case.
They could meet halfway by making the "scrollbar readjustment" chunk size somewhat larger than the "load new data" chunk size, so you get the on-demand loading efficiency of the latter with fewer scroll-bar jumps, but I don't think they could make the former too large without creating more usage problems than they're solving.
With on-demand loading you would set the virtual height of the scrollable area to the height necessary to show all (or a huge chunk of) your data, and then page in chunks of the data as it approaches the viewport.
Then dragging the scrollbar works, and touch scrolling shows a reasonable representation of how much data is left.