I wouldn't say it 'doesn't matter'. We get by, sure; but that doesn't mean the situation is ideal. On my laptop, if I try pinch-zooming, safari seems to do bilinear scaling, and firefox CPU usage will dramatically spike. The former is noticeably ugly; the latter is bad for battery life, and may stutter if work is being done in the background (laptops and phones don't yet have hundreds of cores).
I would bet that if you profile Firefox WebRender there very little of the time is actually spent rasterizing glyphs. It takes like 10 microseconds to rasterize a small glyph, it's really nothing. Source: This is what I spent a ton of time profiling in the past.
And Safari should just rerasterize glyphs while zooming. In 2022 there's no technical reason why it can't.