> First of all, to be clear, I'm not talking about the hardware clock. That really only matters when you read from it, and you generally do that once: at boot.
That's absolute time that's read at boot, from a battery-backed real-time clock (RTC). Relative time can still drift. Crystal resonators with multipliers and PLL are used to supply the CPU with the signal distributed by its clock network. The crystal is a voltage-controlled oscillator (VCO), so it is tightly controlled (could be by voltage controlled by propagation delay within the chip, for example), but it can still drift over time and with temperature. As the article mentions, this can be responsible for system time error, "Your clock's pace will always vary a tiny little bit for different reasons". Operation of the computer shouldn't be impacted since things sync to the clock, but absolute timekeeping can be.
One additional thing that wasn't yet mentioned is that freqency depends also on gravitation, so orientation of your motherboard could matter (for the curious - https://www.youtube.com/watch?v=zILwgQhjC_Q ). I'm not really sure it's big enough to matter though.