Please look up studies by Mathew Walker. They did numerous studies with people claiming coffee did not affect their sleep. What they found was while those people would fall asleep, and sleep throughout the night, the quality of the sleep (i.e. minutes in deep sleep) was substantially worse.
But I agree that abuse is what is causing majority of the problems (and it was in my personal case)
My confidence in Matthew Walker was enormous after reading "Why We Sleep", then dropped to around 0 since reading this: https://guzey.com/books/why-we-sleep/
A lot of it doesn't require any domain knowledge to verify.
Doesn't change the information in his book about how it is genetically determined how quickly you digest the caffeine. My dad can easily have a coffee around 19:00 and still sleep well, while when I have a coffee in the afternoon I will have big trouble sleeping.
The alleged failures in "Why We Sleep" go as far as quoting papers and books claiming they say "X", where in fact they claim "not X". So to me, Walker saying or quoting something bears 0 informational content (almost).
But different rates of caffeine tolerance and digestion certainly conform with my personal observations, Walker or not.
But I agree that abuse is what is causing majority of the problems (and it was in my personal case)