I've picked up and dropped a moderately sized caffeine habit at various points in my life. The biggest effect on me of being off caffeine is a reduction of my tolerance for caffeine when I have some next (smaller amount has bigger effect) and clearing out the addiction (can go days without caffeine without headache). I don't see appreciable changes in my quality of sleep (exercise does that!) or my ability to focus.
I have every expectation this differs dramatically person to person, and I encourage anyone with any sort of habit to examine it, but what you say does not necessarily apply to everyone.
I have chronic and severe sleep issues - they're probably psychological but I've subscribed to a theory of being "non-24" for some time.
I've sworn off caffeine twice in an effort to regulate my sleep better and neither time has it helped me in getting to sleep. What it does do is in the "morning" [ie when I wake] make me more alert, I'm not in caffeine withdrawal, I feel better.
If you're going to try it then my advice is to ensure you still drink enough: substitute caffeine drinks with water for example.
For me going cold-turkey it takes about 1 week for the severely painful headaches to stop and for me to be able to think clearly again.
So far as I can determine caffeine has potential long term positive effects, doesn't have lasting negative effects. I slip back in to it because it's in the house and at times I have to use something to wake me up when I'm sleep deprived. It's pathological.
I have every expectation this differs dramatically person to person, and I encourage anyone with any sort of habit to examine it, but what you say does not necessarily apply to everyone.