I was the exact same until I started using relative line numbers. Then I can just look at a line and see that it's N lines down and jump there immediately. With j or k repeat I'll often over shoot and then have to go back which is kind of annoying.
The information is also more relevant. How many lines a function or block has might make a difference, but the absolute file position is, more often than not, irrelevant.
Funny, I was thinking exactly the opposite. Stack traces, logs, test failures, warning, etc will always reference an absolute number. If you want to tell a coworker where to find a section of code, you use the absolute number.
Relative numbers are useful to save a few keystrokes. But in every other context, absolute numbers are what's relevant.
