I've been using Colemak for about 6 years now. The difference for me wasn't just speed, but rather that I learned to type "wrong" on Qwerty. I could never get myself to touch type, I always had to look at the keyboard. It turns out that forcing myself to switch layouts fixed that, for whatever reason. To this day I can type with Colemak without looking but have to look if I want to use Qwerty.
Exactly! I never touch-typed properly with Qwerty. It was trying to learn proper touch-typing with Qwerty that I got so frustrated trying to unlearn old habits that I picked up Colemak instead.