I switched from FF when it was sluggish.
Funny, the reason I don't switch back is the opposite of yours: I don't want to miss Vimium... how does Vimperator compare to Vimium?
I was in the same boat. I tried to switch back to Firefox a while back. Some people prefer Vimperator and Pentadactyl, but after using Vimium for a while, I found them to be a little too complicated and, well, different from what I was used to. It's possible I could have adjusted, but I didn't bother at the time and quickly returned to Chromium.
Later I discovered a Firefox addon called VimFX that basically aims to be "Chromium for Firefox": A very simple set of Vim-like shortcuts, configured in almost exactly the way Chromium is. Once I started using it, I felt right at home in Firefox, and I haven't looked back since. It's got a few warts (delay when using gg, Ctrl-F rebound to PageDown by default), but all things considered I'm pretty happy with it.
