I have the latest versions of Safari and Firefox, on the latest version of OS X. Safari has about 15 extensions, has been used daily for the past 5 years (and I haven't deleted its history, though I clear the cache one every couple of months). And I use Firefox "only" if I need to access the web-based GMail (which happens roughly every 2 months), and there are no extensions installed in Firefox.
Considering the above, Safari still is (much) faster than Firefox on my machine. I don't care if it can render JS 5% faster - the whole experience is sluggish and slow on my machine. Of course, YMMV.
Just wanted to point it out to you that there are legitimate reasons why people say Firefox is slow.
Out of curiosity: have you tried "Reset Firefox" (in about:config – you won't lose anything if you don't have add-ons)? Most users experience a huge speedup when they use it.
In my experience, Firefox gets slower and slower the longer it's been open and with many tabs open. I believe it to be related to GC. Unfortunately, there aren't any hints (like per-tab activity/memory/allocation/timer readouts) as to which tabs are contributing the most to the problem. Sometimes I'll guess the miscreants and closing enough of them helps, but more often things only improve after a restart (essentially unloading all tabs until revisited).
It does seem add-ons are implicated (and perhaps especially Firebug, even if Firebug is not activated in any tab). NoScript may also be implicated; especially with regard to pages that have some but not all script sources enabled.
Considering the above, Safari still is (much) faster than Firefox on my machine. I don't care if it can render JS 5% faster - the whole experience is sluggish and slow on my machine. Of course, YMMV.
Just wanted to point it out to you that there are legitimate reasons why people say Firefox is slow.