And this may still be true in 2023, but the problem is that most of the parallelized grep replacements (e.g. ripgrep, ag, etc.) are SO much faster than grep that the much small speed differences between them doesn't provide much of a basis for differentiating them.
I use ag (typically from inside Emacs) on a 900k LOC codebase and it is effectively instantaneous (on a 16 core Ryzen Threadripper 2950X). I just don't have a need to go from less than 1 second to "a bit less than less than 1 second".
Speed is not the defining attribute of the "new greps" - they need to be assessed and compared in other ways.
In 2016, I'd say speed was definitely a defining attribute. ag has very significant performance cliffs. You can see them in the blog post.
But as I mentioned in my comparison to qgrep elsewhere in the thread, everyone has different workloads. And for some workloads, perf differences might not matter. It really just depends. 900 KLOC isn't that big, and indeed, for simple queries pretty much any non-naive grep is going to chew through it very very quickly.
The blog post also compares Unicode support, and contextualizes its performance. ag essentially has zero Unicode support. Unicode support isn't universally applicable of course---you may not care about it---but it satisfies your non-perf comparison criteria. :-)
In my experience they are all horribly i/o bound and the search takes as long as files load from disk and that's quite long, after that the difference can't possibly be meaningful. When files are in cache search time is dominated by time it takes me to navigate the file system and write the command, and again performance difference can't possibly be meaningful.
> When files are in cache search time is dominated by time it takes me to navigate the file system and write the command
This suggests your corpora are small. If you have small corpora, then it should be absolutely no surprise that one tool taking 40ms and another taking 20ms will matter for standard interactive usage.
I use ag (typically from inside Emacs) on a 900k LOC codebase and it is effectively instantaneous (on a 16 core Ryzen Threadripper 2950X). I just don't have a need to go from less than 1 second to "a bit less than less than 1 second".
Speed is not the defining attribute of the "new greps" - they need to be assessed and compared in other ways.