I like some of the information in this, and unfortunately I missed the conversation when this made it to the front page a couple weeks ago, I just wasn't in the right headspace.

But I disagree with one of the value judgements:

> As you add more and more users, you learn less and less because you will keep seeing the same things again and again.

No! The frequency of errors is your priority list for UX fixes. You may not, as they claim, find more errors with more than 15 people, but more data samples gives you information about distribution and frequency of errors (do people make this mistake once in a while or every single time?).

You might not need any of that data to make a rough priority list, or it might be helpful for negotiating scope. It'll depend a lot on context.

If you go through that site's (very long) history, you'll see their conclusions that quantitative UX research isn't useful for single projects.

For discovering features of the population, yes, quantitative research is great. But for making your software better, you are much better iterating small qualitative tests.

The point of usability testing is primarily to identify sources of roadblocks and confusion. It helps you understand where people can get stuck and why, but not the frequency.

Doing usability testing is both expensive and time consuming. It requires someone to analyze the videos and the testers are typically paid. So it's a very poor choice for creating frequency data.

Once the usability issues have been uncovered, it might make sense to look at the relevant analytics or add better tagging to get that frequency data.