Just to nitpick one of the claims in an otherwise well-reasoned comment - of course a lot of Chinese don't speak the national language. That's because for many Chinese people, Mandarin isn't their mother tongue - other Chinese dialects like Wu, Yue, etc. [1] As the BBC article you linked states, this is an issue of "linguistic unity" more than a symptom of a bad education system. Japan, as a much more homogeneous society, did not have this problem.

[1] http://en.wikipedia.org/wiki/Dialects_of_Chinese