The xargs command can invoke arbitrary commands, possibly including scripts that themselves invoke xargs, so a system using xargs -P recursively could very well benefit from a governor limiting the total number of jobs run in parallel. It might be unlikely or uncommon (I've certainly never seen it in practice) but nothing prevents it from happening.

And if you're using xargs somewhere in the middle of a parallel build, having xargs -P respect the jobserver's limits would be a nice touch.