JPEG uses Y'CbCr, not YUV per se. The matrix transformation is applied to gamma-encoded R'G'B' (typically sRGB), not linear RGB.

http://poynton.ca/PDFs/YUV_and_luminance_harmful.pdf

The only important bit of pedantry over analog/digital and prime is reminding people that no one deals with linear light unless you already know you're doing so, and that blending in nonlinear space is rarely correct, no matter how common it is to do so.

I think we are in agreement? What you are saying is that conversion from linear YUV to linear RGB is, indeed, linear.

Further, the transformation from Y'CrCb to gamma-encoded R'G'B' is also a linear operation. Right?

It depends what you want to do with the data. If you want to send your sRGB-profile Y'CbCr JPEG image directly to an sRGB display then you are done after applying a matrix.

If you want to composite multiple images, do other intermediate processing, or display the image on an arbitrary non-sRGB display, you probably want to convert to a linear space along the way.

So we are in full agreement, then.