I may be conjuring things, but I think many early video cards actually outputted a TV or component video signal, which uses YUV (=YCbCr?) color, so it is RGB that would have required conversion, not YUV.