If the goal is to save memory, you can go farther than this. You could store the JPEG in RAM, compressed, and have the GPU decompress & shade. Each of the DCT blocks can be converted massively parallely on GPU.
You can't do image decompression in GPGPU because the last step (Huffman coding) is not parallelizable - in fact if any compression is parallelizable it's not compressed enough. But you can do it in dedicated hardware.
