Redshift is indeed a matter of speed. But due to the expansion of the universe, relative speed and distance are directly related (Hubble's law).

So farther away means faster relative speed and thus more redshifted (Doppler effect) Farther away also means older light (due to the finite speed of light).

Putting that all together means that to observe old light from the start of the universe we have to look in the IR spectrum.