Redshift is indeed a matter of speed. But due to the expansion of the universe, relative speed and distance are directly related (Hubble's law).

So farther away means faster relative speed and thus more redshifted (Doppler effect) Farther away also means older light (due to the finite speed of light).

Putting that all together means that to observe old light from the start of the universe we have to look in the IR spectrum.

As I understand it redshifting is due to the doppler effect, whose formula only depends on the relative speed between the observer and the sender.

However, it also seems like due to galaxies further away having a larger expansion speed, typically they are more redshifted.

It’s both, but the contribution from distance will be much greater than from the relative speed for an old, distant galaxy.