You can apparently opt into something called "Passthrough" that makes (most) Spotify clients download MP3s directly from your CDN, but it's not the default and has certain requirements on your format and uptime.