Does leaving tap water out help with the salt problem? I've read that after 4 days, most of the chlorine will evaporate from tap water - and this is good for plants.
Hypochlorite (like they add for disinfection) will break down on standing, or faster with aeration. Chloramine (also added for disinfection) won't, which is why aquarists use various chemical treatments to remove that.
Neither is too important for plants, though. The usual problem for plants is chloride, which can't be removed except by processes like reverse osmosis or distillation. Most plants can easily survive chloride at typical concentrations in tap water; but if you don't water to runoff then that concentration may slowly increase in the root zone, eventually harming the plant. You typically want something around 10% runoff, though that will vary with your plant species, water, fertilizer, etc.
Others have answered the question already, but I want to extent on my comment: Long standing water, same as boiled water loses some CO2. CO2 in water forms carbonic acid in part, which dissolves calcium salts. Removing the acid, will make some of the calcium salts insoluble (that’s why you get a buildup in water boilers). Most other salts are unaffected by that. So the disposal of collected water under the pot is without an alternative. Even if you’re using demineralized water, you need to flush the plant’s toilet.
