I would think there's a difference between regular stimuli (through touch, sound, sight, taste, smell) causing your brain to react -- possibly drastically, and deliberately changing your brain chemistry in an "unnatural" way by taking drugs.

One chemical change is caused by the brain working as designed, while the other is a deliberate attempt to change the brain directly.