To be fair it's quite difficult to make out the words when listening to the song. I must have heard it dozens of times and somehow I got the idea that "brown sugar" must have been some American slang I just didn't know for some drug, because I couldn't tell the words apart.
Jagger was 26 years old in Alabama making light of almost every serious issue of the time. Thats really not that absurd or different than what “okurrs” now. A reference to Cardi B’s chart topping songs about personally being a scamming sex worker and general misandry. The people have always had the power to create consequences and simply did not on these topics, and now do so arbitrarily on some predictable topics.