In some places, tax collection was performed for an occupying power, e.g., the Roman empire. Hence, it was rightfully seen as "money taken away" instead of "funding public infrastructure".