By the very expansive definition that you advocate for, what exactly do we do that isn't math?
One could argue that all we do is turn thoughts, senses, and memories into further thoughts and appropriate actions, which is applying a pattern, which is math. But at that point the definition is too broad to be helpful for anything but playing word games.
There are no rules. On the other hand, code is very formal. It is a set of rules. It has structures. There is wrongness and you can do verification that is not the result of opinion nor is it subjective.
But yes, math is very broad. But it doesn't cover everything. At least not at this time.
One could argue that all we do is turn thoughts, senses, and memories into further thoughts and appropriate actions, which is applying a pattern, which is math. But at that point the definition is too broad to be helpful for anything but playing word games.