But again, you'd have that problem whether you used Homebrew or not, as soon as you tried to (even manually!) install the official macOS binary distribution of TeX, or XQuartz, or PostGIS, or...
Homebrew just acknowledges that these external third-party binary distributions (casks) are going to make a mess of your /usr/local — because that's the prefix they've all settled on burning in at compile-time — and so Homebrew tries to at least make that mess into a managed mess.
And, if some other system is already managing /usr/local, but isn't expecting the results of these programs unpacking into there, it's going to be very upset and confused — again, regardless of whether or not you use Homebrew. So it'd be better for those other systems to just... not do that.
/usr/local isn't supposed to be managed. It's supposed to be the install prefix that's controlled by the local machine admin, rather than by the domain admin. Homebrew just happens to be a tool for automating local-admin installs of stuff.
Homebrew just acknowledges that these external third-party binary distributions (casks) are going to make a mess of your /usr/local — because that's the prefix they've all settled on burning in at compile-time — and so Homebrew tries to at least make that mess into a managed mess.
And, if some other system is already managing /usr/local, but isn't expecting the results of these programs unpacking into there, it's going to be very upset and confused — again, regardless of whether or not you use Homebrew. So it'd be better for those other systems to just... not do that.
/usr/local isn't supposed to be managed. It's supposed to be the install prefix that's controlled by the local machine admin, rather than by the domain admin. Homebrew just happens to be a tool for automating local-admin installs of stuff.