You can set a per-app language for Safari in macOS/iOS System Settings instead of using the system language

Oh yeah, you’re right. But that just gets you Chrome’s behaviour (the first preferred language has to equal the browser UI language).

So I suppose GGP is mostly right, in the sense that most browsers get this wrong (except Firefox).

Yeah it's still not ideal.

As someone who lives between 3 languages what I'd really like is a browser setting for language per-site. E.g. I want my Swedish bank's site in Swedish, not the English translation, I want Google Maps in Japanese so I can see the Kanji for the station names, but I want the AWS console in English. Each of these sites have their own toggles but they are very inconsistent and keep resetting, the browser would have done a much better job.

I wonder if this can be done with a browser extension.

Yes, and this is the actual issue: user agent configuration for preferred language is poor, users blame sites when they can't read the site, so it's in the site's best interest to ignore the broken thing and use a heuristic.

Yeah, I suppose you’re right about that.