A capability model fixes this: the application launches a file chooser which it does not control, which can then return it an fd or other token giving the application the capability to read from or write to a specific file.
Of course, this would be a considerable effort. But it's probably where we're going.
I've long argued for applications to use file choosers they do not control, but not for security reasons. I've wanted it for UI reasons.
Somewhere I've got a screenshot of a Linux system I had with half a dozen applications trying to open a file. Each used a different GUI framework or widget toolkit, each of which had its own file dialogs, and so the user gets presented with half a dozen different file choosing UIs. It was completely ridiculous.
Making the UI suck less was not a good enough reason to get any of the major GUI frameworks to support separating file choosing from apps. Maybe improving security will be a good enough reason.
Major GUI frameworks agree with you and have already adopted this approach and it is already used by many applications, and this has already been the case for years.
As mentioned above, Flatpak is already doing the two most popular toolkits. Programs just don't use toolkits to open files. So there is not a lot of "buck" here, and the tail is rather long.
Can't the sandbox override the corresponding syscalls? E.g. open() and readdir() which then block until the user has selected the file? This means you'll have to click the file twice (once for the sandbox and once for the application) but at least it is more secure.
Of course, this would be a considerable effort. But it's probably where we're going.