I have a... partial counter-argument to this. Publishers would absolutely jump at the chance to screw over individual authors with "unofficial" derivative works.
In fact, we know this to be the case because its how publishing used to work pre-Berne Convention. Imagine the mid-90s anime fansubbing scene, except everyone is a for-profit business who is able to obtain copyright over their translations of other people's work. It was incredibly scummy and hostile to artists in ways that would make Jeff Bezos[0] blush.
There's also a related problem of dividing "official" and "unofficial" works in such a way as to be legally airtight. "Noncommercial use" is actually really narrow, because the work itself is a commercial product and its production costs money. Think about how many fan artists have Patreon campaigns these days: sure, you don't have to pay to view their work, but they are still running the same kind of business that the original/official release is using. It's the sort of thing more appropriate for the original creators to just have a blanket "I don't care about people who do X, Y, or Z" policy/license rather than peeling back the legal protections on certain creative jobs in a way that, say, the scumbags at large media conglomerates might figure out how to monetize. Or, in other words, that fanartist with a Patreon is arguably protected from corporate competition by the fact that their work is illegal but practically unprosecuted in a way you can't put into law.
None of this would matter, of course, if copyright terms were limited to a short time period. All of the more onerous copyright rules I mentioned above were far more tolerable in a world where copyright on works people cared about regularly expired. If lawmakers had known that copyright terms would get blown out to life plus a tail period that is basically another life term, they probably would not have allowed derivative works to be controlled in this fashion.
[0] Owner of Amazon, the company that put your local bookstore out of business and then made it incredibly easy to resell other people's books on their platform
In fact, we know this to be the case because its how publishing used to work pre-Berne Convention. Imagine the mid-90s anime fansubbing scene, except everyone is a for-profit business who is able to obtain copyright over their translations of other people's work. It was incredibly scummy and hostile to artists in ways that would make Jeff Bezos[0] blush.
There's also a related problem of dividing "official" and "unofficial" works in such a way as to be legally airtight. "Noncommercial use" is actually really narrow, because the work itself is a commercial product and its production costs money. Think about how many fan artists have Patreon campaigns these days: sure, you don't have to pay to view their work, but they are still running the same kind of business that the original/official release is using. It's the sort of thing more appropriate for the original creators to just have a blanket "I don't care about people who do X, Y, or Z" policy/license rather than peeling back the legal protections on certain creative jobs in a way that, say, the scumbags at large media conglomerates might figure out how to monetize. Or, in other words, that fanartist with a Patreon is arguably protected from corporate competition by the fact that their work is illegal but practically unprosecuted in a way you can't put into law.
None of this would matter, of course, if copyright terms were limited to a short time period. All of the more onerous copyright rules I mentioned above were far more tolerable in a world where copyright on works people cared about regularly expired. If lawmakers had known that copyright terms would get blown out to life plus a tail period that is basically another life term, they probably would not have allowed derivative works to be controlled in this fashion.
[0] Owner of Amazon, the company that put your local bookstore out of business and then made it incredibly easy to resell other people's books on their platform