This only works if you ignore intellectual property laws, which isn't gonna happen once AI leaves the "research lab" phase and becomes a commercial industry.
This comment confuses me a bit, as AI improving via feedback on its generations introduces no new concerns. Which new intellectual property concerns are introduced via reinforcement learning?
An aside, but people use and pay for Copilot, it is out of the research lab phase.