Are you referring to the process of requirement engineering? Because although I agree its a fuzzy natural language interface, behind the interface should be (heavy should) a rigorously defined & designed system, where fuzzyness is eliminated. The LLMs need to work primarily with the rigorous definition, not the fuzzyness.
It depends on the use case. e.g. Music generation like Suno. How do you rigorously and logically check the output? Or an automated copy-writing service?
The tests should match the rigidity of the case. A mismatch in modality will lead to bad outcomes.
Aha! Like that. Yes that's interesting, the only other alternative would be manual classification of novel data, so extremely labour intensive. If an LLM is able to do the same classification automatically it opens up use cases that are otherwise indeed impossible.
