This is a good idea that you could likely build out using just embeddings and performing semantic search over them. ChatGPT could be relegated to a role of translating intent into semantic searches and refining the output. You could even imagine fine-tuning an existing LLM to do this as well given a large enough corpus of examples.

The problem with this implementation is that it just blindly executes whatever ChatGPT says - that's quite scary.