here's an awesome post on the landscape https://hamel.dev/blog/posts/prompt/

I remember reading that, good stuff.

I'd like to see an injectable mitm like proxy that can rewrite payloads. Many of these frameworks are useful, but when they go off the rails, they hard to modify and introspect.

It would be nice if LLMs had a way to speak an annotated format, like XML that was able to encode higher level information in a coherent manner over "well formed" addhoc text.

LLM libraries are in a crazy state right now. It is like JS frameworks 2015, a new one that demos well every other day.

one idea we're cooking is to offer a proxy with a hosted reformatting model on-board, to rewrite payloads on their way back in the case of type parse failure. fructose, the clientside sdk, would be optional