I believe we'll see two main types of AI/LLM libraries/frameworks emerge like has happened for ORMs (at least in Python). - The "Sqlalchemy" level: just bridge the gap between code and LLM - The "Django" level: additionally handle how data is indexed for RAG, standard patterns of connecting the pieces, possibly GUI, support for plugins.

It seems langchain and llamaindex are aiming to be the latter, and I'm building https://magentic.dev to be the former. I'd be interested to get your take on whether these abstractions would allow you stray from the narrow path while still being helpful!

I took a brief look (~5 minutes). My $0.02 is that it's not clear what problem you're trying to solve. I get what some of the features do (e.g., templated prompts) but it would be v helpful to have an example of how you actually use magentic, versus the non-magentic way. It feels like a lot of syntactic sugar, if I'm being honest (not a bad thing, but something you might want to be clear about, if that's the case.)

I would love some feedback on my library magentic which aims to have the best features of both approaches! https://github.com/jackmpcollins/magentic

If you are using Python, check out the package I've been building, magentic https://github.com/jackmpcollins/magentic It supports structured outputs and streaming, and aims to avoid making unnecessary abstractions (but might require some more understanding of LLM patterns as a result).

Also recently released is pydantic-ai, which is also based around pydantic / structured outputs, though works at level of "agents". https://github.com/pydantic/pydantic-ai

I'm building magentic for use cases like this!

https://github.com/jackmpcollins/magentic

It's based on pydantic and aims to make writing LLM queries as easy/compact as possible by using type annotations, including for structured outputs and streaming. If you use it please reach out!

I'd be really interested to hear what abstractions you would find useful for RAG. I'm building magentic which is focused on structured outputs and streaming, but also enables RAG [0], though currently has no specific abstractions for it.

[0] https://magentic.dev/examples/rag_github/

I've been building magentic [0] around similar ideas! Structured outputs and keeping control flow in python is definitely the way to go to keep things lightweight and easy to integrate. Having state management built in is cool! I'd be interested to hear what kinds of projects this has enabled you to build.

[0] https://github.com/jackmpcollins/magentic

Thanks for the kind words! I'm a fan of magentic :) One of the projects I've built with promptic is https://pdf-to-podcast.com

I'm working on a very abstraction light package for LLMs called magentic. The goal is to handle the main boilerplate of building with LLMs, but leave the abstractions up to the user. I'd be interested to hear your opinion on it vs others.

https://github.com/jackmpcollins/magentic

Interesting! How does it compare to something like Mirascope? I feel like I'm constantly on the hunt for the slimmest llm interface layer, and mirascope is my most recent attempt.

I found `ell` pretty appealing, but I still prefer using the raw openai/anthropic packages.

I'm building magentic, which aims to be a middle ground between the current "no framework" and "too much framework" options. It handles retries, logging/tracing, concurrency with asyncio, and other boilerplate, without mandating any prompts or particular patterns. Please check it out for the next comparison!

https://github.com/jackmpcollins/magentic

We'll check it out! Definitely interested in finding the right level of abstraction in these frameworks

Pydantic also have support for parsing partial JSON. https://docs.pydantic.dev/latest/concepts/json/#partial-json...

  from pydantic_core import from_json

  partial_json_data = '["aa", "bb", "c'  
  
  result = from_json(partial_json_data, allow_partial=True)
  print(result)  
  #> ['aa', 'bb']

You can also use their `jiter` package directly if you don't otherwise use pydantic. https://github.com/pydantic/jiter/tree/main/crates/jiter-pyt...

That's neat, I hadn't seen that. Docs were lacking so I submitted a PR: https://github.com/pydantic/jiter/pull/143

That gif is really cool! I built a Python package magentic [0] which similarly parses the LLM streamed output and allows it to be used before it is finished being generated. There are plenty of use cases / prompts that can be refactored into a "generate list, then generate for each item" pattern to take advantage of this speedup from concurrent generation.

[0] https://magentic.dev/streaming/#object-streaming