This is a great point. Not that far. We also snapshot the desktop for "slow" non-streaming updates to the UI. We could push these into Claude itself to act on or describe or whatever.

Hi all. This is an announcement post for a project I'm looking to get very early feedback on.

I've been using an AI coding assistant for a while (Cursor, Cline, etc.) and found that quite a few problems are caused by the model not having up to date or relevant examples of problems I'm working on.

So I created Kodit, an MCP server that aims to index your codebases and offer up relevant snippets to the assistant.

This works well when you're working with new projects, private codebases, or updated libraries.

I'm launching now to get as much feedback as I can, so do give it a try and let me know what you think!

- [Docs](https://docs.helixml.tech/kodit/) - [Repository](https://github.com/helixml/kodit/) - [Announcement post](https://blog.helix.ml/p/helix-kodit-open-source-mcp-server?r...)

For me, one of the most important parts of the spec is formulation of the DAG of tasks. Whether that be calling other LLMs or some retrieval mechanism.

What does everyone think about the pros/cons of a formal DAG specification vs using natural language? E.g. defining the DAG in yaml vs. something more natural language like writing logic and making calls in the prompt text?

Check out https://unify.ai/chat if you're interested in a router optimised for cost/ttft/performance for commercial language models.

Under the hood right now it's on-prem llama3 + a pretty basic RAG pipeline. The coolest thing about this technically is that it's all running totally privately.

But the main goal is to make websites more efficient. To get your customers to the answer they need faster.

Your sentiment is correct, but it's more of a spectrum. Fine tuning can learn facts (otherwise how would the foundation models learn facts?). But it needs those facts in the training dataset. If you have an infinite amount of facts, then you can memorise all of them.

The challenge arises when it becomes hard to generate that training data. If you just have the raw text and pop that in the context (i.e. RAG), then the LLM can be just as factual without any of that hassle.

Q2: identifiers in the prompt to say "you've been trained on this, only answer questions about this".

Q3: Depends on the size of the training data/docs. For the average PDF, about 30 minutes.

Give it a try!

> If you have an infinite amount of facts, then you can memorise all of them

pigeon-hole?

Not literally infinite, but Llama2 scale models can handle about 10 trillion tokens.