That's likely a change that needs to be made to either https://github.com/simonw/llm-gguf or https://github.com/simonw/llm-llama-server

... OK, I got the second one working!

  brew install llama.cpp
  llama-server --jinja -hf unsloth/gemma-3-4b-it-GGUF:Q4_K_XL

Then in another window:

  llm install llm-llama-server
  llm -m llama-server-tools -T llm_time 'what time is it?' --td

Wrote it up here: https://simonwillison.net/2025/May/28/llama-server-tools/

Simon, at this point you have a lot of LLM-related tools, and I am not sure which one is outdated, which one is the newest, fanciest stuff, which one is the one that one should use (and when), and so forth.

Is there a blog post / article that addresses this?

For my stuff it's basically just the latest https://llm.datasette.io and its collection of plugins.

If you're interested in what I recommend generally that changes a lot, but my most recent piece about that is here: https://simonwillison.net/2025/May/15/building-on-llms/

Great, this works here. I wonder, with extra-openai-models.yaml I was able to set the api_base and vision/audio: true. How do I do this with the llama-server-tools plugin? vision works, but llm refuses to attach audio because it thinks the model does not support audio /which it does).

EDIT: I think I just found what I want. There is no need for the plugin, extra-openai-models.yaml just needs "supports_tools: true" and "can_stream: false".