I am retired, so maybe I don’t get a say in this, but as a human who enjoys life in nature and hanging with family and friends, and also in periods of free time constantly exploring technology to understand the tools I use to the best of my abilities:
I find using minimal-capability local models or cheap commercial models like deepseek v4 flash to be the most satisfying because I am a major partner in solving problems or simply trying to better understand the world. I do like access to very strong models a few times a week.
A friend’s son and a young tech friend in town have very different views than I do because they are struggling in a tough job market and want a competitive advantage. I am grateful that I am not in that position.
Being able to set up a local tool chain that costs little more than the electricity to power your video card, I think, could be an important “know how to make a good dev environment” kind of skill once more companies start getting priced out of unsubsidized frontier model services.
But it’s sad that the base developer skill set is so devalued that learning something that would take people a few weeks to train you on is what gets your foot in the door.
This might be useful when ‘coding in chat mode’: I have a few scripts that I run in a project directory that takes a prompt from me, and creates a single long one-shot prompt that I can paste into a chat window and ask that any generating code is inside markdown code blocks for easier copy/pasting. Also, pardon the plug, but you can read my new tiny book free online that documents my experiences using agentic coding on my 16G Mac and my 32G Mac: https://leanpub.com/read/local-coding-agents
when I started using QAT recently, I stopped trying to improve my configuration after that. I will try tuning my local environment again in a few months, but with QAT things are good enough for now.
yes, whether for home experiments or at work, it is good practice (good hygiene) to be able to swap out both agentic harnesses and models. It is important to have a good strategy for exporting skills, etc.
I run something very similar except for directly using pi as the agentic harness I use little-coder that wraps pi with reasonable defaults for running local models. Even though my local setup is a bit slow, it is a thrill to do real work completely locally.
Thanks for your comment, I especially like “If you have software engineering skills and are not an accidental vibe-coder, honestly, try it out and stop burning money.”
I thought that using Opus with the Gemini Ultra subscription was in many ways awesome, but I simply feel happier using DeepSeek v4 flash with OpenCode (so fast!) of v4 pro when required.
I just asked Siri a few weather questions and named the city where I live, nailed it. My favorite digital device is my Apple Watch and if Siri improves over the next hear or two, that will be great for me.
I am on a wait list for the ‘better Siri model’ - what iOS and macOS betas just shipped with is awful. I do think the Apple Foundation Model built in to the system is better: I was using it from Python yesterday and it performs tool calling accurately and it is a very small model.
Thanks apfel looks useful! I have been experimenting with Apple's foundation models for almost a year and they are useful for embedded applications. I have been taking a deeper dive into local agentic coding tools (starting with 'little-coder --model ollama/gemma4:12b-it-qat') and I put together a tiny free book with some setup advice that might save people a few minutes of setup time: https://leanpub.com/read/local-coding-agents
I have been fairly much pissed off at the "hype in hyperscaler" AI growth (data center environmental and other societal costs) and I support anything we can do to promote local and private AI.
Sorry to be off topic, but I have a question: has anyone installed the latest beta iOS and macOS, and if so what is the current status of Gemini integration?
once you update to the latest iOS, there's an option to join the waitlist. No indication of timing, but until that moves forward Siri is what you get today, as far as I can tell.
I agree that many AI businesses will go bust and they deserve it, but the tech is good.
I can recommend my own layered approach, using the lowest capability models that get stuff done:
1. I maximally use local models like gemma4:26b-a4b-it-qat for everything that works with this free option.
2. I like paying for inexpensive APIs for mid-tier models like deepseek v4 flash, gcp-5-mini, gemini-2-flash for things that option 1. fails at. This option is almost free.
3. Pay for more expensive APIs like deepseek v4 pro, gemini 3.5 flash, etc. This option is not too expensive.
4. If all else fails on a class of tasks, then pay for awesomeness of Claude Opus. $$ expensive, I try not to use unless absolutely necessary.
I think developers and companies that just cram everything into Claude Opus are unprofessional.
LLMs we all agreed were amazing back in 2023-2024.
What's happening now with AI is more of a corporate phenomenon quite removed from the actual tech.
Yes LLMs are useful, but replacing customer support with an LLM that gives user accounts away, or calling LLMs on a loop where the bottleneck is your checkbook and calling it AGI, those are phenomenons that are separate from LLMs.
We agree more than you may believe. I have worked in the field of AI since the early 80s, symbolic AI, simpler neural networks. I only believe in using any tech if it serves human needs, is privacy preserving, etc.
I find using minimal-capability local models or cheap commercial models like deepseek v4 flash to be the most satisfying because I am a major partner in solving problems or simply trying to better understand the world. I do like access to very strong models a few times a week.
A friend’s son and a young tech friend in town have very different views than I do because they are struggling in a tough job market and want a competitive advantage. I am grateful that I am not in that position.
reply